Over the past five years, the healthcare sector has adopted artificial intelligence at an unprecedented rate. Diagnostic imaging algorithms, sepsis early-warning systems, predictive readmission models, and medication safety tools have moved from research papers to clinical floors. Yet the frameworks governing how these systems are tested, validated, and monitored in real-world environments have not kept pace.
The result is a fragmented landscape where developers apply self-certification processes that vary dramatically in rigor, where regulators lack the technical capacity to scrutinize algorithmic outputs, and where patients and clinicians bear the downstream risk of systems that were never stress-tested against the full complexity of real clinical environments.
The Limitations of Developer Self-Certification
Most AI medical device submissions to regulatory agencies like the FDA or CE Mark bodies still rely primarily on performance evidence generated by the developer itself. While this evidence is often technically rigorous — encompassing held-out test sets, sensitivity and specificity metrics, and ROC curve analyses — it systematically underrepresents the conditions under which these tools will actually be used.
A retrospective chest X-ray algorithm validated on images from three academic medical centres in Massachusetts will not automatically generalise to community hospitals in rural Ghana, where equipment calibration varies, patient demographics differ significantly, and the pre-test probability of disease shifts. Self-certification cannot surface these gaps because developers rarely have access to the breadth of real-world data required to expose them.
An algorithm that performs with 94% accuracy in a curated research dataset may perform at 71% in a community clinical setting — a gap that self-certification would never reveal, but an independent audit is specifically designed to surface.
What Independent Auditing Looks Like in Practice
Effective independent auditing of clinical AI is not simply re-running the same benchmark tests. It involves prospective stress-testing under distribution shift conditions, systematic subgroup analyses across protected characteristics including race, age, sex, and socioeconomic status, and red-teaming exercises designed to identify adversarial failure modes.
Independent auditors should also have the authority to examine training data provenance, assess data governance practices, and evaluate whether consent frameworks cover downstream algorithmic use. In financial services, credit model audits routinely include full data lineage reviews. There is no principled reason why the clinical AI sector should hold itself to a lower standard, given that the stakes — patient safety and equitable care delivery — are at least as high.
Building the Infrastructure for Audit at Scale
The GHAI Foundation has spent the past two years developing an open, modular audit framework designed to be applicable across healthcare settings in both high-income and lower-middle-income countries. The framework defines four tiers of audit intensity, from a rapid screen for low-risk decision-support tools to a comprehensive multi-site prospective study for diagnostic tools that inform high-stakes clinical decisions.
We are working with academic partners in twelve countries to build a federated dataset infrastructure that allows audit institutions to test algorithms against locally representative data without requiring that sensitive patient data leave national jurisdictions. This approach mirrors the federated learning paradigm that has gained traction in AI development, applied instead to verification and accountability.
The path to safe, equitable clinical AI is not solely a technical challenge — it is an institutional one. Mandatory pre-deployment independent audits, backed by regulatory enforcement and supported by internationally shared infrastructure, are a prerequisite, not an optional enhancement. We call on regulatory bodies, healthcare systems, and AI developers to make this the new minimum standard.
