There is a persistent mythology in the clinical AI sector that regulatory approval is an end state. A CE Mark, 510(k) clearance, or Health Canada authorisation is treated as a stamp of validated safety, after which the developer's primary accountability relationship is with the market rather than with the patients whose care will be shaped by the system.
This framing is not only epistemically incorrect — it is dangerous. AI systems do not operate in static environments. Patient populations shift. Clinical workflows evolve. Electronic health record systems are upgraded. A system validated in 2023 on a particular data distribution may perform very differently in 2026 on the data it actually encounters in clinical practice. Without structured post-market surveillance, there is no mechanism to detect this drift until it has already affected patient outcomes.
What Real-World Evidence Actually Requires
Effective post-market AI surveillance is not simply collecting complaints or tracking adverse events. It requires continuous monitoring of model input distributions to detect data drift before performance degrades. It requires ongoing subgroup performance monitoring to identify emerging demographic disparities that were not present at the time of initial validation. It requires structured clinician feedback channels that enable systematic capture of cases where AI recommendations were incorrect, even when no formal adverse event occurred.
A sepsis prediction algorithm may miss the early warning it is designed to detect in 12% of cases. If none of those missed cases generate formal adverse event reports — because clinicians or nurses catch the deterioration through other means — the surveillance system will show a clean record while the algorithm quietly fails a meaningful proportion of patients.
The GHAI Foundation's post-market surveillance framework draws on methods developed in pharmacovigilance — specifically the use of statistical process control charts to monitor model performance metrics continuously, with pre-specified alert thresholds that trigger mandatory review. We are working to establish a cross-institutional registry where de-identified performance data from deployed clinical AI systems can be aggregated, enabling the sector to learn collectively from real-world evidence rather than each institution navigating AI surveillance in isolation.
