The European Union's Artificial Intelligence Act entered into force in August 2024 and is now in its phased application period. For clinical institutions across the EU — whether they are deploying commercially procured AI tools, developing their own algorithmic systems, or customising foundation models for clinical use — the question is no longer whether the AI Act applies to them, but how it applies and what they must do to achieve compliance.
High-Risk Classification: What Falls Under the Act's Strictest Requirements
Annex III of the AI Act defines AI systems that are categorically classified as high-risk. Healthcare is directly addressed in Annex III, Point 5, which covers AI systems intended to be used as medical devices or for the management and operation of critical digital infrastructure for health. The practical scope extends to AI systems that inform or support clinical decisions with significant potential impact on individual patients — meaning diagnostic AI, prognostic algorithms, and treatment recommendation systems will almost universally meet the high-risk threshold.
High-risk classification triggers a substantial set of obligations. Providers — which in many cases means the developer or the institution that has customised a foundational model — must establish a quality management system, conduct conformity assessments, register in the EU AI Act database, prepare technical documentation, implement human oversight measures, and establish post-market monitoring systems.
Deployers — including hospitals and healthcare systems that are using but not developing AI tools — still carry obligations: they must ensure suitable measures for human oversight, monitor the system's performance, inform users of its limitations, and report serious incidents to national authorities.
Practical Steps for Clinical Institutions Now
Healthcare institutions should begin by conducting a comprehensive inventory of AI systems currently in use or procurement pipeline, and classifying each against the Act's risk categories. Systems that meet the high-risk threshold require immediate action: review existing contractual arrangements with providers to clarify accountability for compliance obligations; assess whether current governance structures include a nominated AI systems officer with defined responsibility for oversight; and establish a process for documenting AI system deployment decisions, including the rationale for deploying each system and the oversight measures in place.
