AI Software as a Medical Device: How to Tell and How to Report Problems

Some AI tools used in health and care do more than help with admin. If a piece of software has a medical purpose, it may be regulated as a medical device, and that changes what a provider should check and what staff should do when something goes wrong. This guide explains, using published MHRA material, how the Medicines and Healthcare products Regulatory Agency decides whether stand-alone software is a medical device, and how problems with software and AI can be reported. It covers Great Britain. For related governance points, see our guide to the CQC's principles on AI for care providers.

A note on the sources: the MHRA's stand-alone software guidance we reviewed is version 1.10f, and its latest revision entry is dated July 2023. The Yellow Card page on software and AI shows no date, and the MHRA's Software and AI as a Medical Device Change Programme roadmap shows an update date of 14 June 2023. The rules in this area have been under review, so check the MHRA website for the current version before you rely on any detail.

What counts as a medical device in software terms

The MHRA's guidance covers standalone software and apps, including in vitro diagnostic devices, under the UK Medical Devices Regulations 2002 (as amended). It is written for developers and for users. Its device determination flow charts ask a series of questions:

1. Is it software, meaning a computer program, or a functional document such as a spreadsheet with macros that needs separate software to run?
2. Does it have a medical purpose? If not, it is not a medical device.
3. If it does, which medical purpose is it? The guidance lists prevention, diagnosis, monitoring, treatment or alleviation, compensation, investigation or modification of anatomy or physiology, and control of conception.
4. Does it work with in vitro data? If so, specific purposes apply, such as information about a physiological or pathological state.
5. Is it an accessory that enables a device to fulfil its intended function? If so, it is treated as part of the device.
6. Does it drive or influence another device? Such software takes the classification of that device.

Functions that are not medical purposes

The guidance lists functions that are not medical purposes. They include monitoring fitness or wellbeing, patient or professional education, storing or transmitting data without change, providing reference information, and administrative tasks. In our view, this is why the intended purpose of an AI tool in a care setting matters: the MHRA lists administrative tasks and reference information as non-medical functions, but diagnosis and monitoring as medical purposes.

Intended purpose is set by the manufacturer

The MHRA says a medical purpose is determined by what the manufacturer states in the device's labelling, instructions for use and promotional material. Promotional material includes adverts, app store descriptions, landing pages and social media. The guidance says disclaimers such as "for information only" do not avoid regulation if medical claims are made elsewhere. It also says the manufacturer's own view is not decisive, because an objective observer may still judge a product to be a device.

Checking an AI tool before you use it

NHS England's information governance guidance on AI also says organisations should check whether an AI system is a medical device under MHRA requirements. The following questions are our practical suggestions, based on the MHRA's approach. They are not MHRA guidance:

• What does the supplier's own material say the tool is for?
• Does the tool claim to diagnose, monitor or advise on treatment, or does it only handle administration or reference information?
• Has the supplier told you whether it regards the product as a medical device, and can they show you how they reached that view?
• Who in your organisation signs off new tools, and have they seen the answers?

Our guide to HIQA's national guidance on AI in health and social care covers the position for Ireland, where different regulators apply.

Reporting problems with software and AI

The MHRA's Yellow Card scheme accepts reports about software, apps and AI that are regulated as medical devices. The Yellow Card page says:

• Anyone can report safety concerns, including members of the public, who can report as a guest user.
• Healthcare professionals in England, Wales and Northern Ireland should report adverse incidents to Yellow Card, following their organisation's medical device policies.
• Healthcare professionals in Scotland report to the Incident Reporting and Investigation Centre and to their local incident recording system.
• Manufacturers, suppliers and their representatives use the Manufacturer's Online Reporting Environment.

It lists the things worth reporting: adverse incidents that caused or nearly caused injury or affected someone's treatment or diagnosis, technical and quality problems such as false positive or false negative results, apps that do not work as stated, and devices found to be defective or unsafe.

Indirect harm

The Yellow Card page highlights indirect harm. This is where a person seeks or avoids further medical advice because of a tool's output, which it says is especially relevant to self-testing tools, symptom checkers and mental health assessment tools. The MHRA's change programme roadmap also lists clarifying reportable incidents, including indirect harm from decisions based on device outputs, as planned work. The roadmap says the safety signal the MHRA receives for software as a medical device is too weak.

The Yellow Card page advises reporting as soon as possible, keeping all references to the software you used, and also contacting the app's developer or owner.

Frequently asked questions

Is every AI tool in a care home a medical device?

No. The MHRA guidance says software without a medical purpose is not a medical device, and it lists administrative tasks and reference information among non-medical functions. What matters is the intended purpose set out by the manufacturer.

Who decides whether a tool is a medical device?

The guidance says the manufacturer sets the medical purpose through its labelling and promotion, but that the manufacturer's own view is not decisive.

Can care staff report a problem with a tool?

The Yellow Card page says anyone can report safety concerns about regulated software, apps and AI. Professionals should also follow their organisation's own medical device and incident policies.

Staff who select or supervise AI tools may find our CPD courses helpful for building their governance knowledge. This article is general information based on MHRA publications and is not legal advice.