Artificial intelligence: who’s liable?
Author: Dr Helen Hartley, Head of Underwriting Policy at Medical Protection
Advancements in medical technology can bring huge benefits for patients and clinicians alike – but new approaches can also mean new risks. Dr Helen Hartley, Head of Underwriting Policy at Medical Protection, looks at where the liability lies for artificial intelligence.
For many, the concept of artificial intelligence conjures images from the darkest recesses of Hollywood imagination: robots running amok and rogue algorithms instigating World War 3.
In medicine, however, its benefits are impossible to ignore – only recently a study in Nature Medicine journal reported on an algorithm that can learn to read complex eye scans.1 When tested, it performed as well as two of the world’s leading retina specialists and did not miss a single urgent case.
But what has not been proven is the infallibility of artificial intelligence (AI). When a mistake does occur, where does the liability lie?
Robots in the dock
Clinicians should ensure any robot or algorithm is used as part of – not in place of – sound clinical judgement and proficiency. Algorithms, including those used by triaging apps, should not be blindly followed without regard to a patient’s particular clinical features or circumstances, such as geographical location, which may impact on the probability of certain diagnoses.
It is expected that the creators and/or producers of these will seek independent advice regarding their indemnity requirements, which may include the potential for multiple serial claims to arise from errors or service disruption affecting an AI product. Similarly, with regard to the use of any surgical equipment, product liability would apply in relation to robot malfunction, whether hardware or software.
In order to minimise the risk of malfunction or errors, any clinician intending to rely on AI equipment should ensure they are satisfied that it is in good working order, that it has been maintained and serviced according to the manufacturer’s instructions, and that product liability indemnity arrangements are in place.
Clinicians should also:
- Obtain and document having taken informed consent from their patients including, where relevant, the benefits and risks of using AI equipment and of other available treatment options.
- Adhere to any local checklists before ‘on the day’ use.
- Only use equipment on which they have received adequate training and instruction.
- Consider the possibility of equipment malfunction, including whether they have the skills to proceed with the procedure regardless, and ensure the potential availability of any additional equipment or resources required in that event.
- Verify the patient’s identity and location when undertaking any form of virtual consultation.
- Consider the safety of the proceeding with a virtual consultation if it would not be easy for them to arrange to see the patient in person for an examination if that was indicated.
- Ensure adequate documentation of the online consultation and consider the need to communicate – with appropriate consent – with any other relevant healthcare professionals involved in looking after the patient.
A GP contacted Medical Protection with concerns over the provision of an online service, commissioned from a private company. The service was to take the form of a practice weblink that takes patients through an algorithm: the conclusion being that the patient may be directed to an ambulance, the community pharmacy or the GP.
The practice had two concerns:
The practice set up procedures to deal with requests that are received in office hours but were concerned about vulnerabilities in relation to the above.
While symptom-checking algorithms are becoming increasingly sophisticated, it remains possible that subtle physical signs may be missed if no clinical examination takes place. To mitigate that risk, doctors should have a low threshold for recommending that the patient attends a local clinician for a face-to-face appointment so they may undergo detailed clinical assessment.
With regard to the second concern, the GP was advised to explore what options were available within their practice to review patient contacts via the online service on Saturdays and Sundays to see whether the risk could be managed within available resources.
If that was not possible, it was recommended that they approach their commissioning group to discuss their concerns and learn whether in commissioning the service, they had already considered that possibility and the need for a solution to be implemented covering multiple practices in the locality, allowing review in the specified timeframe of patients directed to seek GP input.
If no such plan was under consideration or accepted as necessary, given that patient safety is paramount, the GP was advised to follow GMC guidance in relation to raising patient safety concerns and to adhere to local procedures for doing so.
- De Fauw J, et al. ‘Clinically applicable deep learning for diagnosis and referral in retinal disease’, Nature Medicine 24, pages1342–1350 (2018); DOI:10.1038/s41591