Artificial intelligence passes radiologists’ specialist exams, just about!

28 Dec 2022, 9 a.m.

X-ray image of hand

Radiologists are specialist doctors that use scans to diagnose conditions, from broken bones to cancers.

In the UK they must pass a specialist exam as part of their training. For the first time, researchers have shown that an artificial intelligence (AI) tool could pass this exam too.

Can artificial intelligence pass the exam?

Radiologists are specialist doctors that use scans to diagnose conditions, from broken bones to cancers. Radiologists in the UK must pass the “Fellowship of the Royal College of Radiologists” (FRCR) exam as part of their training. In one part of this exam, the trainee radiologists are asked to interpret 30 X-rays within 35 minutes and need to get >90% correct to pass.

A publication in the BMJ Christmas issue, led by Dr Susan Shelmerdine at GOSH and Dr Jonathan Weir-McCall at University of Cambridge, shows for the first time, that an artificial intelligence (AI) tool could pass this exam too.

Training artificial intelligence

The commercially available AI tool had been trained on datasets of almost 1 million chest and bone X-rays and taught to identify seven key problems including broken bones and abnormalities in the lungs.

10 challenging mock rapid-reporting examinations were evaluated by the AI and by 26 newly-qualified radiologists. The AI was able to pass half the number of mock examinations that the radiologists could pass, when only the images the AI had been trained on were included.

The AI tool also correctly identified 90.5% of all the X-rays that most radiologists also got correct. When the AI was incorrect, it was mostly (64.2%) flagging an abnormality that wasn't there, a false-positive, rather than missing a problem.

There were around 10 X-rays where the AI identified abnormalities that most trained radiologists missed. These were often in images of the extremities, such has the hands and feet, where there are many small bones to assess. This suggests AI could be particularly useful to look at scans which are more time-consuming to evaluate.

This is the first time an AI tool has gone head-to-head with humans on a radiology exam like this. Although radiologists came out on top, the AI got good grades too, even though it wasn’t built with the intention of passing exams!

AI tools like this are becoming more advanced each year, and we hope this study can help people understand how AI can support clinical work already being carried out by doctors to provide better care for patients. AI won’t replace doctors anytime soon, but there’s real potential for AI support, particularly where specialised radiology services are limited.

Dr Susan Shelmerdine, consultant radiologist and lead author at GOSH, and honorary associate professor at UCL GOS Institute of Child Health

First patient treated with groundbreaking gene therapy trial

A baby boy born with a rare condition has become the first in the world to be treated with a new, potentially lifechanging, investigational gene therapy on a clinical trial at Great Ormond Street Hospital (GOSH).

Professor Darren Hargrave appointed NIHR Senior Investigator

We’re delighted to announce that Professor Darren Hargrave has been named as a NIHR Senior Investigator in this year’s prestigious awards list.

Reflecting on rare disease day 2025

Rare Disease Day is an international event which takes place annually on the last day of February. The date is chosen because in leap years it is February 29, the rarest date!

First-of-its-kind study will assess psychological impact of sudden cardiac death on children

Great Ormond Street Hospital (GOSH) and University College London (UCL) will lead a pioneering study with the hope of revealing more about how best to help children who have experienced an unexpected and often traumatic loss in the family.