A new study led by researchers from the University of Cambridge has revealed that an artificial intelligence (AI) model can more accurately assess eye problems compared to specialistic doctors and provides advice.
Results published in the journal PLOS Digital Health showed that the GPT-4 model had a similar level of clinical knowledge and reasoning skills as specialist doctors.
Researchers tested the large language model against doctors at different stages in their careers, including unspecialised junior doctors, trainees and expert eye doctors.
After being presented with a series of 87 patient scenarios involving a specific eye problem, including extreme light sensitivity, decreased vision, lesions, itchy and painful eyes, the doctors were asked to give a diagnosis or advice on treatment by selecting from four options.
Results showed that GPT-4 scored significantly better in the test than unspecialised junior doctors and gained similar scores to trainee and expert eye doctors.
GPT-4 and GPT-3.5 are both trained on datasets that contain billions of words from articles, books and other internet sources.
GPT-4 uses ChatGPT to provide responses to human queries, which are capable of providing more accurate and empathetic messages than human doctors.
In addition, the study tested GPT-3.5, the Pathways Language Model 2 and the Large Language Model Meta AI 2 with the same set of questions.
Results showed that GPT-4, overall, gave more accurate responses when compared to them.
Researchers believe that the model could help to improve healthcare as part of the clinical workflow as it can provide eye-related advice, diagnosis and management suggestions in well-controlled contexts such as triaging patients or where access to specialist healthcare professionals is limited.
Further development is needed to help fine-tune and develop these models, which are currently being conducted around the world.
Dr Arun Thirunavukarasu, lead author of study, University of Cambridge, said: “With further development, large language models could also advise GPs who are struggling to get prompt advice from eye doctors.”
