More accurate psychiatric diagnoses thanks to artificial intelligence: USI's REMEDI Lab publishes a pioneering study

The Università della Svizzera italiana (USI) marks a turning point in the integration of Artificial Intelligence and mental health. An interdisciplinary team consisting of Prof. Andrea Raballo, Full Professor at the Faculty of Biomedical Sciences and expert in psychiatry, Federico Ravenda, PhD student at the Faculty of Informatics, and Prof. Antonietta Mira, Full Professor at the Faculty of Economics and expert in statistics, has published the world's first study directly comparing the diagnostic capacity of the latest generation of language models (Large Language Models, LLMs) with that of a selected panel of internationally renowned psychiatrists.

Developed by the REMEDI Lab (REthinking MEntal health through Clinical and Data Intelligence) within the Euler Institute, the work represents a milestone in so-called 'augmented psychiatry', opening up practical scenarios for the use of Artificial Intelligence as a clinical support tool.

A direct exchange with experts

The study draws on previous international research (Urkin, Parnas, Raballo, Koren, 2024) that highlighted significant shortcomings in human diagnostic procedures. Only 33% of the 30 psychiatrists involved, from leading academic institutions in Europe, the United States and Asia, were able to correctly identify two genuine clinical cases of schizophrenia spectrum disorders presented in the form of text vignettes.

Latest-generation LLMs pass the test

Using the same clinical case studies, USI researchers tested the diagnostic capabilities of some of the most advanced LLMs available today (including GPT-4o, Claude-3.5, Gemini 1.5, Mixtral-8x22B, Llama-3, and Phi-3.5). Although these models were not specifically trained in psychiatry, the results were surprising: the best-performing LLMs matched or exceeded the accuracy of the top third of human clinicians, even identifying the most subtly expressed forms of schizophrenia.

A new frontier for mental health?

The results open the way to new applications of Artificial Intelligence in psychiatry, with important ethical and operational implications:

• Transparency and traceability: models offer explicit diagnostic reasoning, increasing reliability and replicability.
• Early diagnosis: awareness of subtle clinical signs can enable earlier treatment.
• Training and clinical support: LLMs can assist less experienced doctors, reducing variability in diagnosis.
• Appropriate care: more accurate decisions mean more targeted treatment and better allocation of resources.

Integration rather than replacement: towards “augmented psychiatry”

As emphasised by the team, LLMs are not intended to replace clinical judgement, but rather to enhance it. The study, which complies with the international TRIPOD-AI guidelines for the use of artificial intelligence in medicine, represents a practical step towards the ethical and responsible integration of these tools into everyday diagnosis and treatment.

‘This study marks a step forward in the development of augmented psychiatry, in which advanced language models become intelligent and transparent allies in clinical decision-making, improving diagnosis, treatment and quality of care,’ explained Federico Ravenda.

Link to the REMEDI Lab study: https://onlinelibrary.wiley.com/doi/10.1111/pcn.13864

Link to the preliminary human benchmark study (Leading International Psychiatrists): https://academic.oup.com/schizbullopen/article/5/1/sgae012/7663760?login=false

Link to the REMEDI Lab: https://www.euler.usi.ch/remedi-lab