AI as a resource in psychiatry: LLMs to identify signs of depression in posts

A group of researchers from Università della Svizzera italiana brought together in the new REMEDI Lab - REthinking MEntal health through Clinical and Data Intelligence, within the Euler Institute, is studying how Large Language Models (LLM) can analyse posts on social media to infer the mental health status of the individuals who authored them. Prof. Andrea Raballo, Full Professor at the Faculty of Biomedical Sciences, and Prof. Antonietta Mira, Full Professor at the Faculty of Economics, spoke about it on "Il Giardino di Albert" (RSI).

"It is a study that aims to evaluate how we can analyse the meaning behind the online traces we leave, such as those found in blogs. These blogs can serve as valuable tools for inferring an individual's mental health state in a specific direction," explained Professor Andrea Raballo. "In our case, we aimed to determine whether we could assess the presence of depressive elements in social media posts using a validated scale. The study demonstrated that an artificial intelligence (AI) algorithm can analyse the semantic traces a person leaves online to model their mental state effectively."

The feasibility demonstration was conducted by utilising the social media platform Reddit, which hosts millions of freely available posts. Artificial Intelligence was used to predict the mental health status of users based on the content of their posts. The surprising fact, as Professor Raballo commented, is that when the predictions were compared with the results of the Beck Depression Inventory, a clinical test used by specialists, it emerged that the AI's predictions were accurate, even though it did not know the results of the tests filled in by the users.

Moreover, as Professor Raballo explained, especially in today's media environment, the use of these methodologies can be an ally in the early diagnosis of depression: "It can allow early recognition of distress before it manifests itself in more clamorous forms. If we think instead of depression or self-harm, timeliness becomes even more important."

Statistical analysis of semantic content is made possible thanks to the considerable progress made by AI; as Professor Antonietta Mira explained: "The data we leave on social media are text strings. With the advancements in AI, particularly through large language models (LLMs), we can now treat texts similarly to numerical data. In technical terms, text strings are converted into vectors of numbers, allowing texts that convey similar concepts to be represented by vectors that are close to one another. This process enables the application of traditional statistical tools to text data as well."

The USI researchers, as part of the REMEDI Lab activities, have conducted several studies using different AI models, as illustrated by Professor Raballo: "The first study focuses on an AI model that, while less sophisticated than a large language model (LLM), still works on the principle of semantic affinity. This principle suggests that the more similar two concepts are, the closer their corresponding vectors will be to each other. The study confirmed that even with a relatively simple and cost-effective AI model, adequate training can enable the inference of a person's state of mind based on their activity on social networks."

Subsequently, a second study was conducted, based on the most innovative AI models, such as, for example, GPT and deep seek: "The second study aimed to retest the already demonstrated capability, which in this case turned out to be superior, as more complex models were used. In both cases, the results were transparent and interpretable."

However, despite the excellent results obtained during the studies, it is not so easy to envisage entrusting even the diagnosis to an AI, as explained by the USI professor: "There are certain aspects of governance, awareness and professional ethics that must be respected. AI at the moment is not a medical device in any way tested; it is rather a series of evolving ecosystems that can also have the function of supporting diagnosis and screening, but they must be used under the supervision of a professional and should never decide a patient's diagnosis alone."

Among users, however, there is an increasing tendency to use AI as a self-help tool, as Professor Raballo reminds us: "There are two main uses for AI in the realm of mental health support. The first is spontaneous self-help, which, in 2025, occurs through ChatGPT instead of traditional platforms like Google or self-help groups, offering a similar experience. The second use involves more advanced AI agents specifically trained to provide components or sub-modules of certain psychotherapies. In the past, attempts have been made to develop basic chatbots for motivational support. With today's tools, these systems are more effective and cost-effective, but they need to be validated through randomised trials to confirm their effectiveness."

In any case, the USI professor recognises great potential in the use of these models: "Since it is such a pervasive tool, it could have enormous potential if controlled and governed, especially in situations where there is no pervasive and ubiquitous health system, for example in developing countries. However, these are systems that must first be tested and analysed, and they must be transparent; otherwise, they become more unpredictable and uncontrollable."

Algorithm transparency is crucial for USI researchers, as explained by Professor Mira: "Our system is as transparent and as reproducible as possible; these are aspects that we value highly. Often, AI algorithms are black boxes: input data goes in, and output responses come out, but what happens inside, what binds certain inputs to certain outputs, is not very clear. An example of interpretability is in the classification of patients according to the severity of depression, which is done by mediating a complex problem, which is the diagnosis of depression, through the search, in social posts, of the symptoms underlying the pathology. The user is then classified based on how they would have responded to the questionnaires used to analyse depressed patients."

Interpreting this type of data presents complex challenges due to the inherent characteristics of the information being analysed. The data is unstructured, containing elements such as emoticons, irony, and self-deprecation, which are nuances that are difficult to capture accurately. One of the advantages of our system is that it is adaptive: among the various posts of a user, the algorithm searches for those most useful in answering a specific question from the validated questionnaire designed to diagnose depression. Existing methodologies in the literature define how many and which posts to consider in an automatic and predefined way; we do this in an adaptive way. If a user gives answers that very clearly indicate a specific type of mental state, we only need to analyse two or three of their posts to classify them with adequate accuracy; if, on the other hand, a user's posts are much more nuanced, we need to cross-reference several posts to determine the degree of severity of depression."

A further challenge in this kind of analysis is the need to distinguish quality data from so-called "junk data": "If junk data enters an AI algorithm, what comes out can only be junk answers, therefore of little value," explained the USI professor. Checking the quality of the data is fundamental. Still, it is difficult to do so on social posts since the basis is not a so-called "randomised" study that offers guarantees of non-bias and representativeness of the data analysed. The data are those available online, and those we used were collected thanks to a previous research project."

Despite the possible critical issues that may arise in using AI, Professor Mira encourages its dissemination in universities: "Initially, I was pretty sceptical, as I was accustomed to traditional statistical tools. However, with the insights from young researchers, I gradually became familiar with these newer tools. I realised that we can no longer ignore their significance; they are becoming increasingly powerful, and their potential is growing at an astonishing rate, leading to a real revolution in the field.

It is essential for universities to conduct in-depth research driven by scientific logic rather than commercial interests. This research should ensure that these tools are transparent and reproducible while also accounting for uncertainty in their predictions."

What prospects are there for a future synergy between psychiatry and AI? "The goal is to cultivate a sufficient level of creativity and curiosity in individuals to be able to use these tools creatively and to rethink some somewhat rigid aspects of the way we do therapy and diagnosis. In my opinion, this is an example of an attitude towards these tools that allows us to rethink them in a more anthropologically and ethically grounded key. I feel the same excitement as I did twenty years ago when we transitioned from conducting research on hard drives to utilising the internet; this represents a monumental shift that cannot be halted. The key is to guide this change consciously so that it can enhance the human ecosystem," Professor Raballo concluded.

The full interview with Professor Andrea Raballo and Professor Antonietta Mira, conducted by "Il Giardino di Albert", is available by clicking on the following link.