Existing evaluations for therapeutic quality are inadequate

Sept 21, 2026

With tens of millions of people turning to LLMs for emotional support every week, ChatGPT is the largest single source of mental health support in the US. Despite this, we have a poor understanding of LLMs’ capabilities and weaknesses in mental health. In particular, we believe evaluations are critical for improving the therapeutic quality of AI models while existing evals unfortunately fall short of measuring what matters. As things stand, we are entering the biggest shift in history of how people receive mental healthcare without a clear picture of its consequences nor robust plans to make them better.

The rise of AI for mental health

There are deep divisions within the American public about whether AI becoming the go-to solution for an increasingly lonely, depressed, and anxious population is good or bad. Some cite the devastating cases where ChatGPT encouraged users to end their lives while others cite the unmet mental health needs in this country and the potential of AI to expand access to care. Some also shift from one side to the other, like this post on r/TherapyGPT:

There is both great opportunity and danger with “AI therapy.” We are worried about what happens when people get used to 24/7 empathetic-sounding AIs, the displacement of human relationships with AI relationships, and human therapy becoming a luxury good only available to the wealthy. At the same time, there are a billion people with a mental disorder, and only 7% of this population receives adequate treatment. We believe in a world where AI can help solve the access issue in mental healthcare, unlock a new scientific understanding of why therapy works, triage care to maximize human expertise, and enable every single person to be a healthy, flourishing human with a rich (human) interpersonal landscape.

To move towards such a world, it’s critical to begin with a precise understanding of AI’s performance. This is typically done through an “eval” — a structured test to quantify how well AI performs a particular task. However, besides measuring current capabilities, evals also lay the groundwork to actually making models better. For example, consider how quickly AI has gone from unreliable code suggestions to autonomously writing over 75% of new code at Google and Anthropic. In October 2023, Jimenez et al. released SWE-bench which measured whether LLMs could resolve bugs in real-world repositories. Five months later, this led Cognition to identify a better way to scaffold LLMs to solve coding tasks, and since then, every AI lab has used evals based on SWE-bench to iteratively measure and improve their models. As in any scientific field, progress depends on reliable ways to test whether new approaches produce meaningful improvements.

What makes for effective evals

Given that evals are an essential part of AI infrastructure, we propose two key desiderata for an effective eval:

1. Ecological validity: does the AI act in an environment similar to the real-world?
2. Predictive validity: does better eval performance translate to better real-world performance?

We can use these two axes to understand why SWE-bench far surpassed its predecessor HumanEval on the question of “can AI write useful code for engineers?”.

For context, HumanEval required language models to write one-off Python functions to pass unit tests.

In contrast, SWE-bench required models to understand real user bugs, navigate a large open-source codebase, and submit coding patches that passed the appropriate tests

From an ecological validity perspective, the experiment almost perfectly matched what software engineers do day to day, and from a predictive validity perspective, the unit tests were carefully selected to only pass when the reported bug was actually fixed. Because of these qualities, SWE-bench better captured what actually mattered towards engineering.

Using this lens, we can understand why certain evals have become popular. As terminal-based coding agents became more common, Terminal-Bench quantified how well models could complete tasks in the terminal (ecological validity). As LLMs were getting integrated into all parts of the economy, GDPval focused on real tasks across 44 occupations (ecological validity), and each task was created and vetted by professionals with an average of 14 years of experience (predictive validity).

Unfortunately, reaching an acceptable bar of ecological and predictive validity for AI therapy is very challenging. Many have begun to tackle this problem, and we plan to spend some time talking about their work as well as the core scientific problems that make evals difficult. Specifically, two features stand out to us: 1) expert therapist judgments are unreliable, and 2) therapy unfolds over long time horizons.

Expert therapist judgments are unreliable

Like creative writing or research taste, therapy is a non-verifiable domain. Unlike math and coding, there are no programmatic methods for evaluating correctness. There is no equivalent to “did the written code pass a set of tests?” or “did you get the correct answer to a math problem?”

To address non-verifiable domains in machine learning, we typically use reinforcement learning from human feedback (RLHF). In RLHF for language models, we aggregate human preferences by asking “do you prefer response A or response B?”, train a reward model to mimic these human preferences, use this reward model as an evaluation, and then train a language model to perform better per this reward model (Ouyang et al., 2022). This method is highly effective and is why language models can follow instructions (Ouyang et al., 2022) and be helpful conversational assistants (Bai et al., 2022).

Implicit in RLHF is an assumption that humans can judge the quality of a response. Unfortunately, in therapy, expert judgments are disproportionately unreliable because effective therapists often view each others’ work as bad therapy. For example, therapists are typically trained in a single theoretical orientation and use that to inform their practice. Given the same client struggling with feeling rejected by their partner who takes a while to respond, a psychodynamic therapist may explore the childhood roots of this fear, an interpersonal (IPT) therapist may help them express their needs, and a cognitive behavioral (CBT) therapist may dig into the belief that a delayed response is rejection. Each would have strong opinions on the right way to treat the client but all could be equally good at bringing the client to health! Witteman and Koele (1999) found that therapists often disagreed about treatment plans across theoretical orientations, and Persons and Bertagnolli (1999) found similar disagreements even within theoretical orientation about case formulations.

These findings put into question whether evaluation approaches that rely on expert judgment can be easily used for therapy. We might see that we cannot get therapists to agree, or when they do agree, they agree on topics that do not have meaningful consequences (e.g., whether a question was open-ended or closed-ended but not on whether it was effective). Regardless, judgment-based evaluations will need to prove predictive validity — that performing better on those evals actually translates into better healing for the client.

We can see this problem rear its head in the MindEval benchmark (Pombal et al., 2025). In this evaluation, an AI patient interacts with an AI therapist for 20 messages and an AI judge rates the conversation using a rubric on a number of axes (e.g. clinical competence, ethical conduct, communication quality). These rubrics were thoughtfully constructed based on existing clinical frameworks, and we show an example below.

However, using four different therapists to evaluate the conversations, Pombal et al. observed low agreement on the quality of a response based on the rubric:

We focus on the yellow items that show MIPSA scores across therapists in the range [0.53, 0.76]. MIPSA is a measure of the percentage of pairwise comparisons that match between two annotators.

Random chance would yield a 0.5 MIPSA, and the therapists perform only marginally above chance with a median 0.58 MIPSA. Thus, despite the eval aiming to evaluate “safe, effective therapeutic care,” performing better on the rubric does not translate to “doing better therapy.” Instead of capturing objective therapeutic skill, it captures subjective preferences.

Rollwage et al. (2026) highlights the other side of the expert judgment problem. They evaluate their AI system with the Cognitive Therapy Rating Scale (CTRS), a CBT-specific competence measure that asks how effectively a therapist implements CBT. While CTRS exhibits high therapist agreement (Vallis, Shaw, and Dobson, 1986), meta-analyses have consistently shown no or small correlations between CTRS and client outcomes (Webb, DeRubeis, and Barber, 2014; Power et al., 2022). Yet again, we lack predictive validity. Even though therapists agree on this rubric, performing better on it does not actually move anything for clients.

Thus, so far, we have yet to see an eval where expert judgments are reliable — where they are both high in agreement and predictively valid.

Therapy unfolds over long time horizons

On top of the lack of clear expert judgment, we must also contend with the fact that therapy unfolds over long time horizons. Therapy occurs over many hours of conversation, and what the therapist says at any given moment is thus a tiny part of the overall success or failure of the therapy. It would be incredibly hard to claim that “a specific moment” makes or breaks the broader therapy.

Zhang et al.’s (2025) CBT-Bench Level III evaluation tests how AI responds to a single user prompt, and each prompt is part of a category (e.g. Establishing Goals, Working with Cognitions) with a specific rubric for that category. Here is one example:

Their ideal response in this case is “I’d like to hear more details about that. If relationship issues are most concerning to you, we can focus our work on that area. In our individual work together, we’ll need to focus on things that you can change. Part of our session today can be brainstorming what some of those things might be.”

But this rubric does not account for the fact that therapy does not happen in a single turn! A therapist might intend to satisfy all four criteria but do so over a few turns, and thus, the response “tell me more” would suffice in the therapy room but utterly fail this rubric.

Many evals try to sidestep this problem by designing AI patients, but doing so comes with its own flaws. Chiu et al. (2024) created BOLT, an eval where AI therapists talk to AI patients and are evaluated according to whether they use behaviors such as “reflect on emotions” at similar frequencies to effective human therapists. However, AI patients do not resemble the diversity of real patients. Wang et al. (2024) highlighted that their AI patients were unrealistically reflective, and MindEval identified that “LLMs often embodied a caricature of the profile, rather than a realistic human being.” Looking at the example conversations from the BOLT paper, there are many instances where clients are highly agreeable and very highly self-aware:

Can you imagine saying this as a patient or hearing this as a therapist? Pretty unlikely! AI often tends towards a compliant and psychologically simple patient whereas real humans are highly complex, confusing, and inarticulate beings. Evals relying on heavy use of AI patients tend to lack realism; because AI patients behave very differently from human patients, the therapy quickly collapses into something unrecognizable. This presents a serious ecological validity problem because we can no longer feel confident that the way that AI therapists behave in the evaluation mimics the way they will behave in real-world settings with real humans.

That said, there is burgeoning research on how to design more effective AI at simulating humans. Transluce has designed a multi-agent architecture to mimic humans in mental health scenarios, while Slingshot AI and Humans& have both adversarially trained LLMs to mimic real-world human conversations. We’re excited to see these improvements and may start to address the difficulty of managing the long horizon of therapy.

Closing remarks

The path towards effective AI for mental health must go through rigorous evaluations. While reaching ecological and predictive validity is challenging, we believe it is one of the most important problems in this domain. We think thoughtful collaboration across the field of psychotherapy research and machine learning is required to grapple with this problem, and we are excited to be bridging this gap and tackling such an inspiring scientific, technical, and societal problem.