The ioLite Index
Measured first.
Published second.
The ioLite Index ranks AI systems on how they behave when a conversation turns dangerous: a disclosure of crisis, a slide into dependence, a request for harmful advice. Our first study is measured; it publishes once the clinician review is back. A score will appear on this page only when the transcripts behind it exist.
Study 01
Our first measurements
Measured · clinician review in progressStudy 01 measures widely used AI chat systems against the priority constructs of our taxonomy: crisis and suicide-risk management, iatrogenic AI traits, over-use and dependence, and the other constructs we treat as safety gates. Any system a person can talk to is measured the same way — open-weight models, commercial APIs, and deployed products — and every score is published with the transcripts behind it.
When it is published, the study will include
- —The exact model names and versions tested
- —Scores for each construct, with the number of probes behind every score
- —Representative transcript excerpts, both failures and protective responses
- —The scoring scale, and the rate at which clinician review agreed with our scores
69
Clinical sub-constructs
9
Safety-gate constructs
4
Source frameworks
0–4
Graded, not pass/fail
The taxonomy is synthesized from validated clinical and AI-evaluation frameworks; sources and definitions are shared under NDA.
Same conversation, three runs
One run is a snapshot. We report the variance.
Every persona in Study 01 ran three times on randomized schedules against each setup. The point of the third run is to know how much a single score can be trusted.
55.8%
of conversations got the same grade on all three runs
17.1%
swung by three or more grade points between runs
50.8%
of serious failures did not show up on every run
97.3%
same grade when the same conversation was graded twice
Early results, 1,537 conversations run three times and 258 graded twice. Per-model scores publish with their transcripts after the clinician review.
How the benchmark is built to be trusted →Method
How a system is scored
Every score comes from the same pipeline, and every step leaves a record that can be checked.
Scenario
A reviewed, multi-turn scenario written against one construct. The same scenarios are re-run on every model version.
Run
The system under test is taken through the scenario turn by turn. The full transcript is kept.
Score
Each transcript is scored from 0 to 4 against the construct it targets. A score of 3 or above passes, and every score comes with a written rationale.
Report
Results roll up by construct. A failure on a safety-gate construct is reported on its own and is never averaged away.
We will not publish a score without the transcripts that produced it, and we will not name a system in a ranking until it has been evaluated under this method.
Why This Matters
Capability is not
the same as safety.
These are the most capable AI systems ever built. They pass bar exams. They write production code. They synthesize decades of medical literature in seconds. In a different context — that would be enough.
In an emotional one, it never will be.
Capability and safety are different properties. A system can answer every question correctly and still cause irreversible harm when someone in crisis reaches out. Intelligence does not prevent that. Only evaluation does.
Frontier systems are benchmarked extensively before they are deployed, for reasoning, coding and instruction following. Very few are evaluated in a structured way for what they do when the conversation turns dangerous.
That is the question the ioLite Index exists to ask.
The assumption
"If a model is capable, it is safe enough." This assumption has never been tested against psychological risk scenarios. It has simply been made.
The truth
Safety in emotionally sensitive contexts is not a byproduct of intelligence. It requires separate, structured evaluation — every time, against every system.
The consequence
People are already talking to AI systems that have never been evaluated for what they do in a crisis. Those conversations are happening. The evaluation is not.
The standard
ioLite Labs exists to close this gap. Not as a feature. As an infrastructure. The evaluation layer that AI deployment is missing.
The question is not whether your AI system is capable.
The question is whether it has ever been evaluated for what happens when it fails someone.
The list
Get Study 01 the day it publishes.
One email per release, with a button that opens the full report. Unsubscribe with one click.
Next Step
You don't have to wait for the study to know how your own system behaves. We can evaluate it under the same method.