Real-time context lifted RCA accuracy up to 50% and reduced token cost to 1/10th

When AI agents investigate on-call incidents, they spend most of their time building context from raw telemetry and stitching it together. They work on top of an observability store and its query stack, and query it to build complete context, doing joins across multiple streams of telemetry, each with its own granularity and cardinality challenges.

Chiron’s approach is different: it gives the agents pre-computed context data. This is possible because of its streaming engine, which continuously computes the up-to-date system state from telemetry as it arrives.

We tried giving Chiron-built context data (instead of raw telemetry from various backends) to several frontier models, and tested their accuracy and token usage on various benchmarks. The results have been very encouraging, and this article discusses them.

Background : Context built in flight

Chiron facilitates AI-driven investigations by correlating telemetry in-flight, in a streaming fashion, and creating lightweight, correlated context across multiple telemetry streams. This provides AI agents with pre-stitched context, including symptom-to-cause correlations, that would otherwise be expensive for agents to derive from raw telemetry themselves.

This pattern held across different models and even across different types of agents, from LLM-based agents to decision models with no LLM at all. This demonstrates that you don't necessarily need the most expensive model to get better results. With the right context, you can get significantly more out of the model you're already using and breaks the general narrative that for better AI SRE performance, one needs to use better and advanced models. With better context data , the result is lower inference cost and higher accuracy at the same time, from the model you already use.

Encouraging results on various benchmarks

The claims of enhancing RCA quality of AI agents are tested upon reputed benchmarks.

SREGym benchmark

SREGym is an open benchmark for AI SRE agents, built at UIUC. It breaks real application on a live Kubernetes cluster, with faults ranging from bad configs and runtime changes disrupting vanilla flow and checks that the agent names the root cause and fixes it, in the same run.

For SREGym we built read-only MCP tools on top of context state APIs. Each tool reads the live state and does not query raw telemetry.

The brief is formed so that it leads to verification. Each lead says why it is linked to the failure, so the agent can check the claim instead of blindly trusting it and to ensure no hallucinated direction is given. Those links come from the correlated context: because symptoms, workloads and changes were already joined in flight.

Here is the opening of a real brief, from a run where a rogue admission webhook rewrote a service's memory limit to 16Mi from 256MB causing pod scheduling failures:

SYMPTOMS (new since the first change):
- nginx-thrift (1 pod): running memory limit 16Mi, pod template sets 256Mi

LEADS (changes near the symptoms; verify each, these are not conclusions):
- [can rewrite pods at creation, and reaches failing nginx-thrift]
  MutatingWebhookConfiguration gatekeeper-mutating-webhook-configuration:
  objectSelector set to {} (+9 more fields) at 06:42:25Z
- [time only] MutatingWebhookConfiguration kyverno-resource-mutating-webhook-cfg ...
- [time only] MutatingWebhookConfiguration cert-manager-webhook ...

(newest data 1 s old. Not visible here: secret values, process memory, request traces)

The brief tells the agent three things:

• What's wrong. The nginx-thrift service is running with a 16Mi memory limit, even though its settings ask for 256Mi. Something changed it as the pod started.
• What might have caused it. Five webhooks changed in the same second. Only one of them can change pods as they start, and it applies to nginx-thrift, so Chiron lists it first and says why. The others are marked “time only”: they changed at the same moment, but nothing connects them to the failure.
• What it can't see. The last line says where Chiron's view ends, so the agent knows what to check for itself.

Claude Code with Chiron and Claude Opus 5.5 (medium effort) diagnosed and fixed every SREGym-Lite fault, in every attempt: 100% diagnosis, mitigation and end to end, at 277K tokens per run, the lowest token count of any entry on the leaderboard by 2x. Published on the SREGym leaderboard.

Not just LLMs

We also verified that the correlated context should help whatever language model processes it. On SREGym we measured the performance of the Jev decision model also with and without Chiron and with no LLM in the loop.

Jev, from TypeSafe, diagnoses by choosing which checks to run and weighing what they return. We ran Jev alone and Jev with context data on the same setup, over 21 faults.

An LLM agent and a decision model work in completely different ways, and share nothing but the context they were given. With Chiron, both did less discovery and got more right. That is the point of a context layer: it improves the consumer you already have irrespective of its capability.

ORCA-bench

ORCA-bench is used for testing incident investigation on a running e-commerce OpenTelemetry Astronomy Shop app, with about 20 services under load where the real faults are injected through feature flags. The agent gets a real SRE person or a customer report such as “customers are having trouble buying” and then it must find every root cause explaining how it happened and cite the evidence. An LLM judge scores the report.

In the paper, agents investigate through Prometheus, Jaeger and OpenSearch behind Grafana. In our run, Claude Sonnet 4.6 read correlated state instead, over 526 public incident tickets and 138 quiet-window controls which mean no fault injected.

The context data increases accuracy by 50% compared to sonnet with grafana with 10x less token usage.

The full setup, including the benchmark knowledge the context was tuned with, is in our ORCA-bench write-up.

What this means for AI SRE

Most of what an AI SRE agent spends goes into finding out what is going on: querying telemetry, reading raw output and carrying it forward turn after turn. Chiron does that work once, in flight, and hands the agent a small correlated brief instead. The agent starts from evidence rather than a blank page.

Across two benchmarks, that changed three things at once:

• Lower inference cost. Fewer discovery steps means fewer tokens re-read on every turn: 154K tokens per incident on ORCA-bench against 1.43M, and 277K per run on SREGym, the lowest on the board.
• Higher accuracy. With the right symptoms and the linked change in front of it, the agent names the real cause more often: 46.2% against 30.9% on ORCA-bench, 100% on SREGym-Lite with RCA accuracy improvement in both Opus and Jev.
• Less room to hallucinate. Every lead comes from correlated telemetry and says why it is linked, so the agent checks claims against evidence instead of filling gaps with guesses.

Chiron doesn't replace the agent or the model. It is a lightweight context layer that makes the AI agent a team already uses faster, cheaper and more accurate.