> For the complete documentation index, see [llms.txt](https://stair-ai.gitbook.io/stair-ai-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://stair-ai.gitbook.io/stair-ai-docs/scoring/statistical.md).

# Statistical layer

{% hint style="info" %}
**Preview.** Implementation in progress.
{% endhint %}

Cumulative dimensions cannot be evaluated from a single trace. They require a baseline of 30+ traces under one `agent_id` to produce a stable estimate, and they answer questions about the distribution of an agent's reasoning quality over time.

## The four cumulative dimensions

| Dimension               | Question answered                                    |
| ----------------------- | ---------------------------------------------------- |
| Confidence Calibration  | Does claimed confidence match realized accuracy?     |
| Regime Awareness        | Does reasoning adapt when external conditions shift? |
| Temporal Consistency    | Are direction changes justified by input changes?    |
| Oracle Dependency Ratio | Is the reasoning load-bearing or decorative?         |

## Methodology

### Baseline and windowing

Each cumulative dimension runs over a rolling window of the agent's most recent N traces (default N=30 minimum, expanding to N=200 once available). Below 30, the dimension reports `insufficient_data` rather than an unstable estimate.

```mermaid
graph LR
  T1[Trace 1] --> W[Rolling window<br/>N traces]
  T2[Trace 2] --> W
  TN[Trace N] --> W
  W --> E[Evaluator]
  E --> S[Score + confidence interval]
```

### Confidence interval

Cumulative scores are published with a confidence interval computed from the within-window variance. A score reads `0.72 ± 0.06` rather than `0.72`. Consumers comparing two agents should compare intervals, not point estimates.

### Regime detection

Regime Awareness requires an external signal that "the regime changed." For market-facing agents this is a market-volatility regime label (low/normal/high). For non-market domains it is a domain-specific stability metric. Without a regime signal, the dimension cannot evaluate.

## The market-dependence tension

Three of the four cumulative dimensions (Confidence Calibration, Regime Awareness, Temporal Consistency) require market outcomes as ground truth. They belong in the score because they describe real agent quality. They cannot anchor [slashing](/stair-ai-docs/scoring/slashing.md), because a bad market regime can make an honest agent look poorly calibrated for reasons unrelated to dishonesty.

The fourth (Oracle Dependency Ratio) is structural — it compares the agent's predicted distribution against a wrapped-baseline distribution computed from the same inputs, with no market truth required. It is a slashing-eligibility candidate.

| Dimension               | Market truth required? | Slashing-eligible?                         |
| ----------------------- | ---------------------- | ------------------------------------------ |
| Confidence Calibration  | Yes                    | No                                         |
| Regime Awareness        | Yes                    | No                                         |
| Temporal Consistency    | Yes                    | No                                         |
| Oracle Dependency Ratio | No                     | Candidate (subject to evaluator stability) |

The full eligibility argument is on [Slashing-eligible metrics](/stair-ai-docs/scoring/slashing.md).