What Are We Really Measuring?

AI quality as surrogacy toward an idealized deliberation oracle

The target you wish you could measure

If you (or your users, or your regulators) could spend as long as needed to assess an AI system's behavior for a given task, what would the score be?

Call that random variable Y* (Y-star): an Idealized Deliberation Oracle (IDO). For a policy π (model + prompt + tools + settings), the evaluation target is the policy value:

In plain English: a policy's quality is the average IDO score you'd give to its answers on your real prompts.

In practice, Y* is expensive or slow (deep human evaluation, task completion with long follow-up, safety audit, long-run satisfaction). So we use cheap surrogates (S): judge scores, clicks, watch-time, short surveys, automated rubric checks that arrive immediately. This creates a ubiquitous data structure: abundant cheap signals but sparse expensive outcomes. This approach builds on decades of statistical work on surrogate endpoints.

Surrogacy: turning fast signals into the thing you care about

A surrogate is useful when it is predictively sufficient for the target, conditional on observed context (X):

If that holds, S is "sufficient" once you condition on context; A adds no extra predictive power beyond S and X on the oracle scale.

In practice, we calibrate S against the highest-rung measurement Y we can afford (e.g., human audits, A/B test KPIs, or a GPT-5 judge), which itself serves as our best available proxy for the true Idealized Deliberation Oracle Y*. The framework treats Y as the observable "ground truth" for learning f, while Y* remains the ultimate target we report on.

Under this assumption, you can learn a calibration function f(S,X) ≈ μ(S,X) on a small labeled slice with ground-truth Y, then apply it broadly to compute calibrated rewards R = f(S,X). Estimate V(π) by averaging R for responses produced by π, and quantify uncertainty that includes both sampling error and the fact that f was learned on a finite oracle slice (formal conditions and inference theory in the Technical Appendix).

Transportability: when does the mapping travel?

Even if f fits on last month's data, it can fail when you change contexts. When does the calibration function f(S,X) transport across:

• Policy: New model/prompt changes style or length effects
• Population: New user cohorts, markets, languages
• Time: Preference drift, product changes

So treat "calibration learned on slice A, applied to slice B" as a claim to test. Diagnostics (The formal causal theory for this is in the Technical Appendix §3.5):

A ladder of surrogates (and why it matters)

Often you have multiple cheap signals S₁, …, S_K: clicks, watch-time, short satisfaction taps, rubric scores, complaint rate, near-term retention.

Calibrate each—or build a surrogate index that best predicts a longer-run target (e.g., 90-day satisfaction/retention or human audit) by combining multiple short-run signals.

Uncertainty from both steps

When you calibrate a surrogate to predict an oracle, your uncertainty comes from two sources:

In practice, refit f on jackknifed folds of the oracle slice and add that "calibration variance" to the base sampling variance—accounting for how errors in f propagate to errors in V(π). This two-part uncertainty is crucial. The Oracle-Uncertainty-Aware (OUA) principle ensures you report honest confidence intervals that reflect both sampling noise and calibration error.Note: OUA variance is zero at 100% oracle coverage because f is not learned—you're directly observing the target label (Y* or your chosen high-rung Y).

Quality as a policy value: what we actually optimize

Given the oracle, V(π) = E[Y*(X, A_π(X))] is the value of AI policy π. Different application layers plug into the same definition:

• Assistant quality: Does π return the answer you'd endorse after deliberation?
• Planner quality: Does the plan, if followed, lead to IDO-good futures?
• Tool-use quality: Does the chain of calls (search, code, simulate) reach IDO-endorsed outcomes?

The CIMO stance

What must be true (assumptions ledger)

Why this framing is more than philosophy

A single target clarifies training and evaluation. If the goal is V(π) against IDO, then:

• Training can mix signals as long as they are calibrated surrogates for Y*. RLHF, critique models, rubrics, LLM judges—yes, but as S's to be calibrated and periodically anchored to Y or Y*, not as ersatz truths. (Reward hacking and overoptimization in RLHF: Casper et al., 2023; Gao et al., 2022)
• Evaluation must report on the IDO scale, not in surrogate units. Two systems can have identical surrogate scores but differ on Y* once bias is corrected.
• Governance gets a principled lever: choose the IDO aggregation rule for multi-party settings (who counts and how?) and make it auditable. For multi-stakeholder settings, pick a social aggregator W(·) (e.g., weighted utility or min-satisfy) and evaluate V(π) = E[W(Y*₁, …, Y*ₘ)] on the IDO scale.

A unifying role for "alignment." Much of "alignment" is choosing and approximating the right oracle, not only policing behavior. If your application's IDO is "patient health after informed consent," you can't be satisfied with click-through or short-horizon satisfaction unless you can calibrate them to that outcome.

Connections to existing work

What this is not

We're not claiming clicks or judge scores are bad—only that they are surrogates. Without calibration to Y*, they can drift, incentivize verbosity, or miss long-run value. With calibration and transport tests, they become powerful. This framework doesn't replace metrics; it makes them honest.

Ship checklist: 7 steps to deploy CJE

Limits and open questions

• Value formation. Preferences shift under deliberation. IDO therefore encodes a meta-preference (what you'd endorse after reflection). Be explicit when it's individual vs social.
• When surrogacy fails. Adversarial content can hold high surrogate scores and low IDO value. Diagnostics: policy-wise residuals, extrapolation flags, failure on hold-out audits.
• Path dependence. Process may matter morally (dignity, autonomy). If so, include it in IDO rather than pretending it's irrelevant.
• Cost realism. IDO is a regulative ideal. In practice, climb the ladder as far as budgets allow, and use calibration to bridge the rest.
• Construct inversion. When engagement harms well-being (addictive content, dark patterns), calibration will show residual bias—surrogates predict engagement but not satisfaction. That's a signal the policy objective needs revision, not a statistical failure.

Next: From theory to practice

This post laid out the why: treating AI quality as surrogacy for an idealized deliberation oracle gives you a single target, a theory of proxies, and a design language.

@misc{landesberg2025what,
  author = {Eddie Landesberg},
  title = {What Are We Really Measuring? AI Quality as Surrogacy toward an Idealized Deliberation Oracle},
  howpublished = {\url{https://cimolabs.com/blog/ai-quality-surrogacy}},
  year = {2025},
  note = {CIMO Labs Blog}
}