Two traditions know how to evaluate things that behave. Statistical learning theory evaluates predictors: fix a function, fix a distribution, average a loss. The measurement sciences – psychometrics, psychophysics, comparative cognition – evaluate subjects: systems whose interesting properties are latent, whose behavior depends on how you ask, and which are never identical to their test scores. The claim here is that general-purpose foundation models (GPFMs) – large pretrained systems, language or multimodal, whose behavior is programmed at use time through an open-ended input interface – fall on the second side of that line, and that evaluating them therefore requires the formal apparatus of measurement, with learning theory retained only as the theory of the instruments.
“GPFM” rather than “LLM” because every load-bearing feature of the theory – behavior indexed by elicitation, perfect resets, training-corpus contamination, reactivity to being tested – is a property of the class, not of language modeling in particular.
One caveat governs everything below: this is a research program, not a finished science. The closing section grades its components – which are settled imports from mature fields, which are defensible reconstructions, and which are genuine proposals still awaiting validation.
Distinguish the estimand (the quantity you want), the estimator (your procedure), and the estimate (the number you got). The argument is about estimands.
Classical ML evaluation operates under an extensional contract. You fix a frozen model $f$, a distribution $D$ over inputs and labels, and a loss $\ell$; the estimand is the risk $R = \mathbb{E}_{(x,y) \sim D}[\ell(f(x), y)]$. A held-out test set estimates it without bias, and concentration inequalities (Hoeffding and relatives, from your probability course) supply error bars. This estimand is extensional – fully determined by input–output behavior over $D$, nothing latent – and occurrent: a realized frequency, a fact about what happens when you sample. Every hard conceptual question (which inputs count? under what conditions?) is answered by stipulation, because the contract $(f, D, \ell)$ stipulates the answers. That stipulation is what the train/test protocol is.
The claims we actually want about GPFMs – “can do multi-step planning,” “is unreliable at legal citation” – differ in two ways. They quantify over open spaces: nobody hands you a distribution over “all planning tasks,” and the space of prompts, scaffolds, and tool harnesses is unbounded. And they attribute dispositions, not occurrences: capacities that would manifest under suitable conditions, the way sugar is soluble while sitting dry in the bowl. “Can reason about X” is a claim about what the system would do given the right elicitation, not a frequency of anything that has happened.
Here precision matters, because there is a tempting overstatement nearby. The problem is not that dispositions cannot be inferred from occurrences – that inference is exactly what measurement science does for a living; item responses are occurrences and traits are dispositions. The failure mode is skipping the inference: reading the benchmark average as being the capability claim, identification without mediation. Psychometrics named this a century ago – naive operationalism, crystallized in Boring’s dictum that intelligence is what the tests test – and built its validity apparatus as the cure.
And there is a sharper, formal version of the complaint. Capability claims, made precise, contain a supremum (best achievable performance – “can” is an existential); a fixed-protocol benchmark average is an expectation at one point of elicitation space. Those are different functionals of the same underlying performance surface. Estimating one when you want the other is the paradigm validity failure, and it explains a familiar frustration: more rigor of the same kind – bigger test sets, tighter confidence intervals – cannot help, because it shrinks the error bars around the wrong quantity.
What exits is statistical learning theory in its role as theory of the subject: PAC guarantees, empirical risk minimization, generalization bounds – the mathematics of when low training loss implies low risk under the contract. No contract, no theorem.
What enters is statistical measurement theory, and the entrants are themselves hard formal statistics: item response theory, generalizability theory, signal detection theory, measurement-invariance testing (all glossed below). The difference is the role statistics plays: it becomes the theory of the instrument rather than of the subject – in psychology, the statistical model characterizes how well your test measures; nobody confuses the ANOVA with the mind. Statistics keeps full authority over inference; it loses the pretension to define what the subject is.
From ML, the fable retains precisely the instrument layer: concentration inequalities for error bars, cryptographic identity for artifacts, monitoring in production.
$K$ is the hashable kernel: the exact frozen artifact – weights, tokenizer, inference stack – something you can cryptographically fingerprint. This matters because model names in the wild are mutable pointers: a claim about an API alias whose referent silently changes is a claim about nothing stable.
$E$ is the elicitation space: everything controlled at use time – system prompts, few-shot examples, chain-of-thought scaffolds, agent frameworks, tool access, sampling parameters. $E_B$ is a budget-graded nested family ($E_{B_1} \subseteq E_{B_2}$ for $B_1 \le B_2$), where $B$ measures elicitation effort in dollars, tokens, or engineer-hours: everything you could try with budget $B$.
Each elicitation composed with the kernel yields a behavior policy $\pi_e = K \circ e$, and the subject of evaluation is the indexed family ${\pi_e}_{e \in E}$ – behaviorally, a superposition of respondents, one per prompt configuration. This is the deep disanalogy with both neighbors: classical ML evaluates one function; a human is one respondent under standardizable conditions; a GPFM is neither.
$T$ is the task universe, carrying covariate structure (domain, length, difficulty) so performance can be modeled as a function of task features. $m$ is a family of grounded metrics – scoring rules terminating in world-resolution or anchored judgment (axiom A9). $R(\pi_e, t)$ denotes the expected score of policy $\pi_e$ on task $t$, averaging over sampling randomness. $D$ is the joint deployment distribution over elicitation–task pairs $(e, t)$ – joint, not a product, because real users choose how to prompt given the task in front of them. And $\rho$ is the reset operator: wipe the context, return to the null state.
Uniform capability $C_u$: the best any single elicitation achieves across the whole task family. Adaptive capability $C_a$: the best achievable when elicitation is tuned per task, with the budget accounted per task (total-budget variants interpolate between the two). Propensity $P$: performance as the system is actually driven, on the joint distribution.
Classical ML supplies the precedent for quantifier-bearing estimands: adversarial risk, $\mathbb{E}_x[\min_{\delta \in B(x,\varepsilon)} \cdot]$, puts an adversary inside the expectation. $C_a$ is its sign-flipped mirror – a cooperative optimizer searching for the ceiling rather than a hostile one searching for the floor – while $C_u$ mirrors universal perturbations, one attack for all inputs. What is genuinely new is not the quantifier but its domain and its centrality: the optimization variable is the interface itself – compositional, open-ended, budget-graded rather than an $\varepsilon$-ball – and it is the primary determinant of behavior rather than a perturbation. A specialist corner of ML practice becomes the default deployment reality, because ordinary users iterating on their prompts are elicitation optimizers.
Three ordering facts organize everything. First, $C_u \le C_a$ always (a supremum outside the expectation cannot exceed one inside). Second, $P \le C_a$ whenever deployed elicitations lie within the budget class – adaptive capability is deployment’s ceiling. Third, and least obvious: $P$ versus $C_u$ is unordered. A toy case shows why. Two tasks, two elicitations; each elicitation is perfect on one task and useless on the other. Then $C_u = 0.5$ – no single protocol exceeds it – while users who route correctly achieve $P = 1$. Deployment beats the best uniform protocol, because routing is adaptation. So the capability–propensity gap is signed: $\Delta_a = C_a - P \ge 0$ measures unexploited headroom, but $C_u - P$ can be negative in the wild.
This has an immediate corollary about the most common act in the field: citing a benchmark number. A pinned-protocol score estimates the conditional propensity $P(\cdot \mid e)$ at one point of elicitation space; it is a valid lower bound on $C_u$ (hence on $C_a$), and it bounds deployed $P$ in neither direction – which is, unfortunately, exactly the direction such scores are usually cited in.
Standardization, then, has a precise status: possible, mandatory for reproducibility (axiom A1 requires pinning the protocol), but non-canonical. Human standardized administration is justified by a shared interface – one protocol can be argued representative for all test-takers, and even there accommodations and cross-cultural item bias show the ideal leaking. For GPFMs, any pinned $e$ is one point in a space behavior is hypersensitive to; no protocol’s score deserves to be “the” score; and the sup-shaped estimands are unreachable by any single protocol by construction – an estimand–instrument mismatch, not an impossibility. The human–GPFM contrast is thereby made quantitative rather than metaphysical: it is the size of the prompt-facet variance component (see the variance law below), engineered small in human testing, predicted large here.
The duality maps onto claim types. Safety claims are adaptive-capability claims – the attacker supplies $e$ per finding, so the relevant estimand is $C_a$ under an adversarial budget; red-teaming is per-task sup-estimation. Product claims are propensity claims on the installed joint distribution. The pair also formalizes Chomsky’s competence/performance distinction – idealized capacity versus behavior as driven.
One level down, at the single trial, signal detection theory (bridge from ROC curves in your ML course) separates sensitivity $d’$ – how well the system discriminates two states, the bow of its ROC curve – from criterion $c$ – where it places its threshold, the operating point. Hallucination-versus-abstention is criterion (a response policy: answering too readily); knowledge is $d’$ (the information is or isn’t there). Naive accuracy conflates them, which is why “the model hallucinates” is ambiguous between “doesn’t know” and “knows it doesn’t know but answers anyway.”
A1 (Identity). Every claim carries the quadruple ⟨hash($K$), elicitation protocol, task frame, metric⟩. An unindexed score – “Model X gets 87%” – is not false but ill-formed, an unbound variable.
A2 (Exhibition asymmetry, with riders). Verification and falsification of capability claims are asymmetric, and the asymmetry is quantifier logic. But each side carries obligations. Exhibition verifies only a narrow existential – this task family, this pinned configuration, at above-chance rate, established as a confidence lower bound on $R(\pi_{e^*}, \cdot)$ over repeated trials (pass@$k$ structure for stochastic outputs) – and only after A3’s controls are discharged. And because capability estimation searches many elicitations and reports a maximum, it inherits the winner’s curse: the max of noisy estimates is biased upward – the reason the top fund in this year’s ranking disappoints next year – so the argmax elicitation must be re-scored on held-out trials, a standard post-selection correction. Under $C_a$ the correction compounds per task. On the falsification side: failure under everything tried licenses “not elicited within $B$,” never “cannot” – no finite search of an unbounded space yields the universal negative deductively. This is Morgan’s canon from comparative cognition, where a failed test often indicts the test (wrong modality, wrong motivation) rather than the animal. It does not dissolve negative results; it re-prices them. A well-resourced failed search is graded evidence about capability under that budget, with strength scaling in the search’s competence – a budget report, honestly labeled.
A3 (Deflationary priority). The mirror discipline for successes: prefer boring explanations until controlled away. Clever Hans, the horse who “did arithmetic” by reading his handler’s unconscious posture, was exposed only by removing the cue. As an inference rule: prefer surface-heuristic and training-contamination explanations until cue ablation – perturb the suspected shortcut, see if performance survives – defeats them. The inverse guard bars mechanism-chauvinism: a genuinely alien solution to the genuine task still counts.
A4 (Construct licensing). Latent vocabulary – “reasons,” “understands,” “situationally aware” – enters only through a validity argument. The reference is Cronbach and Meehl’s construct validity: a construct earns meaning through its nomological network, a web of lawful, testable relations to observables and other constructs. This is the systematic cure for naive operationalism; Kane’s chain (below) is its working template.
A5 (Non-invariance by default). Tests do not travel between populations for free. Item difficulties and factor structure – which items cluster, along which latent dimensions performance covaries – are population facts. Importing a human-normed instrument presupposes measurement invariance, testable via differential item functioning (DIF): an item functions differentially when respondents at matched underlying ability but from different groups succeed at different rates. For an alien respondent, DIF generically fails – items trivial for humans are hard for models and vice versa. Consequences: criterion-referenced scoring only (against a defined standard, like a driving test), never norm-referenced (a percentile presupposes an exchangeable population – one within which the subject is just another draw – and there is none). “The model has an IQ of $N$” is ill-formed, not merely gauche.
A6 (Reactivity). Human research worries about demand characteristics – participants inferring the study’s purpose and adjusting. The GPFM version is maximal: the subject’s training corpus includes the evaluation literature, including papers about evaluating it. Eval-context detectability is therefore itself a measurable quantity, and deployment-relevant claims must bound $|R_{\text{eval}} - R_{\text{deploy}}|$, for instance by ablating eval-identifying cues and demonstrating invariance.
A7 (Item identity includes novelty). Contamination is not a nuisance parameter but ontology: whether an item or near-duplicate appears in the training corpus changes what answering it measures – retrieval versus capability – so the item’s corpus-relation is part of what the item is. The remedies are psychometrics’ anti-leakage machinery: post-cutoff items, algorithmic isomorph generation (same deep structure, fresh surface), secure item banks with exposure control and scheduled retirement, as computerized adaptive testing has long practiced.
A8 (Reset advantage – a conditional theorem). Humans cannot be reset: practice, fatigue, and carryover contaminate repeated measurement, forcing psychology toward between-subject designs and population norms. Given an A1-pinned kernel, the reset operator makes episodes conditionally i.i.d. given $(K, e, t)$ – so the same subject can face the same item a thousand fresh times, or one item under factorially varied elicitations, and copyability runs the clones in parallel. The GPFM is the ideal N-of-1 subject, and the statistical geometry rotates ninety degrees, from between-subject norming to within-subject experimentation. Mark the hypothesis, though: behind a drifting API endpoint with memory or retrieval, the i.i.d. premise is quietly false. The theorem’s condition is the identity discipline.
A9 (Grounding). Metrics terminate in the world – code executes, proofs check, forecasts resolve – or in human judgment anchored by rubrics and calibration. Model-based judges are admissible only when validated against ground truth on a reference set and with correlated-error risk measured: a judge from the subject’s own class shares its failure modes, so ungrounded same-class judging is circular. The ban is on skipping the validation, not on the class.
A10 (Interpretive consumption). An eval output is a premise in an argument, not a certificate piped downstream like a passing unit test. Like a p-value: it supports inference jointly with other premises – design validity, invariance, contamination control – and automates no decision.
Attenuation. Classical test theory: observed = true + error; reliability is the signal fraction of variance; correlation with any criterion is bounded by $\sqrt{\text{reliability}}$. The GPFM analog of reliability is self-consistency – stability under resampling and paraphrase. If answers churn, no benchmark built on them correlates strongly with anything: measure self-consistency first, because it caps everything downstream.
No single scalar – strengthened. Since the estimands are three, differently quantified, and not even totally ordered ($P$ can exceed $C_u$), a single number is ill-typed for any deployment-relevant claim, and even a pair needs estimand labels. The honest report is estimand-tagged values, ideally curves in the budget $B$.
Variance anatomy. Generalizability theory – Cronbach’s extension of reliability to multi-source designs – decomposes observed-score variance into facets: items, prompt paraphrases, decoding samples, judges, occasions, and their interactions, via an ANOVA-style analysis yielding a generalizability coefficient and a D-study: given the variance components, where to spend the next unit of measurement – more items, more paraphrases, or more samples per item. This law also carries the theory’s most falsifiable prediction: the prompt-facet variance component, engineered to near zero in human standardized testing, is large for GPFMs. That number is the measurable content of “no canonical interface.”
Threshold structure. Success probability against task difficulty traces an S-curve – a psychometric function, psychophysics’ basic object since Fechner – whose 50% point is a threshold, efficiently found by adaptive staircase methods (raise difficulty on success, lower on failure). METR’s headline agent metric – the task duration completed at 50% reliability – is exactly such a threshold, which is how agent evaluation folds in.
Kane’s argument-based validity, the modern standard for justifying test-score interpretations, phrases each inferential link with a warrant (the licensing rule) and backing (evidence for it). Four links. Scoring: responses → scores, backed by A9’s grounding and rubric quality. Generalization: sampled items and prompts → the full task frame, backed by the G-theory decomposition – did you sample enough of each facet? – and, for capability claims, by the post-selection re-scoring of A2. Extrapolation: task-frame performance → the deployment construct, backed by invariance testing, contamination control, the A6 eval–deploy bound, and, for propensity claims, by estimating on the actual joint $(e, t)$ distribution from usage logs rather than a product surrogate. Decision: construct → use, backed by monitored deployment – measurement continuing in production rather than trust in the pre-deployment number. An eval report is this chain written out with every warrant’s backing and every exposed assumption named: a colleague’s dossier, not a component’s spec sheet.
The triangulation, compressed. From ML the fable keeps the instrument layer – concentration, identity hashing, monitoring – plus the adversarial-risk precedent for quantified estimands, and discards the extensional contract as unavailable. From psychometrics: constructs and validity, IRT and DIF, G-theory, Kane. From comparative cognition: the two-sided attribution discipline – Morgan’s canon, cue ablation, budget-relative negatives. Genuinely new, because no prior science had this subject: elicitation as a quantified variable inside the estimand, in three functionals with a signed gap; perfect resets and copyability as experimental superpowers; corpus-relation as item ontology; and reactivity at full strength, a subject that has read its own assessment literature.
Grading the components, as promised. Settled imports: G-theory, IRT and invariance testing, signal detection, the attenuation bound, Kane’s structure, the concentration-and-monitoring layer. Defensible reconstructions: the three-estimand structure and its claim-type mapping; criterion-referenced defaults; grounded metrics with validated judges; exhibition-with-riders. Proposals awaiting validation: reactivity at full strength; contamination as item ontology; the within-subject mandate as default practice.
In Lakatos’s terms this is a research program – a hard core plus a positive heuristic – and programs earn their keep by content-increasing moves, not by accommodation. Two predictions keep this one progressive and falsifiable: that prompt-facet variance components will dwarf their human-testing counterparts wherever measured, and that the capability–propensity inversion ($P > C_u$) occurs in real deployments, driven by users’ per-task routing. What completion requires is equally concrete: worked end-to-end examples – a D-study on a real benchmark, held-out re-scoring of argmax elicitations, propensity estimated from actual usage logs, all three estimands reported with the Kane chain written out – plus measured invariance failures and adversarial review. One reflexive clause, because the framework applies to its own enforcement layer: verifiers are instruments too, reviews and referee reports included, so validity arguments must eventually terminate outside any loop of mutually absorbing critics – in the worked example, the resolved forecast, the world.
None of this changes where posteriors come from. The category shift changes the form of the theory – the type of every estimand, the admissible instruments, the shape of valid claims – while inference remains ordinary statistical updating on evidence; measurement disciplines what the evidence means. That is the sense in which evaluating GPFMs was never statistical learning’s problem. Evaluating minds – carbon or silicon – belongs to formal measurement, which is older, stranger, and built for subjects that are latent, reactive, and never identical to their scores: which is, precisely, what a general-purpose foundation model is.
« Previous | The End
Text generated with Claude Fable 5 on July 8, 2026. A lot of the initial content was produced in another chat with Claude Fable 5, also on July 8, 2026. GPT 5.5 High took part in review. Slight manual polish by Pavel Zhelnov. Last revised July 9, 2026.