Learning from Learner Theorem — ZH version (working skeleton)

Status

A working skeleton for a new version of the Learning-from-Learner (LFL) theorem, initiated by Zhonghao (2026-07-22). It deliberately reuses the metaphysical model of Tianyi’s LFL Theorem (in progress) so the two versions stay directly comparable; the sections copied verbatim are marked. The theorem statement itself, the baselines, and the experimental harness are open slots — see the final section.

Working principles

This version is built under three commitments, which constrain every design choice below.

1. Testability first. There might be many different versions of the Learning-from-Learner theorem. This one is deliberately built to be the version that can be quickly tested in experiments: whenever a formulation choice trades elegance against experimental accessibility, accessibility wins.
2. Beat stable baselines. The resulting method should perform better than the baselines — whatever the right baselines turn out to be — and those baselines must themselves be stable: reproducible, well-understood reference points, not moving targets. A comparison against an unstable baseline establishes nothing.
3. Scaling-up potential. The construction should be competitive with frontier models on some dimensions of human-relevant performance. Which dimensions those are is itself an open question — they are yet to be defined, and then tested — but the formulation should not foreclose them.

Metaphysical model — the policy-extended Dec-POMDP

We follow the policy-extended Dec-POMDP model, as it is conventional in this project: it makes this version easy for readers to engage with and to compare against the older version, and there is for now no strong reason to discard it. The notation, the model definition, and the rollout definition below are copied verbatim from lfl-theorem-in-progress so that the two documents cannot drift apart silently.

Notation and conventions

For a set $X$, $\Delta [X]$ is the set of probability distributions on $X$, and $X^{\ast }$ is the set of finite sequences over $X$. In a kernel written $\mathcal{T}(s^{\prime };s,\mathbf{a})$, the semicolon separates the point at which the output distribution is evaluated ($s^{\prime }$) from the inputs ($s,\mathbf{a}$), so $\mathcal{T}(\cdot ;s,\mathbf{a})\in \Delta [\mathcal{S}]$.

Time runs as $a_0,o_0,a_1,o_1,\dots$, so an agent acts on the history available so far and the observation $o_t$ is generated from the post-action state $s_{t+1}$. There is no observation before the first action.

An uppercase symbol such as $S$, $S_t$, or ${\mathbf{S}}^{\prime }$ denotes a random variable, and the matching bold lowercase symbol $s$, $s_t$, ${\mathbf{s}}^{\prime }$ denotes a value it can take. Bold symbols correspond to vector-valued entities.

Negative subscripts, like ${\mathcal{O}}_{-i}$, denote ”… of all agents other than $i$”.

Definition (PED-POMDP)

A policy-extended Dec-POMDP (PED-POMDP) is a 10-tuple

$$\left(\mathcal{S},\mathcal{I},\{{\mathcal{A}}_i\},\mathcal{T},{\mathcal{T}}_0,\{{\mathcal{O}}_i\},\{{\omega }_i\},\{{\mathcal{R}}_i\},\gamma ,\{{\pi }_i\}\right),$$

where every family written $\{{\cdot }_i\}$ is indexed by $i\in \mathcal{I}$, while $\mathcal{S},\mathcal{I},\mathcal{T},{\mathcal{T}}_0,\gamma$ are global, and

• $\mathcal{S}$ is the state space,
• $\mathcal{I}$ is the agent index space,
• ${\mathcal{A}}_i$ is the action space of agent $i$,
• $\mathcal{T}(s^{\prime };s,\mathbf{a}):\mathcal{S}\times {\prod }_i{\mathcal{A}}_i\to \Delta [\mathcal{S}]$ is the state transition kernel,
• ${\mathcal{T}}_0\in \Delta [\mathcal{S}]$ is the initial state distribution,
• ${\mathcal{O}}_i$ is the observation space of agent $i$,
• ${\omega }_i(o;s):\mathcal{S}\to \Delta [{\mathcal{O}}_i]$ is the observation kernel of agent $i$,
• ${\mathcal{R}}_i(s,\mathbf{a}):\mathcal{S}\times {\prod }_j{\mathcal{A}}_j\to \{0,1\}$ is the (stochastic) reward of agent $i$,
• $\gamma \in [0,1)$ is the discount factor, shared by all agents, and
• ${\pi }_i\left(a_{T,i};\{a_{t,i},o_{t,i}{\}}_{t=0}^{T-1}\right):({\mathcal{A}}_i\times {\mathcal{O}}_i{)}^{\ast }\to \Delta [{\mathcal{A}}_i]$ is the policy of agent $i$.

Under a joint policy, agent $i$ seeks to maximize $\mathbb{E}\left[{\sum }_{t\ge 0}{\gamma }^t{\mathcal{R}}_i(s_t,{\mathbf{a}}_t)\right]$. Rewards are per-agent, so the model is general-sum, and the cooperative Dec-POMDP is the special case ${\mathcal{R}}_i\equiv \mathcal{R}$. The $\{0,1\}$ codomain is WLOG: any bounded real-valued reward affinely rescales into $[0,1]$ and is realized as a Bernoulli draw with that mean, preserving expected return up to affine equivalence and hence the optimal-policy set.

Definition (Rollout)

A $T$-step rollout of a PED-POMDP is a tuple

$$\rho =(\{s_t{\}}_{t=0}^T,\{{\mathbf{a}}_t{\}}_{t=0}^{T-1},\{{\mathbf{o}}_t{\}}_{t=0}^{T-1}),$$

where $s_t\in \mathcal{S}$, ${\mathbf{a}}_t=(a_{t,i}{)}_{i\in \mathcal{I}}\in {\prod }_i{\mathcal{A}}_i$, and ${\mathbf{o}}_t=(o_{t,i}{)}_{i\in \mathcal{I}}\in {\prod }_i{\mathcal{O}}_i$: the state sequence, joint actions, and joint observations of a gameplay of exactly $T$ time steps. Under the time convention, $s_{t+1}$ is the post-action state of step $t$ and ${\mathbf{o}}_t$ is emitted from $s_{t+1}$, so the state sequence has $T+1$ entries and ends at $s_T$. Write $h_{t,i}(\rho )=\{(a_{u,i},o_{u,i}){\}}_{u=0}^{t-1}$ for agent $i$‘s history at step $t$ read off $\rho$. Since the PED-POMDP fixes every policy, it induces a probability distribution over $T$-step rollouts,

$$\mathrm{Pr}[\rho ]={\mathcal{T}}_0(s_0)\prod _{t=0}^{T-1}\left[\prod _i{\pi }_i\right(a_{t,i};h_{t,i}(\rho )\left)\right]\cdot \mathcal{T}(s_{t+1};s_t,{\mathbf{a}}_t)\cdot \prod _i{\omega }_i(o_{t,i};s_{t+1}),$$

and $\rho$ carries all information needed to describe the gameplay: in particular every reward is determined, $r_{t,i}={\mathcal{R}}_i(s_t,{\mathbf{a}}_t)$. In this sense a PED-POMDP is a closed stochastic system rather than a game to be solved, and the rollout distribution is its complete behavioral description.

flowchart LR
  S["state s_t ∈ 𝒮<br/>(includes metacognitive + policy information, folded in)"]
  H["agent i (human learner)<br/>policy π_i on own history"]
  A["agent j (assistant)<br/>policy π_j on own history"]
  S -- "ω_i: observation o_t,i" --> H
  S -- "ω_j: observation o_t,j" --> A
  H -- "action a_t,i" --> T["transition 𝒯(s'; s, 𝐚)"]
  A -- "action a_t,j" --> T
  T --> S

Two remarks from the source document carry over unchanged and are load-bearing here: the state can be made to encode metacognitive information (each agent’s uncertainty over how its peer, or itself, accesses the world folds into $\mathcal{S}$), and the state can encode policy information (parameterized policies and priors over their parameters also fold into $\mathcal{S}$, with a one-time private revelation to the owning agent). We adopt both maneuvers, so everything an agent might be uncertain about — the world, the channels, the other agent’s policy — lives in one state space and is handled uniformly.

Theory of mind

Definition (informal). An agent exercises theory of mind when it must infer another agent’s cognitive information processing — that agent’s observation channel, belief state, and policy — none of which is directly observable to it.

In the PED-POMDP this needs no new machinery: by the two folding remarks above, everything that constitutes agent $j$‘s cognition (its observability configuration, its policy parameters, and — through the egocentric construction — its private history) is encoded in components of the state, and agent $i$‘s inference about it is carried by its belief over that state. Theory of mind, formally, is the restriction of agent $i$‘s belief to the state components that encode agent $j$‘s cognition.

On terminology. Earlier documents in this project (and the public explainer) use “metacognition” for this capacity. We use theory of mind here, for the following reason: in the psychology literature the two terms are conventionally distinct — metacognition (Flavell) is cognition about one’s own cognitive processes, while theory of mind (Premack & Woodruff) is the attribution of mental states to other agents. The capacity at the center of LFL is the assistant’s inference about the human’s cognition, which is theory of mind under the accepted convention. The project’s earlier usage stretched “metacognition” to cover the union of both directions (Tianyi’s remark defines it as “cognition about one’s own or another’s cognitive process”), which is a defensible unification — an agent tracking the reliability of a source needn’t care whether the source is itself — but it is nonconventional, and a reader arriving from the literature will mis-file it. Until there is a strong reason in favor of the unified term, we keep “theory of mind” for other-directed inference and reserve “metacognition” for the self-directed case, naming each explicitly where the difference matters.

The problems this theorem is set up to address

This section states explicitly what the Learning-from-Learner algorithm is for. The theorem, once stated, should be judged by whether it makes progress on these.

Central problems

(a) A verifiable predictive model of the user. The assistant must build a predictive model of the user — and there must be a way to verify that user model against behavioral data. Verification is the hard half: the same behavior is compatible with many user models, so the verification procedure must state which assumptions it leans on and which distinct predictions the model makes that behavior can actually check.

(b) Intervening without manipulating. The assistant intervenes on the user’s beliefs and behavior — that is the point of assistance — but must do so without manipulating or deceiving them. The purpose of the intervention is to simulate a future ideal self that is recognisable by the current human but not limited by them: recognisable, so the current person retains authority over the trajectory; not limited, so growth beyond the current horizon remains possible. The tension between those two requirements is the crux, and the theorem must take a position on it.

(c) Protecting autonomy. Throughout, the assistant’s actions must be free from coercion and misinformation. This is the constraint surface within which (a) and (b) operate: whatever the theorem licenses, it must not license influence that works through force or falsehood.

Instrumental problems

Resolving the central problems requires resolving at least the following, which are not goals in themselves but obstructions on every path to the goals.

(a) Reward unidentifiability. There are multiple reward functions that can explain the same set of human behavioral data. Any user model in the sense of central problem (a) therefore either commits to explicit disambiguating assumptions, or gains identification through richer observation channels and interventions. The theorem must say which.

(b) Path dependence. One’s behavior plays a role in deciding what information one receives next. The data the assistant learns from is therefore not exogenous: it is partly the product of earlier interactions, including the assistant’s own past influence. A theorem that treats the human’s behavioral record as an independent sample will be wrong in exactly the cases that matter.

(c) Selecting among future selves. There are millions of potential future selves one could simulate. Two mechanisms are needed: a conversation mechanism — how the current human and a simulated future self actually interact, and what from that interaction is to be trusted — and a recognition mechanism — a way for the current human to recognise those future selves as theirs, without the recognition test collapsing into the current self’s horizon and re-importing the entrenchment problem. This instrumental problem is where central problems (b) and (c) meet.

flowchart TD
  IA["instrumental (a)<br/>reward unidentifiability"] --> CA["central (a)<br/>verifiable user model"]
  IB["instrumental (b)<br/>path dependence"] --> CA
  IB --> CB["central (b)<br/>intervene without manipulating"]
  IC["instrumental (c)<br/>selecting among future selves"] --> CB
  IC --> CC["central (c)<br/>protect autonomy"]
  CA --> T["the LFL theorem<br/>(statement: open slot)"]
  CB --> T
  CC --> T

Open slots (to be filled)

The skeleton above fixes the model, the vocabulary, and the problem statement. What remains open, in rough dependency order:

1. The theorem statement. Working principle 1 constrains its shape: it should be a claim whose truth or falsity an experiment of realistic size can bear on — for instance an identifiability claim (under stated assumptions, the user model of central problem (a) is verifiable from a feasible amount of behavioral data), or an optimality-gap claim (an assistant satisfying stated conditions improves a defined human-performance metric over the baselines). To be chosen and stated.
2. The baselines. Working principle 2 requires stable ones. Candidates worth vetting for stability: a fixed neutral-prompted assistant (no learning-from-learner mechanism), a standard RLHF-style assistant (the oracle-loop baseline), and a no-assistant condition. To be fixed before any headline experiment.
3. The human-performance dimensions. Working principle 3 leaves these explicitly undefined. Candidates must be dimensions where “competitive with frontier models” is measurable. To be defined and then tested.
4. The experimental harness. Given 1–3, the quick-test loop that principle 1 demands.

Appendix A — How theory of mind has been evaluated in psychology

The psychological literature on measuring theory of mind is unusually beautiful as measurement literature: the field spent five decades refining one question — what observable behavior would prove that an agent represents another agent’s mind, rather than just their behavior? — and its hardest-won lessons are directly relevant to this project. This appendix surveys the landmarks.

A.1 The founding question and the false-belief criterion

The term enters science with Premack & Woodruff (1978), “Does the chimpanzee have a theory of mind?” — the chimpanzee Sarah watched videos of a human struggling with a problem (unreachable bananas, a locked cage) and reliably chose the photograph depicting the solution, which the authors read as her attributing a goal to the actor. The deep contribution, however, came from the commentaries. Dennett (1978) pointed out a confound that has organized the field ever since: as long as the target’s belief is true, predicting behavior from their belief and predicting it from the world itself give the same answer — so no true-belief task can ever distinguish a mind-reader from a competent behavior-reader. The diagnostic case is false belief: only an agent that represents the other’s representation can predict that someone will act on a belief the predictor knows to be wrong.

That criterion became a task in Wimmer & Perner (1983): Maxi puts his chocolate in the green cupboard and goes out; his mother moves it to the blue cupboard; where will Maxi look? Children below roughly age four answer “the blue cupboard” — they answer from the world, not from Maxi’s mind — and around age four the answer flips. Baron-Cohen, Leslie & Frith (1985) compressed the paradigm into the Sally–Anne task and put it to clinical use, finding that most autistic children in their sample failed it while children with Down syndrome of comparable mental age passed — the birth of the ToM account of autism. Two companion paradigms triangulated the same construct: the unexpected-contents (“Smarties”) task (Perner, Leekam & Wimmer 1987) — a candy tube turns out to contain pencils; what will your friend think is inside? and what did you think, a minute ago? — which tests false-belief attribution to others and to one’s own past self; and the appearance–reality task (Flavell, Flavell & Green 1983) — a sponge painted like a rock — which tests holding two representations of one object at once. The capstone is the meta-analysis of Wellman, Cross & Watson (2001): across 178 studies and every task manipulation the field had tried, the developmental transition around age four is robust — manipulations shift difficulty but not the trajectory — which is about as replicable as developmental psychology gets.

A.2 Raising the ceiling: second-order and “advanced” tests

First-order false belief saturates by age five or six, so the field built harder instruments. Perner & Wimmer (1985) introduced second-order false belief — “John thinks that Mary thinks the ice-cream van is at the church” — recursion one level deeper, passed around age six to seven. For older children and adults: Happé’s Strange Stories (1994) probe the pragmatics that require mental-state attribution — irony, white lies, double bluff, persuasion — where the literal reading and the intended meaning diverge; the faux pas test (Baron-Cohen et al. 1999) asks subjects to detect that a speaker said something they shouldn’t have known would hurt, requiring simultaneous belief and emotion attribution; the Frith–Happé animations (Abell, Happé & Frith 2000) show bare geometric triangles whose motion adults spontaneously describe in intentional terms (“the big one is encouraging the little one”), measuring the propensity to mentalize with all social surface features stripped away. The Reading the Mind in the Eyes Test (Baron-Cohen et al. 2001) — inferring mental states from photographs of eye regions — became the most-used adult measure, and is also the field’s cautionary tale: later psychometric work argues it loads heavily on emotion recognition and vocabulary rather than belief attribution, a reminder that a task’s popularity and its construct validity are different things.

A.3 Implicit measures, infant claims, and the replication reckoning

Onishi & Baillargeon (2005) moved the measure from verbal answers to looking time: fifteen-month-olds look longer when an actor reaches where a toy actually is rather than where the actor last saw it — as if the infants expected false-belief-consistent behavior three years before they can pass the verbal task. Southgate, Senju & Csibra (2007) sharpened this into anticipatory looking — infants’ eyes go to where the mistaken actor will search, before she moves. The same method crossed species: Krupenye et al. (2016, Science) found great apes’ anticipatory looking also tracks a human actor’s false belief, reopening Premack & Woodruff’s question with Dennett-compliant methodology.

Then the reckoning. Systematic replication attempts (Kulke & Rakoczy 2018; Poulin-Dubois et al. 2018, among others) found the anticipatory-looking effects fragile — some paradigms replicating inconsistently or not at all — while the verbal false-belief transition stood firm. And Heyes’ “submentalizing” critique (2014) supplied the theoretical knife: many implicit successes can be explained by domain-general attention and behavior-rules (“agents return to where they last interacted with an object”) with no mental-state representation anywhere. The field’s current, chastened position: the explicit false-belief transition is one of the most solid findings in developmental psychology; what the implicit measures measure is genuinely unsettled.

A.4 Adults are not at ceiling

A different line established that adult ToM is a performance under load, not a switch that stays on. In the director task (Keysar et al. 2000; 2003), an adult follows instructions from a director who cannot see some objects on a shelf; adults reliably make egocentric errors, reaching for objects the director cannot possibly mean — despite perfectly knowing the director’s visual access when asked. Epley et al. (2004) framed the mechanism as anchoring-and-adjustment: one’s own perspective is the anchor, adjustment toward the other’s is effortful and habitually insufficient. The measurement moral: possessing the concept and deploying it under time pressure are different capacities, and a good evaluation battery must separate them. (Wellman & Liu 2004 built the developmental version of this gradation: a Guttman-scaled progression — diverse desires → diverse beliefs → knowledge access → false belief → hidden emotion — that replaced the pass/fail cliff with a scale.)

A.5 Evaluating theory of mind in machines — and the moral for this page

The LLM era imported the psychology wholesale, then re-learned its hardest lesson at speed. Kosinski (arXiv:2302.02083) ran classic false-belief vignettes on GPT-class models and claimed emergent ToM; Ullman (arXiv:2302.08399) replied by showing that trivial alterations to the vignettes — a transparent container, a trustworthy bystander announcing the contents — collapse the performance, exactly the signature of pattern-matching on task surface rather than representing the protagonist’s mind; Shapira et al. reached the same verdict at benchmark scale (“Clever Hans or Neural Theory of Mind?”, 2023). Purpose-built machine batteries followed the psychology’s design playbook: ToMi (Le, Boureau & Nickel 2019) generated false-belief stories with controlled confounds; BigToM (Gandhi et al. 2023) generates items from explicit causal templates over percepts, beliefs, and desires so that each inferential link is separately testable; FANToM (Kim et al. 2023) embeds belief-tracking in multi-party conversation, where information asymmetry arises naturally rather than by narrator fiat; and Strachan et al. (2024, Nature Human Behaviour) ran humans and models through a shared battery (false belief, irony, faux pas, strange stories), finding the profile uneven across tests in both directions.

The through-line from Dennett to Ullman is one sentence: behavioral success on any single task underdetermines the mechanism, so the entire craft is designing conditions under which the competing explanations finally predict differently. That is, verbatim, this page’s instrumental problem (a) — reward unidentifiability, where multiple user models explain the same behavioral data — transposed into the key of measurement. The psychological literature is fifty years of accumulated technique for exactly the deconfounding problem the LFL evaluation harness (open slot 4) will face, and it should be mined as such: false-belief-style diagnostics (conditions where the assistant’s model of the user and the user’s actual state diverge), trivial-alteration robustness checks in the style of Ullman, and load manipulations in the style of the director task all transfer.

Related

LFL Theorem (in progress) — Tianyi’s version · lfbl-mdp-in-progress · lfbl-mba-in-progress · lfl-foundations-qa