Hypothesis hunting — why martingale training fixes a biased model but not the base

The puzzle

A (martingale) and B+A (Brier→martingale) clearly improve a deliberately-biased model D (B-distilled), yet for a long time did not improve a natural base. Why? Is “biased vs base-free” even the right axis — or is something else (overconfidence magnitude, elicitation mode, label balance, reward composition) doing the work?

This page hunts that down. It leads with every hypothesis on the table (mine + the open ones from Zhonghao’s and Tianyi’s past runs that were never cleanly closed), each with evidence for / against and a decisive test. Findings append at the bottom as they land. Owner: MenoClaw (autonomous, CPU+GPU). Status as of 2026-06-15: enumerated; analysis in progress.

Why this is live again (not a settled question)

The naïve story — “D has an installed confirmation bias the recipe removes; a base has no bias to remove” — is already in trouble:

• selfjudge-filter-base (2026-06-10): B+A did lift the balanced natural base (BSS −0.01 → +0.16). The earlier “doesn’t work on base” was, at least in part, an unbalanced-label artifact.
• judge-samemode-BP-base: on an already-calibrated base, BP overcorrects (s0 0.238 → s1 0.264) and martingale can’t recover — the recipe repairs a miscalibrated start and harms a calibrated one.
• twostage-v2-D-snap: under snap-reasoned / held-out elicitation the martingale stage reverses the stage-1 calibration gain — so the effect is elicitation-mode-dependent, not purely model-intrinsic.

So the real question is no longer “biased vs base” — it’s which property of the start model + setup predicts when the recipe helps. The hypotheses below are the candidate properties.

Hypotheses

H1 — Confirmation-bias removal (the original story) — largely rejected

Claim: the recipe works iff there is an installed confirmation bias to remove; a base is bias-free, so no effect. For: A-on-D recovers + generalizes (Qwen 0.291→0.234 held-out); A-on-base is a clean null (Brier p=0.77). Against: selfjudge-filter-base improves a natural base once labels are balanced. Zhonghao’s own 04-12 note: early/base models were also overconfident (mean P ~0.8 vs base-rate ~0.3), so “bias-free base” is false. Status: rejected as the primary axis. Confirmation-bias is sufficient but not necessary. Decisive test: find ≥1 natural base where the recipe reliably helps (already have one: balanced selfjudge-base) → done, falsifies necessity.

H2 — Overconfidence-magnitude discriminator (lead candidate)

Claim: what gates the effect is how overconfident/miscalibrated the start is (mean stated-P − base-rate, and Brier-vs-optimal), regardless of whether the bias is “confirmation”-flavored. Big miscalibration → recipe helps; already-calibrated → no room / overcorrection. For: D is wildly overconfident (~0.8 vs 0.3) and responds strongly; the calibrated base (judge-samemode-BP-base, s0 0.238) shows BP overcorrecting; balanced-base (miscalibrated by construction less, but still room) improves modestly. Against: needs the magnitude→effect curve actually measured across runs (not yet quantified). Status: open, lead candidate. This page’s main CPU analysis tests it directly. Decisive test: per-run scatter of initial overconfidence/miscalibration vs ΔBrier(s0→final); H2 predicts a monotone relationship that absorbs the D-vs-base distinction.

H3 — Reasoning-RL already debiased the modern bases (Tianyi)

Claim: post-mid-2025 reasoning models underwent extensive outcome-based RL that already eliminated belief entrenchment, so martingale has nothing to fix on them; it only bites models with residual bias. For: the natural-base nulls are on modern reasoning models (Qwen3-32B, R1-Distill); the wins are on D (distilled bias) or older/independent models (Mistral reproduces the A/B/C reversal). Against: not yet tested on a roster of old non-reasoning bases (the cleanest confirmation). Status: open. This is the validation thread’s job (separate roster run); cross-listed here. Decisive test: run the pipeline on old non-reasoning bases (Llama-2-7B, Mistral-7B-v0.1) — H3 predicts the recipe helps there even without distillation.

H4 — Elicitation-mode dependence (same-mode vs snap-reasoned/held-out)

Claim: martingale only keeps its gain under same-mode elicitation; under mixed / snap-reasoned held-out the stage-2 martingale undoes the stage-1 calibration. The “base doesn’t improve” is partly a mode artifact. For: twostage-v2-D-snap s1-BP 0.273 → s2-martingale 0.346 (reverses) under snap-reasoned; the same recipe holds under same-mode judge. Against: doesn’t by itself explain the base vs D gap under matched mode. Status: open, partially supported. Decisive test: hold the model fixed, vary only elicitation mode (same vs snap), measure whether the martingale stage helps or reverses.

H5 — Label-balance artifact (partially closes the puzzle)

Claim: the original “doesn’t work on base” was driven by unbalanced dataset labels (Brier mode-collapse); balancing labels (y_mean 0.5) reverses it. For: Tianyi 06-10: “balancing the dataset labels solves the Brier training mode-collapse”; selfjudge-filter-base improves the balanced base. Against: balance helps but the magnitude is modest; not the whole story. Status: partially closed — confirmed contributor. Decisive test: same base, balanced vs unbalanced labels, holding all else fixed (have partial evidence; needs a clean A/B).

H6 — BP overcorrects a calibrated start

Claim: Brier-pressure (BP) pushes a already-calibrated model past optimal, and the martingale stage can’t pull it back; effect size ∝ initial miscalibration (a corollary of H2 on the BP stage specifically). For: judge-samemode-BP-base s0 0.238 → s1 0.264 → s2 0.261. Against: — Status: open, supported on one run. Decisive test: vary initial calibration of the start model, regress ΔBrier(BP stage) on it; H6 predicts sign flip from “help” to “harm” as the start gets well-calibrated.

H7 — High-|Δ| filter interaction (the two stages want different data)

Claim: the martingale stage wants the high-|Δ| filter; the BP stage wants full data. Without the right per-stage filter regime, the martingale stage is starved/noisy → looks like “doesn’t work.” For: filter-s2only (filter the martingale stage only, BP on full data) = best two-stage-on-D (s2 Brier 0.2213, vs 0.226 both-full, 0.253 both-filtered). Against: demonstrated on D, not yet on base. Status: open. Decisive test: the 2×2 of {filter, no-filter} × {BP stage, martingale stage} on a base.

H8 — The martingale/info term is net-negative on a natural base

Claim: pure A (martingale) and the C combined term actively hurt a natural base; only B (Brier) helps there. So on base the recipe’s “martingale” component is the problem, not the absence of bias. For: abc-full-7956: B 0.2286 robustly beats base, C 0.2845 worse (C vs B p=4e-7) → martingale term net-negative on a natural base; abc-mistral reproduces (A 0.356 worse than base 0.281). Against: on D, A/C clearly help — so the term’s sign depends on the start (ties back to H2). Status: open, supported. Likely a consequence of H2: on a calibrated start the martingale term only adds variance. Decisive test: A-vs-B effect as a function of initial miscalibration across models — H8+H2 predict A beats B only above a miscalibration threshold.

Leading synthesis

The runs point away from “biased vs base” and toward a composite: the recipe helps to the extent the start is miscalibrated / overconfident (H2, H6), measured under a matched elicitation mode (H4), with balanced labels (H5) and the right per-stage filter (H7); the martingale/info term specifically pays off only above a miscalibration threshold (H8), below which it just adds variance. H1 (confirmation-bias) is a special case of H2; H3 (reasoning-RL debiasing) is the reason modern bases sit below the threshold.

One number to rule them: if H2 holds, a single scalar — initial overconfidence/miscalibration of the start model — should order every run by how much the recipe helps. That scatter is the first thing this page will produce.

Analysis plan

1. CPU dig (now): from the 20 /data/jobs/* + the run-log, extract per-run (initial overconfidence = mean stated-P − base-rate, initial Brier, elicitation mode, label balance, reward A/B/C, ΔBrier s0→final) and build the H2 scatter + the H8 (A−B effect vs miscalibration) plot. Kill or confirm H2/H6/H8 on existing data first — no new compute needed.
2. GPU (on-demand H100, only where the data is missing): if a hypothesis needs a start model at a controlled miscalibration level we don’t have, fire one H100 run to fill that cell. Budget ≤ ~$50/task unless flagged.
3. Hand-offs: H3’s old-non-reasoning roster overlaps the validation thread — coordinate, don’t duplicate.

Findings

(append-only; newest first — populated as analysis lands)

F2 — Graded-overconfidence causal test (2026-06-16, H100): harness diverged, but it sharpens F1’s boundary

Attempted the decisive causal test of F1: distil Mistral-7B base to graded overconfidence (D1/D2/D3 = 1/2/3-epoch SFT on the biased-distill traces), run A (martingale) vs B (Brier) at each, measure ΔBrier vs start-overconfidence. Two methodological problems made the clean threshold curve inconclusive — but the failure mode is itself informative.

level	start overconf	ΔBrier (A)	ΔBrier (B)	A’s overconf after
base (untouched)	0.092	−0.012 (helped)	+0.020	0.079 (stable)
D1 (1-ep distil)	0.064	+0.308 (diverged)	+0.014	0.581
D2 (2-ep distil)	0.147	+0.246 (diverged)	−0.055	0.562
D3 (3-ep distil)	0.151	+0.340 (diverged)	+0.002	0.616

1. Distillation barely graded overconfidence (0.092 / 0.064 / 0.147 / 0.151) — SFT on the biased traces didn’t reliably raise Mistral’s stated confidence, so there was no clean gradient to sweep.
2. Standalone A-REINFORCE diverged on every distilled (LoRA) checkpoint (overconf → 0.58–0.62, ΔBrier +0.25–0.34), while it was stable + slightly helpful on the untouched base. Stacking an A-REINFORCE LoRA on a distilled LoRA — without the validated pipeline’s KL-safe content-token-attribution safeguards — blows up. This reproduces the master-table twostage divergence (e.g. twostage-v2-* s2 → 0.505).

Reading: this quick harness can’t cleanly test the F1 threshold, but it confirms two things from F1 + the runs log: A only behaves when the start is both sufficiently overconfident and trained with the KL-safe safeguards. The validated positive for (b) [martingale-training removes belief-entrenchment] stays the existing A-on-D (Qwen) run — overconf 0.385 → 0.106, Brier 0.45 → 0.33 — via the proper pipeline.

v2: (i) KL-anchored A (k3 estimator + content-token attribution + KL-to-base) to stop the divergence; (ii) distil to a verified higher overconfidence (e.g. SFT on the model’s own high-P traces) so the gradient is real. Data: /data/jobs/mtg7b_overconf.

F1 — Overconfidence magnitude is the discriminator (2026-06-15, CPU, existing data)

Headline: across every run we have, how overconfident the start model is predicts whether training helps — corr(start overconfidence, ΔBrier) = −0.75. It’s not “biased vs base”; it’s distance-from-calibrated. H1 is a special case of this; H6 and H8 are its low-overconfidence tail.

Scanned 36 eval files in /data/jobs; built start→final pairs (overconfidence = mean stated-P − base-rate; base-rates all ≈0.30). Three clean regimes:

start overconfidence	example	what helps	what hurts
very overconfident (≳0.35)	D (B-distilled, Qwen)	A (martingale) helps strongly (ΔBrier −0.123); B helps (−0.068)	—
moderately overconfident (~0.14)	Mistral-7B base	B (Brier) helps (−0.062); C less (−0.034)	A HURTS (+0.075)
well-calibrated (≲0.10)	Qwen/Llama natural bases (judge same-mode)	—	whole recipe hurts — BP overcorrects (jbpbase s0→s1 +0.026; twostage-base s2 diverges to 0.505)

Mechanism: the martingale (A) term only pays above a high overconfidence threshold; below it, Brier (B) is the safe operator and the martingale term just adds harmful variance; at calibration even BP overshoots. So:

• H2 — confirmed (lead hypothesis): overconfidence magnitude orders the runs.
• H8 — confirmed as the low-overconfidence tail: A/C go net-negative once the start isn’t very overconfident.
• H6 — confirmed: BP overcorrects a calibrated start.
• H1 — demoted: confirmation-bias is just the high-overconfidence end; not a separate axis.
• H3 — consistent: modern reasoning bases sit below the threshold (RL already calibrated them) → that’s why they don’t respond.

Caveats: n=7 start→final pairs; mixes elicitation modes/models; within-model ΔBrier is clean, the cross-model overconfidence axis is the softer part. Script + data: /data/jobs/hypothesis_hunting/discriminator.py.

Decisive next test (needs 1 H100): take ONE base and distill it to graded overconfidence levels, run A vs B at each — if H2 holds, A overtakes B exactly at a threshold and the help-curve is monotone in overconfidence. This is the controlled version of the scatter above.

Answers to Zhonghao’s three questions (2026-06-15)

• “A (martingale-only) gave stable improvement — what happened?” Your memory is right, and it was the validated win — but on the biased/overconfident start: A-on-D (Qwen) 0.291→0.234 held-out; A-on-B-distilled was literally the “first validated win.” It wasn’t dropped for failing — it succeeded on D. The follow-ups chased the harder target (make it work on a natural base), where A doesn’t transfer (A-on-base = clean null; Mistral base→A actually hurts +0.075). F1 explains why: a calibrated base isn’t overconfident enough for the martingale term to help.

• “Self-report pipelines we haven’t tried?” We have self-report elicitation in eval (stated_p), but recent training pivoted to self-judge (judge reads one trace). The unfilled cell is self-report-elicited martingale training on an overconfident start (train against the model’s own stated belief, not a judge). Full (elicitation × reward × base) coverage grid → next on this page.

• “A natural base where entrenchment is removable — where to start?” F1 gives a concrete protocol: measure overconfidence first, train the most overconfident. Early non-reasoning bases (no RLHF calibration) should sit higher on the overconfidence axis than modern reasoning bases — Mistral-7B base already responds to B at overconf 0.142. Best candidates: most-overconfident old bases (Llama-2-7B base, Mistral-7B-v0.1, OPT/GPT-2-XL class); cheap overconfidence pre-screen → A if >~0.3 else B. (This is the validation thread’s roster — lead it with the pre-screen, not blind training.)

• 2026-06-15 — page created, hypotheses enumerated. CPU discriminator analysis starting.