Meno Research Hub

高考志愿 — Active Human Identification (elicit ↔ LLM, running 画像, report)

3 min read

Codebase (2026-06-27): this engine now lives in meno-gaokao-counsellor (core/…), extracted from reflection-game — see gaokao-standalone-plan. Module paths below use the new core/ layout.

The layer that turns the game from “AI talks at you” into co-creation: each scene actively elicits what only the human knows, folds it into a running 画像 (human model), and ends with a competing-objectives user report. Shipped 2026-06-23.

Why — the theory (VOI routing under complementary expertise)

The design question (ZH, ideas): in real time, which information should come from the LLM vs. be elicited from the human, so co-creation is better off?

Treat it as value-of-information (VOI) routing. For each candidate piece of info, weigh:

  • decision-relevance — how much knowing it changes the action/ranking (VOI proper);
  • source error — which of {LLM, human} has lower calibrated uncertainty on it.

Route each item to the lower-error source, prioritized by VOI ÷ cost. The split falls out:

  • LLM supplies public / aggregate / base-rate facts (salary, course structure, employment) — cheap, broad, and the human usually doesn’t know them. (→ the grounding-layer.)
  • Elicit from the human the private, idiosyncratic variables — values, constraints, family situation, “what they can’t let go of.” The LLM’s estimate there is a high-variance guess; the human is the low-variance oracle.

Real-time policy = greedy active learning / Bayesian experimental design: each step, query/supply the item that most reduces remaining decision-relevant uncertainty.

Two load-bearing subtleties:

  1. LLM error is heteroscedastic — low on aggregates, high on specifics/tails (where it hallucinates). So “LLM for facts” only holds where it is calibrated; use calibrated uncertainty, not nominal. (This is also why the per-scene caveats stress treating specifics as fiction.)
  2. Preferences are partly constructed in the asking, not retrieved (Slovic/Lichtenstein). “Elicit from the human” is not a clean read of a fixed value — the query perturbs the target. For an ideal-preference instrument that is the mechanism, not a bug, but it breaks the naive “human = ground-truth oracle” assumption: we co-construct, not read.

Principled form: each turn, elicit the variable with the highest VOI × (human − LLM error gap), while accounting for the elicitation itself moving the target. That is the LfL / ideal-preference active-identification problem (Russell et al., Learning the Preferences of a Learning Agent, is the closest formal handle).

What shipped (3 phases)

  1. Per-scene elicitation. unified_turn emits an elicit field — a scene-tailored 「最想听你说」 question targeting human-privileged variables (family considerations, the real crux, city vibe, how the news shapes their belief). Rendered with a same-line text + 🎙 input.
  2. Running 画像 update. Answering POSTs /rank_elicit → update_narrative (v4-pro) folds the answer into the point-form 画像 and corrects now-stale points; the 🪞 panel re-renders. The 画像 is no longer generated once at start — it iterates each scene. (profile.elicited logs the Q&A.)
  3. User report. At the end, /rank_report (v4-pro) emits a reflective report from the 画像 + elicited log + final ranking. Framing is deliberate: present current state, never predict pitfalls; surface the student’s competing objectives and invite them to weigh the trade-offs (with an open door to keep talking / re-reflect).

Where it lives (code)

gaokao/prompts.json → unified_turn (elicit), profile_update, user_report (and the same keys in prompts.defaults.json — the loader only emits BASE keys). core/ranked.py → update_narrative, user_report. core/web.py → /rank_elicit, /rank_report. gaokao/rank_page.html → elicit block, submitElicit, loadReport. Human model = build_narrative (initial) → update_narrative (per scene), both v4-pro; see profile-construction + the 15 dimensions. Engine context: WTE · investigator · architecture flowchart.

Instant feedback