Reflection Game — Log

This page = Updates · Feedbacks · Bugs for the reflection game.

Not an experiment-runs log (this isn’t an experiment project). Three living sections: what shipped, what people said, what’s broken. Newest first.

Planned — redesign (ranked-list + investigator model) · 2026-06-14

Per ZH: pivot the 高考 game from “A/B two lives” to a single trajectory + adjustable ranked list + investigator agent. Action items (awaiting build go):

1. Starter — drop sliders → initial ranked list of 5 combos (专业@学校) + a confidence rating + an auto-generated profiling quiz (most-informative multiple-choice Qs, max uncertainty-reduction about the person).
2. Side panel — the 5-option list, always visible, drag-to-reorder; each reorder logged with its stage.
3. Single main trajectory — follows the current #1; stages map to ideal-preference-checklist factors (course load → daily life → finances → city → career…), woven in implicitly.
4. Investigator agent / factor-stage — surfaces how #1 fares on that factor vs one strong contender among the 5 that differs (new info, not a verdict).
5. Re-rank → pivot — moving top-1 is recorded; the trajectory continues/regenerates from the new #1.
6. End — output the new ranked list; self-check against the 反思均衡检查清单 (ideal-preference checklist); collect final confidence; show initial→final shift.
7. Reuse — full-catalogue grounding (investigator facts) · checklist (factor sequence) · emergent engine (trajectory) · no-right-answer principle. The investigator = the multi-agent investigator made concrete.

Updates

Shipped changes / new capability, newest first.

Date	Update
2026-06-18	场景下线 / Scene retired: 性别与行业 — removed 性别与行业 (gender_and_industry) from the active WTE pool per ZH (no gender info on the user; revive when demographic data exists). Kept in scenes.json with retired: true; ranked.py loader skips retired scenes. Pool 18 → 17, live on the test link. 134e0ef. → Scenes.
2026-06-18	1K eval — diversity holds at scale — 1000 grid profiles × 7 stages (996 ok, 1.7h): rotation 5.43/7, family-default 3%, 18/18 scenes, 580 distinct dim-sets (58% unique — pigeonhole, healthy). Answers ZH’s ‘still diverse at 1000?’ → yes, and steadier than 24×7. → Eval.
2026-06-18	Eval: simulated player — PLAYER=1 adds an LLM that plays AS the profile and reorders its ranking per stage (the headless run never reorders → ranking-shift was a dead 0). Now a real belief-update signal. Validation 10×5: 38 reranks, 10/10 reorder, 7/10 top-flip. d45f0d7. → Eval.
2026-06-17	Human-dimension data layer + unified inference — human_dimensions.json (14 ideal-preference tension-axes = belief state) + unified_turn: ONE call does WTE(scene) + investigator(contender=max-discriminator + question + investigate) + story-writer(prose), driven by the human-dim uncertainty map. Commit 76db101; data-layer page → Human-Dimension Data Layer. Verified no deepseek-v3 quality drop; one fewer call/stage.
2026-06-17	World-Turning Engine (P0) — next stage is chosen per-person by value-of-information (an LLM modeler builds an ideal-preference uncertainty map → picks the highest-uncertainty, ranking-relevant, avoided dimension) instead of a fixed 7-stage order. gaokao/wte.py, commit 1badf2b. → World-Turning Engine · Ideal-Preference Dimensions.
2026-06-17	Profile = one synthesized NL portrait — replaced the weak 8-pole vector with an LLM-synthesized 150–250字 natural-language profile from all sources; drives world/storyline/investigation. Commit 8a4a6c3. → Profile.
2026-06-17	雨来 (Yusoong) wired live — partner 专业 detail + 专业判别 (误区/红线) nationally + 录取分/等位分 province-gated (no empties) injected into the investigator above the local KB. Commit 4899c46. → 雨来 Integration.
2026-06-17	Editable-prompts editor — the live game’s generative prompts (investigate · translate · frame-test · profile-narrative · WTE) editable at /prompts; saves take effect next call, guarded against dropping placeholders. → Per-Stage Anatomy.
2026-06-17	Data-layer subpages — 就业 · 学校/专业 · 录取 · 总览 (catalog + source-tier + integrity). → Data Overview.
2026-06-16	Real employment data wired into grounding — grounding.py now feeds the model report-level uni 就业率/升学率/出国率/起薪 (with 口径) from the harvest; dropped the miscalibrated distilled per-major salary. Commit 7cb8bb0. → Grounding Layer.
2026-06-16	雨来 (Yusoong) partner-data integration — client gaokao/yulai.py (6 endpoint groups: 专业 / 等位分 / 录取分 / 学生画像…), source tier 雨来 → KB → web-search → qualitative, + bilingual 对接 subpage. Commits c271fff / 1925872. Status: partner API went 404 → live (auth-gated) on 6/16, but our prod key is rejected (INVALID_API_KEY) — awaiting a valid yl_live_… key. → 雨来 Integration.
2026-06-16	Uni-employment harvest — real data, no top-N — 2347 unis crawled, 2121 with verified 就业质量报告 data (2023 就业率 · 1296 升学率 · 691 salary; honest null for the 226 without a published report). Replaces the miscalibrated LLM-distilled salary. gaokao/real_data/uni_employment.json, commit c271fff.
2026-06-16	毕业去向 opening questionnaire — rank 保研 / 考研 / 出国 / 就业 / 考公考编 / 创业 at intake; folds into the profile so every stage’s investigation conditions on your 去向 priority. Commit b5b5477.
2026-06-16	All 高考 subpages bilingual — 8 subpages converted to 先中后英 (per-section 中→英). research-hub 65f15f9→6ceeba6.
2026-06-14	Web-search + 反思均衡-algorithm subpages — faithful impl writeups: web search (OpenRouter web plugin, the SRC_RULE prompt verbatim, _clean() / _auth_sources() post-processing) → Investigator 调查员; the 反思均衡 推演 framework (profile use · least-examined-aspect · bias) → 反思均衡 Algorithm. Commit 7a7d41f.
2026-06-14	Per-stage anatomy doc — component-by-component (storyline · investigation · tip · 看懂这个专业 · framing-test) with model + exact prompts → Per-Stage Anatomy.
2026-06-14	Grounding — full coverage — KB extended 50 → 906 majors (full 本科专业目录, 0 distill errors). Commit 0dccc06. → Grounding Layer.
2026-06-14	Ideal-preference checklist — systematic inventory of factors that move people toward 反思均衡 (substantive factors · biases · formation dynamics · how the game surfaces each). → Ideal-Preference Checklist.
2026-06-13	Blur fixed — journey/scene canvas scales by devicePixelRatio (crisp on retina). Commit 536e035.
2026-06-13	Quality-overhaul plan published — root-cause response to the genericness feedback → Quality Overhaul Plan.
2026-06-13	Backend workflow doc (EN/CN) — full per-turn pipeline + prompt anatomy → Backend.
2026-06-13	Grounding layer shipped (v1) — per-major knowledge (50 majors, fact/opinion-labeled) injected into scene gen + a concreteness contract; scenes now use real courses/careers/pivots. → Grounding Layer. Commit 35e617f.
2026-06-11	Visual/interactive backbone — adopted meno-sh/Agenda-Explainer (scroll-driven rough.js engine) into interaction-visual/ (PR #23).
2026-06-11	高考志愿填报 variant MVP — playable: intake → live both futures → backward-reasoning mirror; on real admission data (2,347-school tier index). Branch gaokao-variant.
2026-06-11	Developer-feedback loop — a ?dev=1 button on these hub pages → JSON → this log + a real-time MenoClaw ping (PR #22 / hub PR #5). See Feedbacks below.
2026-06-10	Save / resume — autosaves decisions; Resume replays them (localStorage).
2026-06-09	Bilingual — 中文 / English toggle on the page; all 7 arcs translated (structure-preserving).
2026-05-24	Emergent world-engine (Track B) — model invents the world live from a thin seed + world-state ledger; ?world=emergent.

Feedbacks

2026-06-19 — ZH (live test, via :zh-claw:)

[pasted an investigator excerpt — “调查员视角 · AI替代性 / 清华计算机系:…AI/算法岗竞争激烈度同比上升47%…需求增长32% / 北大数学系:…华尔街量化基金2025年数学系招聘量逆势增长21%”] “An example of a test i was in. Qs: • Where the data is extracted from? (sorry i didn’t save this session; or you regardless have record?) • How was the answer given, based on the data pulled?” → Honest finding: those specific numbers are model-fabricated, not extracted. Pipeline = 3 grounding layers (major_grounding KB + uni_employment 就业报告-distilled + live 雨来) + OpenRouter web plugin (≤4 snippets) → deepseek-v3 writes the prose; when it lacks a real figure it confabulates one with a credible-looking citation. Logged sessions show the pattern plainly: 对口率85%[admissions.hku.hk] with empty sources; 起薪中位数28万 / 35%倦怠 sourced to an invented “白皮书”; 家长满意度+12pp（阳光高考） — a metric 阳光高考 doesn’t publish. His exact session wasn’t saved/auto-logged. Same slop family as did_you_know / 35岁. Fix proposed: (a) numbers only from structured verified fields (uni_employment/雨来) with real 来源 + qualitative-only LLM prose, or (b) post-hoc number-guard. Awaiting go on (a).

2026-06-19 — ZH (live test, via :zh-claw:)

“来源:[object Object]、[object Object] 来源等你弄好links再更新吧？现在给我留个这样我没法给人测试，之前著名来源挺好的，先留下这个版本。” → pre-existing bug, not the link work. The websearch model sometimes returns 来源 as objects ({title,url,…}); old _auth_sources passed them straight into the frontend .join('、') → [object Object]. Fix on disk: _auth_sources normalizes objects to their title → 来源 always renders clean names; clickable {title,url,tier} links parked until URLs are backfilled, per ZH. New gaokao/sources.py. Live after a gaokao restart.

2026-06-18 — ZH (live test, via :zh-claw:)

“Let’s retire the 性别与行业 for the moment (and document in both log and scene building https://hub.meno.sh/projects/reflection-game/scenes), because for now we don’t know about gender. We may revive it later when we have more demographic info. To the same test link.” → retired gender_and_industry from the active WTE pool (kept in scenes.json with retired: true; ranked.py loader skips retired → 18→17 active), documented here + on the Scenes page, restarted the live service so it’s off the test link. 134e0ef.

2026-06-18 — ZH (live test, via :zh-claw:)

“After you done, maybe you can continue with the bot adjusting the order?” (quoting the eval caveat: 排序变化 0/24 是无头测产物,没模拟用户重排) → built a simulated player (gaokao/eval/player.py, PLAYER=1): an LLM acting AS the profile reorders its 志愿 ranking each stage like a real human. Ranking-shift went 0 → live (10/10 sessions reorder, 7/10 top-flip, 38 reranks on a 10×5 validation). Flagged: player is over-reactive (A→B→A oscillation, anchors on each stage’s spotlight instead of converging) → proposed tuning stickiness before a full-1K player pass. d45f0d7.

2026-06-18 — ZH (live test, via :zh-claw:)

“35岁 这个关键信息是怎么切入我们的志愿模拟的？我感觉它出现频率蛮高的，我不确定35岁是不是一个太强烈的stereotypy 和现实已经不符” → diagnosed: we seeded it ourselves. Not pulled per-major from a real source — build_grounding.py listed “35岁” as a literal example failure-mode in the generation prompt, so the model stamped 「35岁职业危机」 into 149/859 majors (17%) — and those 149 are exactly the high-traffic ones (计算机/软件/AI/金融/机械/土木/设计…), which is why it surfaces so often. The live 18-scene pool itself contains no “35岁”; the _FALLBACK_FACTORS career ask does (“35岁问题”) but that path is only a fallback. ZH is right it’s an over-applied stereotype. → fix proposed in-thread (drop the canned example + scrub bare 「35岁危机」 tags from majors where ungrounded; keep only a qualified/sourced form). Awaiting go.

2026-06-16 — Tianyi (relayed by ZH, verbatim)

1. 最开始的选择题或许可以改成选择+排序题（选出一个或多个再排列顺序），这样方便有超过一个想选项的情况
2. 每个专业内部也存在多种不同路径（搞绩点/竞赛？搞科研？搞学工？搞社团活动？），可以在开始的问题里面对这些不同路径也做选择+排序，后面的模拟可以有针对性
3. 毕业后薪资看起来比较miscalibrated，也没有就业率的信息；如果我们没有这方面的可靠数据的话，建议把就业相关的信息都删掉
4. 关于一些冷门一点的专业有点想当然（我问了北大元培，看起来AI觉得这是一个什么都学的混合专业，但其实只是第一年上通识、后三年并入选定的专业），结合上internet search agent可能会好点
5. 点击”看懂XXX”之后，加载出来的信息框闪了一下就消失了
6. 这一个环节(the bias page) 不太确定作用是什么？这三个到底是bias还是legit update似乎很难说；以及，三句话都可能被误以为是AI调查得到的结论，但我理解这是套模板得到的假设情景，可能有些误导性。

2026-06-18 — ZH (automated eval request, via :zh-claw:)

“Can we build a ton of profiles and do automatic testing? If so I want a batch of profiles (number unsure yet), ways to build diverse profiles (!!!), and testing results, and some meta analysis if possible.” → building an eval harness: diverse synthetic-profile generation (spanning family-pressure / self-clarity / risk / value-lean / region / 去向), a headless batch runner, and meta-analysis (dimension-rotation health, family-default rate, personalization, scene/dim coverage).

2026-06-18 — ZH (live test, via :zh-claw:)

“I cannot tell whether the new version is improved, as the entire game is centered around (i) 家庭期望 vs 自我期许 (as this is written in profile) (ii) 父母 in anyway they can attempt to influence my choice.” → diagnosed: the unified WTE re-picked family_expectation_vs_self as target_dim in 6/7 stages because examined human dimensions weren’t tracked (only scenes) → fixed to rotate across distinct dimensions (d8f8b0e, see Bugs).

2026-06-17 — ZH (architecture + docs, via :zh-claw:)

Architecture (open design): separate human ideal-preference dimensions (the belief state) from scenes (the 18 probes). The 不确定性 × 决策相关性 × 尚未检视 score should run over human dimensions, shared by WTE (pick the scene that best probes the top-VOI human dims) and investigator (probe that human dim; pick the contender that maximally discriminates on it — currently arbitrary). Wants: a human-dimensions data layer (analogous to scenes.json) + a scene→human-dim mapping; clarity on who writes the scene (today: the investigator’s prose, no separate story-writer). Highly uncertain we’re on the best approach — ideas welcome. → MenoClaw’s design proposal in-thread (belief-over-human-dims, scenes-as-probes, contender = argmax|Δ| on targeted dim, seed a small curated human-dim set + LLM-extend). Docs: stop referencing per-stage-anatomy from other pages (it’ll be retired gradually) — state needed things on the referencer’s page; prompts are not editable (drop that claim); on ideal-preference-dimensions, say what it is before what it is not. → all done (9826d9e).

2026-06-17 — ZH (live testing, via :zh-claw:)

The 「💡 你可能没考虑过」 line is broken on many levels — fabricated, unsourced slop (e.g. claiming 兰州大学 gives a broader network than PKU, with no source, never searched); every case looked nonsensical. Retire it entirely (the WTE decides what to examine now). Also retire 「看懂这个专业」 and 框架稳定性测试 for now (may revive grounded versions later). Earlier same session: the 看懂 button flashed-and-vanished (fixed); a transient /rank_translate 500 (now degrades gracefully).

2026-06-16 — Zhilin (relayed by ZH, verbatim)

1. ”🪞 你和AI共同理解的你”这部分好像不太影响后面出的题，至少我填入的（do the most good😂），基本没有在后面的题里面体现。我不确定目前背后是怎么设计的。我觉得这个也和tianyi说的第二点相关，就是到底想走什么路，其实也取决于自己的期待画像，能不能做点匹配和结合？
2. 我在” 你更想要哪种毕业去向?按想要程度点选排序(可多选;再点一次取消)“首选是创业，但后面的题似乎给我写的基本都是传统求职就业相关的～我整体上觉得首页的题的偏好对后续的题目生成影响比较低，不确定是什么问题。
3. 后面的题目似乎基本都是两两比较，所以我填的后三个专业没出现过。我不确定有没有可能在第一题的时候，如果填写者初始信心较低，可以把五种选择都生成一个可能场景，让学生感受一下？然后挑一个最感兴趣的深入？如果学生本身比较uncertain，感觉我们也保持开放一点更好？
4. 目前后面七道题，但感觉做起来同质化程度有点高，不清楚能不能让不同题目的variance大一点？

2026-06-16 — ZH (data-layer + dev-tools threads, via :zh-claw:)

A run of data-quality + scope pushes. Grounding still suboptimal — within-major paths (绩点/竞赛 · 科研 · 学工 · 社团) should be choosable & rankable at the start; wants career direction + salary per major/uni. Do everybody, not top-N unis. The 5-path preference set felt arbitrary (“think twice”). Salary looks miscalibrated + no 就业率 — double-check every source. Add a 保研/考研/出国/就业/考公考编/创业 opening questionnaire. Make all 高考 subpages bilingual (先中后英, per section) and prompts editable directly on the hub pages (dev edits drive the live game). Integrate 雨来 (partner data) + a 对接 subpage + investigator source-coordination. → Shipped: uni harvest · 毕业去向 questionnaire · bilingual subpages · 雨来 client. Pending: editable-prompts editor (queued); a valid 雨来 prod key.

2026-06-13 — ZH (via :zh-claw:)

The 高考 storylines are too generic (no per-major grounding), the profile is ignored, options are incoherent across A/B (CS/law leaking into a design-vs-materials run), and the canvas text is blurry. Asked for a full plan first. → Full response: Quality Overhaul Plan. Blur fixed; grounding layer in progress.

Developer feedback flows through the dev-feedback loop: the ?dev=1 button on these pages → developer/feedback/devfeedback.json (source of truth) → rendered here and on the dedicated subpage, with each item pinged to MenoClaw in real time and tracked open → resolved → killed.

→ Developer feedback table (live, auto-rendered from the JSON).

Player (in-game) feedback is logged per session via the in-game box → sessions/ jsonl + the /dev dashboard.

Bugs

Status	Bug	Notes
🟢 fixed (disk; needs restart)	来源 renders [object Object]	websearch LLM sometimes returns sources as objects; old _auth_sources passed them to the frontend .join → [object Object]. Fix: normalize → clean name strings (gaokao/sources.py, _auth_sources). Links deferred until URLs backfilled. Live after gaokao restart.
🟡 open (fix proposed)	Investigator fabricates statistics	deepseek-v3 invents precise figures (X%, 起薪中位数, 招聘+21%) + fake/credible citations despite the anti-fabrication prompt rule. Numbers are NOT in the grounding KB / 雨来 / web. Fix: (a) numbers only from structured verified fields w/ real 来源 + qualitative-only prose, or (b) post-hoc number-guard.
🟡 todo (small)	target_dim hygiene	1K run: 97% of stage target_dims are the 14 canonical human-dims, but 2.5% leak a scene key (ai_replaceability / postgraduate_and_barrier …) and 0.2% are zh/en variants of canonical keys. Cosmetic for the eval; fix = snap/validate target_dim to the 14 canonical keys (post-process or constrain unified_turn).
🟢 fixed	Simulated player over-reactive	The PLAYER=1 bot reordered too eagerly (A→B→A oscillation). Fixed (v2, af4d307): materially-new-info gate (default=keep), told its last move so it won’t flip back, temp 0.5→0.3, + hard anti-revert guard (no returning to a visited order). Same 10×5 slice: oscillating sessions 9/10→0/10, reranks 38→20, still responsive (10/10 move, top-flip 6/10 — now real convergent moves).
🟢 fixed	「35岁」over-seeded stereotype	ZH noticed 35岁 surfaced too often. Root cause: build_grounding.py gave the generator 「35岁」 as an example failure-mode → baked into 150 tags across 149/859 majors (the popular ones), not independently sourced. ZH’s principle: investigator-surfaced evidence is fine, a canned prompt example is not (it biases generation). Fix (9fa674c): dropped 「35岁」 from the generation prompt (+ no-meme guard) and the _FALLBACK_FACTORS career ask; scrubbed all 150 baked 「35岁危机」 tags (149 failure_modes + 1 reputation_note) → 0 remaining. Age-ceiling concerns now only surface from live investigator evidence per-major.
🟢 fixed	WTE stuck on one human dimension	ZH’s 6/18 test ran 6/7 stages on family_expectation_vs_self → every scene framed around 父母. unified_turn tracked examined scenes not examined human dims; now threads examined dims + instructs rotation + no family-default. Verified 5 distinct dims/5 stages. d8f8b0e.
🔵 design / proposed	Human-dim vs scene-dim split + shared VOI	belief/uncertainty-map over human ideal-preference dimensions (not scenes); WTE picks scene that best probes top-VOI human dims; investigator probes that dim, contender = argmax|Δdim| vs #1. Awaiting ZH go to build the human-dimensions data layer + refactor.
🟡 todo	Human-dimensions data layer	human_dimensions.json (ideal-preference axes, seeded from 叶晓阳 checklist + 8-pole model + named tensions) + scene→human-dim mapping.
🟡 todo	Story-writer not separate	the scene prose is written inside investigate(); no distinct story-writer module yet (hence not on the hub). Possible future split.
🟢 fixed	Docs: per-stage refs / editable claim / ordering	stopped referencing per-stage-anatomy from investigator; dropped ‘prompts editable’; ideal-pref-dims now what-it-is-first. 9826d9e.
🗑️ retired	「💡 你可能没考虑过」(did_you_know)	fabricated unsourced comparative claims; removed from the stage UI + investigate prompt (e7353fa). The WTE-chosen question + top/contender takes carry the investigation now.
🗑️ retired (revivable)	「看懂这个专业」button	retired for now; endpoint /rank_translate left dormant for a future grounded version. e7353fa.
🗑️ retired (revivable)	框架稳定性测试 (frame-test)	retired; last stage now goes straight to the ending. /rank_frametest dormant. e7353fa.
🟢 fixed	看懂 flash-and-vanish + transient 500	_d(false) so the panel isn’t self-cleared; gen-failures return 200-with-error not 500. d750316/01040a9.
🟢 fixed / live	雨来 partner API now live	valid yl_live_… key in menoclaw/.env (2026-06-19); verified enabled, major_detail returns evidenceStatus: verified. Runtime yulai_major_block active; also fixed a key-name bug there (红线→redLine). Static major_grounding.json enrichment (authoritative courses/分类/误区/红线 + provenance→雨来核验) in progress.
🟢 fixed	Salary miscalibrated + 就业率 missing	distilled major-level first_job_salary_rmb_monthly dropped; grounding now surfaces real 就业质量报告 uni-level 就业率/升学率/出国率/起薪 with 口径 (lookup_uni/uni_employment_block, 2121 unis, honest-null). Commit 7cb8bb0. Limit: salary is 全校/学院级 (691 unis), per-major stays qualitative.
🟡 queued	Editable prompts on hub	a dev-facing editor on the 高考 pages whose edits drive the live game — externalize gaokao prompts → editor → live. Not built yet.
🟢 fixed	Blurry canvas text (人生A / 第N幕 + prose)	backing store now scales by devicePixelRatio (536e035).
🟢 fixed (v1)	高考 storylines too generic — no per-major grounding	grounding layer shipped (50 majors); scenes use real specifics. doc. Real-report validation pending.
🟡 open	Option incoherence — options not locked to chosen major/identity; domains leak across A/B	per-side identity lock + option-validity gate; plan §3.
🟢 fixed	Emergent turn-1 returned truncated/invalid JSON	V4-Pro reasoning ate max_tokens=2000; raised to 8000.
🟢 fixed	Dev-feedback button silently failed from Netlify PR previews	receiver CORS/validation rejected preview origins; now accepts *.netlify.app.
🟢 fixed	Gaokao pipeline KeyError when a life-thread hit a 429	run_trajectory now retries API errors w/ backoff; pipeline tolerates a failed life.
🟡 open	Latency — emergent / 高考 sessions ~2 min (V4-Pro ~50s/stage; V3 ~19s)	provider throttles concurrent reasoning calls; needs caching + higher API tier.
🟡 open	Live demo links are ephemeral (cloudflared quick-tunnels) + the 高考 link is un-gated	harden before wider sharing (named tunnel + a gate).
🟡 open	Multi-turn storyline coherence verified only on short runs	full 6-stage × 2-life sessions need human playtesting.