How far should an embodied system change after unexpected experience? Research OS, Mismatch Lab, and Builder Lab by Seonhye Gu.
ARCHIVED · mock→physics direction · superseded 2026-07-29 · do not cite or extend Current Paper 002: WM expansion · archive index
Supersedes: v0.1 · Pre-reg: paper002_prereg_v0.3.md
| Analysis | Mock | Isaac |
|---|---|---|
| H1 · H3 rank transfer | (\bar M_i) continuous median | (I_i) continuous median |
| H2 enrichment | — | (Y_i) binary majority |
Evaluated after both planner Isaac legs complete (or rule-only if operational gate fails):
H1 (rule n=20) → H2 (rule n=20) → H3 (LLM vs rule)
Not used to gate LLM Isaac execution. Operational continuation: paper002_operational_gate_v0.1.md.
rho, reason = spearman(M_bar, I_continuous)
ci_lo, ci_hi = bootstrap_spearman(M_bar, I_continuous, B=2000, seed=20260728)
p_perm = permutation_spearman(M_bar, I_continuous, B=1000, seed=20260728)
Pass: rho ≥ 0.50 ∧ ci_lo > 0.25.
Per tier (top n=10 · bottom n=10):
Pass (all required):
Supporting (report · not pass criterion):
| k / n=10 | One-sided exact p |
|---|---|
| 8 | 0.0547 |
| 9 | 0.0107 |
8/10 passes H2 via rate thresholds; binomial p = 0.0547 is reported as borderline supporting evidence.
Evaluated after rule + LLM Isaac complete · not gated by H1/H2 at rule leg.
Δρ = ρ_LLM − ρ_rule
Bootstrap paired/stratified preserving planner identity (B=2000):
Pass H3b: CI_lower(Δρ) > −0.15
| ID | Variant |
|---|---|
| S1 | Seed-43 export only |
| S2 | Binary mock rank |
| S3 | Kendall τ |
| S4 | Pooled 40-candidate ρ (descriptive) |
| S5 | Middle-tier 10 candidates (if run) |
| Script | Role |
|---|---|
merge_study2_mock_consensus.py |
Five-seed median |
export_study2_isaac_specs.py |
Planner-scope export |
compute_study2_h3_mock_isaac.py |
ρ · bootstrap · tiers |
Extend compute script with --continuous (default in v0.3).
{
"prereg_version": "v0.3",
"mock_score_type": "cf_margin_median",
"export_scope": "planner",
"spearman_rho": 0.0,
"bootstrap_ci_95": [0.0, 0.0],
"h1_pass": false,
"h2_pass": false,
"h3_pass": false
}