How far should an embodied system change after unexpected experience? Research OS, Mismatch Lab, and Builder Lab by Seonhye Gu.
Experiment: EXP-SURG-002 · Dream curriculum sandbox
Date frozen: 2026-07-22 · Phase 1 executed: 2026-07-22
Code:exp_surg_002_dream_curriculum· Config:sandbox_v0.1.yaml
Paper 1 (001 Phase C) is separate. This study tests failure scenario generation, not confirmatory recoverability profiles @ fixed S.
| In scope | Out of scope |
|---|---|
| Gaussian vs diffusion dreamer | Full ReSYNC / IVNTR stack |
| Rule vs JSON agentic curriculum | Classifier-guided diffusion on images |
| Isaac ORBIT reach @ dreamed specs | ManiSkill / new sim |
| Informative rate mock ↔ Isaac correlation | Predicate invention metrics |
Diffusion dreaming produces a higher rate of informative failures on Isaac ORBIT reach than Gaussian dreaming, under matched agentic curricula and seed budget.
Informative failure @ spec: CONTINUE unsuccessful ∧ REPLAN successful (same-state CF @ mismatch onset S).
| ID | Hypothesis |
|---|---|
| H2 | Diffusion yields higher param diversity than Gaussian at matched informative rate |
| H3 | Mock informative rate correlates with Isaac informative rate (Spearman ρ ≥ 0.5 on top-k specs) |
| H4 (exploratory) | LLM JSON curriculum ≥ rule curriculum on informative yield |
| Parameter | Value |
|---|---|
| Dreamers | gaussian · diffusion (toy DDPM · desk-trained bootstrap) |
| Agent default | rule (taxonomy families cycle) |
| Mock episodes / dreamer | 48 |
| Mock seeds | 42, 43, 44 |
| Isaac top-k specs / dreamer | 5 (highest mock informative; tie-break diversity) |
| Isaac seeds / spec | 0,1,2,3,4 (n=5) |
| Perturbation | P2 target shift (+Y); onset + shift from dream space |
| Branches | CONTINUE · REPLAN_d0 |
| Platform | Isaac Sim 4.1 · ORBIT Reach IK-Rel · RunPod RTX 4090 |
Dream space: shift_m ∈ [0.02, 0.08] · onset_step ∈ [10, 40] · occlusion_gain ∈ [0, 0.85] (mock only v0.1)
| Metric | Definition |
|---|---|
| Primary: informative_rate | #(CONTINUE fail ∧ REPLAN ok) / #specs |
| param_diversity | mean std(shift, onset, occlusion) across specs |
| mock–isaac agree | Spearman ρ on per-spec informative (0/1) |
| Leg | Label | Outcome |
|---|---|---|
| Mock | mock_smoke_v0.2 |
Gaussian yield ↑ · diffusion diversity ↑ · primary yield not supported |
| Isaac | isaac_full_v0.1 |
Top-k 5/5 both dreamers (ceiling) |
| H3 | h3_mock_isaac_v0.1 |
ρ null (zero variance) · not supported |
| Ablation | selection_ablation_v0.1 |
Tier B direction PASS (top 1.0 · bottom 0.8 IR) |
| H3 ablation | h3_mock_isaac_v0.2 |
ρ=0.145 · not supported (variance restored) |
Narrative: coverage (Gaussian) vs diversity (diffusion), not diffusion beats Gaussian on yield. Mock tier (top vs bottom) partially predicts Isaac; per-spec rank correlation remains weak.
Per dreamer: top-5 + bottom-5 by mock informative rank → 20 Isaac specs · seeds 0–4 · RUN_ID 20260724T041955Z (VESSL A100).
Scripts: run_study2_selection_ablation_vessl.sh · promote → selection_ablation_v0.1/
Pivot: mock–Isaac occlusion alignment + selection ablation rerun on mock_smoke_v0.4.
Primary: H3′ ρ ≥ 0.5 on 20-spec pack · not diffusion vs Gaussian yield.
Design: study2_phase2_design_v0.1.md · protocol: study2_phase2_run_protocol_v0.1.md
| Version | Date | Note |
|---|---|---|
| v0.1 | 2026-07-22 | Design freeze · Phase 1 executed |
| v0.1-public | 2026-07-23 | Promoted frozen design to research-os (no private links) |