diff --git a/.project/handoffs/20260425_warcouncil-cycle1-batches.md b/.project/handoffs/20260425_warcouncil-cycle1-batches.md index a1f5a0f7..1ed1cbca 100644 --- a/.project/handoffs/20260425_warcouncil-cycle1-batches.md +++ b/.project/handoffs/20260425_warcouncil-cycle1-batches.md @@ -100,14 +100,20 @@ After all three batches land successfully and status updates are committed, run - **Correct binary hash** (deployed to addon, confirmed `set_budget_ms` registered): `0d127464096539475ae7fd9786eab8af545aeec4a39900234c3a4dfe5e9f07d7` - **TODO for build infra**: `build-gdext.sh:find_build` should use `$CARGO_TARGET_DIR` when set, not hardcode `.local/build/rust/` -### Step 2: p1-22 batch — IN PROGRESS +### Step 2: p1-22 batch — IN PROGRESS (third attempt) -- Aborted first attempt (stale binary, `p1-22-budget-20260425_174041`) — only 2 seeds ran, all with `set_budget_ms` errors. Results invalid. -- Fresh batch launched: tmux session `p1-22-batch2` on apricot -- Output dir: `.local/iter/p1-22-budget-20260425_180000` -- Started: 2026-04-25 17:44 PDT -- Poll: monitor running locally, checks every 30 min for `/tmp/p1-22-done.flag` -- Pass criteria: ≥5/10 victories, ≥2 distinct winners +History: +1. **First attempt** (`p1-22-budget-20260425_174041`) — aborted, stale binary, `set_budget_ms` errors on every AI turn. Results invalid. +2. **Second attempt** (`p1-22-budget-20260425_180000`) via tmux session `p1-22-batch2` — tmux server died mid-run on multi-tenant apricot before any seeds completed. Seeds 1+2 ran in isolation; orchestrator gone, no further seeds dispatched. Killed and discarded. +3. **Third attempt (current)** — `nohup` launch (no tmux), parent PID 1237118 on apricot. Output dir `.local/iter/p1-22-budget-20260425_180742`. Started 2026-04-25 18:07 PDT. + - Pass criteria: ≥5/10 victories, ≥2 distinct winners + - Monitor: `/tmp/p122_status.py`, polled every 5 min from local + - Status flag: `/tmp/p1-22-done.flag` written by `/tmp/p1-22-launch.sh` after `huge-map-5clan.sh` exits + +Lessons learned (recorded for future coordinators): +- **Don't use tmux on multi-tenant apricot for long-running batches** — server can die unrelated to your job. Use `nohup ... log 2>&1 & disown` instead. +- **`build-gdext.sh` ignores `CARGO_TARGET_DIR`** — see Step 1 ISSUE FOUND. After running with `CARGO_TARGET_DIR=/tmp/...`, always `sha256sum` the deployed addon `.so` against the build's `.so` and `cp` manually if they differ. +- **Monitor inline-python via SSH gives stale/cached results** — write the script to a file on apricot first, then `ssh apricot python3 /tmp/script.py`. Direct evaluation through nested quoting in monitor commands produces unreliable readings. ### Steps 3-4: p0-02 + p0-01 — PENDING p1-22 completion