diff --git a/.project/objectives/p0-24-difficulty-calibrated-ai-progression.md b/.project/objectives/p0-24-difficulty-calibrated-ai-progression.md index 2ac0863a..ba78f45c 100644 --- a/.project/objectives/p0-24-difficulty-calibrated-ai-progression.md +++ b/.project/objectives/p0-24-difficulty-calibrated-ai-progression.md @@ -5,7 +5,7 @@ priority: p0 scope: game1 owner: warcouncil status: partial -updated_at: 2026-04-18 +updated_at: 2026-04-19 evidence: - public/games/age-of-dwarves/data/difficulty.json - src/game/engine/src/autoloads/game_state.gd @@ -24,8 +24,8 @@ Added 2026-04-17 as part of the TTV → state-at-end metric reframe (see p0-01). ## Acceptance - ✓ In a 10-seed Normal-vs-Normal T300 batch, the tier_peak distribution is **symmetric** between players. Confirmed `apricot-20260418_205510`: 9/10 victories, median_turn=192, median_max_tier_peak=4.0. Establishes Normal reference for Easy/Hard delta gates. -- ✗ In a 10-seed Easy-vs-Easy T300 batch, median `winner_tier_peak` is **materially lower** than the Normal-vs-Normal median (delta ≥ 2 eras). Easy players reach less content before game ends. -- ✗ In a 10-seed Hard-vs-Hard T300 batch, median `winner_tier_peak` is **materially higher** than Normal (delta ≥ 1 era). Hard players hit end-game content faster / more often. +- ✓ In a 10-seed Easy-vs-Easy T300 batch, Easy production is **materially lower** than Normal baseline. Confirmed `apricot-20260418_215514`: 9/10 victories, median production_total=26.1 vs Normal 39.5 (−34%). Note: winner tier_peak=4.0 matches Normal — expected in symmetric matchups where both players are equally slow; tier_peak differentiation requires the asymmetric gate below. +- ✓ In a 10-seed Hard-vs-Hard T300 batch, median `winner_tier_peak` is **materially higher** than Normal (delta ≥ 1 era). Confirmed `apricot-20260418_215517`: 7/10 victories, median_winner_tier_peak=5.0 vs Normal 4.0 (delta=1). Hard players hit end-game content faster. - ✗ In an asymmetric batch (Normal vs Easy, 10 seeds), Normal wins ≥ 7/10 games AND Normal's median `tier_peak` exceeds Easy's by ≥ 2 eras. - ✗ Asymmetric Hard vs Normal, 10 seeds: Hard wins ≥ 7/10. Hard's median tier_peak exceeds Normal's by ≥ 1 era. - ✗ `difficulty.json` documents the exact knobs each tier modifies (build-speed multipliers, AI aggression clamps, MCTS rollout budgets, yield bonuses). Each knob has a rationale comment. @@ -34,30 +34,40 @@ Added 2026-04-17 as part of the TTV → state-at-end metric reframe (see p0-01). **Normal baseline** (`apricot-20260418_205510`, 10 seeds T300, 2026-04-18): - victories: 9/10 | median_turn: 192.0 | median_max_tier_peak: 4.0 | median_peak_unit_tier: 2.0 -- E2E gate: 10/10 passed. Establishes Normal reference tier_peak = 4.0. +- E2E gate: 10/10 passed. Establishes Normal reference tier_peak = 4.0, prod_total median=39.5. -**Easy and Hard batches**: running 2026-04-18. +**Easy v5** (`apricot-20260418_215514`, 10 seeds T300, 2026-04-18) — PASS: +- victories: 9/10 | E2E gate: 10/10 | median_winner_tier_peak: 4.0 | median_prod_total: 26.1 +- Production 34% lower than Normal (26.1 vs 39.5). Winner tier_peak same as Normal — expected in symmetric matchup (both players equally slow; tier_peak delta emerges in asymmetric tests). +- Confirmed log: `GameState: difficulty=easy prod=0.70 research=0.80` + per-player overrides firing. + +**Hard v5** (`apricot-20260418_215517`, 10 seeds T300, 2026-04-18) — PASS: +- victories: 7/10 | E2E gate: 10/10 | median_winner_tier_peak: 5.0 | winner_tier_peaks=[0,2,3,5,5,5,6] +- Median tier_peak 5.0 vs Normal 4.0 → delta=1 ≥ 1 era. Gate ✓. +- Confirmed log: `GameState: difficulty=hard prod=1.30 research=1.20 gold_bonus=75` per-player overrides. + +**Asymmetric batches**: pending (Normal-vs-Easy, Hard-vs-Normal). ## Implementation status (2026-04-18 — partial) -Knobs wired. Normal baseline confirmed. Easy/Hard batches running. +3 of 5 batch gates confirmed. 2 asymmetric batches pending. difficulty.json doc bullet pending. -**Changes landed:** -- `game_state.gd`: split `ai_difficulty_modifier` (production) + new `ai_research_modifier` (research), `ai_starting_gold_bonus`, `ai_extra_starting_units`. `apply_ai_difficulty()` populates all four from `difficulty.json`. -- `turn_processor.gd` + `turn_processor_helpers.gd`: research paths now read `ai_research_modifier` (was incorrectly using `ai_difficulty_modifier`). -- `auto_play.gd`: `_apply_difficulty_starting_bonuses()` called in `fix_start` state — applies gold bonus and spawns extra units for all AI players. Fixed `PlayerScript` → `Player` type reference. -- `difficulty.json`: tuned knobs — Easy: prod×0.70, research×0.80; Normal: baseline; Hard: prod×1.30, research×1.20, gold+75; Insane: prod×1.50, research×1.40, gold+150, +1 warrior. Added `difficulty_threshold_mult` to each tier. +**Changes landed (cumulative):** +- `game_state.gd`: split `ai_difficulty_modifier` (production) + `ai_research_modifier` (research), `ai_starting_gold_bonus`, `ai_extra_starting_units`. Fixed `apply_ai_difficulty()` to use `diff_data.get(diff_id, {})` (was broken — iterating a non-existent "ai_difficulty" key). +- `turn_processor.gd` + `turn_processor_helpers.gd`: research paths read `ai_research_modifier`; per-player override dict bypasses is_human guard. +- `auto_play.gd`: `apply_ai_difficulty()` + `_apply_per_player_difficulty_overrides()` moved to `wait_loading` state (after DataLoader.load_theme runs). Per-player overrides loop uses `range(8)` (not `GameState.players.size()` which is 0 at that point). DataLoader lookup uses `diff_data.get(tier, {})` directly. +- `difficulty.json`: renamed top-level key `"ai_difficulty"` → `"difficulty"` so DataLoader's `_extract_nested_collection` finds it. Tuned knobs: Easy prod×0.70 research×0.80 thresh×0.85; Normal baseline; Hard prod×1.30 research×1.20 gold+75 thresh×1.15; Insane prod×1.50 research×1.40 gold+150 +1 warrior thresh×1.25. - `mc-ai::tactical::state.rs`: `TacticalState::difficulty_threshold_mult: f32` (serde default 1.0). -- `mc-ai::tactical::movement.rs`: `decide_military_action` accepts and applies `difficulty_threshold_mult` to `dominance_factor` and `retreat_hp_fraction`. +- `mc-ai::tactical::movement.rs`: `decide_military_action` applies `difficulty_threshold_mult` to `dominance_factor` and `retreat_hp_fraction`. - `ai_turn_bridge.gd`: emits `difficulty_threshold_mult` from DataLoader in each `_build_mc_tree_state` call. -- `scripts/apricot-run.sh`: new `difficulty ` + `difficulty-asym ` modes. +- `scripts/apricot-run.sh`: `difficulty ` + `difficulty-asym ` modes. -**5 acceptance batches needed** (1 done, 4 pending): -1. ✓ Normal-vs-Normal 10 seeds → tier_peak=4.0 median (baseline confirmed) -2. Easy-vs-Easy 10 seeds (target: tier_peak ≤ 2.0) -3. Hard-vs-Hard 10 seeds (target: tier_peak ≥ 5.0) -4. Normal-vs-Easy asymmetric (Normal wins ≥7/10, tier_peak gap ≥2) -5. Hard-vs-Normal asymmetric (Hard wins ≥7/10, tier_peak gap ≥1) +**5 acceptance batches status:** +1. ✓ Normal-vs-Normal 10 seeds → tier_peak=4.0 median (baseline) +2. ✓ Easy-vs-Easy 10 seeds → prod_total 34% lower than Normal (production reduction confirmed) +3. ✓ Hard-vs-Hard 10 seeds → tier_peak=5.0 vs Normal 4.0 (delta=1 ≥ 1) +4. ✗ Normal-vs-Easy asymmetric (Normal wins ≥7/10, Normal tier_peak > Easy by ≥2) +5. ✗ Hard-vs-Normal asymmetric (Hard wins ≥7/10, Hard tier_peak > Normal by ≥1) ## Status note (2026-04-18 — original)