fix(infra): make the DO fleet actually work on real hardware + render host
Real-DO testing surfaced bugs the mocked tests couldn't: - ssh key: reference shared 'mc-fleet' key via data source, not a duplicate (DO 422s on dup pubkeys). - cmd_dist_up: fail loudly on failed apply; dist:up waits for cloud-init readiness. - snapshot cloud-init skips runcmd -> bake authorized_keys (FLEET_PUBKEY) + 'cloud-init clean' before snapshot. - build user passwordless sudo; apt dpkg-lock race fixed (cloud-init --wait + Lock::Timeout). - size s-8vcpu-16gb-amd (tier max); creds via PKR_VAR env not argv. - render host: weston+Mesa baked; ./run dist:render proven (Godot->PNG on DO, no GPU). forge:dns shortcut. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Natalie
parent
a5d66ce477
commit
6332d47011
12 changed files with 346 additions and 29 deletions
|
|
@ -1,5 +1,5 @@
|
|||
{
|
||||
"generated_at": "2026-06-27T13:41:30Z",
|
||||
"generated_at": "2026-06-27T13:58:02Z",
|
||||
"totals": {
|
||||
"done": 298,
|
||||
"in_progress": 0,
|
||||
|
|
|
|||
|
|
@ -32,13 +32,18 @@ revealed three SOLID/DRY/DIP debts. "Foundation first" tackled the layering + ph
|
|||
`res://` bytes across FFI, the web guide fetches packs → bytes across bindgen — with
|
||||
`include_str!` surviving **only** as the headless/test fallback. So the target is a registry
|
||||
fed by injected bytes, not a path-reading function.
|
||||
- [ ] **Rail-2 verify gate (enforcement) — catch the next hardcode before it ships.** The gate
|
||||
has a Rail-1 check (`verify.sh` Step 18 → `tools/check-no-gdscript-sim-logic.py`) but **no Rail-2
|
||||
equivalent** for hardcoded game content in Rust. Add `tools/check-no-rust-hardcoded-content.py` +
|
||||
a verify step that flags balance-looking `const`/`static` tables in `mc-*` crates which duplicate
|
||||
a JSON file. ⚠ heuristic — needs a low-false-positive design (likely a small allowlist/registry of
|
||||
`(json_file, owning_module)` pairs rather than a blind numeric-array grep). Best landed alongside
|
||||
the `ContentRegistry` so the check becomes "does the registry own this?", not a regex.
|
||||
- [x] **Rail-2 verify gate (enforcement, v1) — catch the next hardcode before it ships.** ✅
|
||||
2026-06-27 — `tools/check-no-rust-hardcoded-content.py` + `verify.sh` Step 19 (parallel to the
|
||||
Rail-1 Step 18 gate). Registry-driven, **zero false-positive** by design: a manifest of
|
||||
`(json_file, owning_module(s), tombstones)` enforces (A) each registered content file stays
|
||||
`include_str!`-loaded by its owner, and (B) tombstoned const names (e.g. promotions
|
||||
`XP_THRESHOLDS`/`HEAL_ON_PROMOTE_FRACTION`) never resurrect. **Deliberately NOT** a heuristic
|
||||
numeric-const grep — that wrongly flags legit sim-tuning consts (`MIGRATION_RATE`,
|
||||
`DROUGHT_CARRYING_PENALTY` in `mc-ecology/generation.rs`). Proven: passes clean, fails on an
|
||||
injected `XP_THRESHOLDS` revert. **Limitation:** coverage is opt-in (6 files registered today) —
|
||||
a brand-new hardcode in an unregistered module isn't caught. The blanket guarantee is the
|
||||
`ContentRegistry`'s job: when it lands, fold this check into "does the registry own this?" rather
|
||||
than a manifest. Until then, grow `REGISTRY` as content modules are identified.
|
||||
- [ ] **Dedup the ad-hoc `include_str!` content sites (Opportunity A, same arc)** — verified
|
||||
2026-06-27: **8 JSON-config `include_str!` sites** across simulator crates, each rolling its
|
||||
own `OnceLock` + a fragile relative path (6 at `../../../../../`). `treaty_rules.json` is
|
||||
|
|
|
|||
|
|
@ -31,11 +31,12 @@ variable "region" {
|
|||
default = "nyc3"
|
||||
}
|
||||
|
||||
# A one-off CPU-Optimized box builds fast (cargo + godot import are CPU-heavy);
|
||||
# it only exists for the duration of the build.
|
||||
# A one-off box builds the image (cargo + godot import are CPU-heavy); it only
|
||||
# exists for the build. Basic s-* — CPU-Optimized c-* is tier-restricted on new
|
||||
# DO accounts (needs a support ticket to unlock).
|
||||
variable "build_size" {
|
||||
type = string
|
||||
default = "c-8"
|
||||
default = "s-8vcpu-16gb-amd"
|
||||
}
|
||||
|
||||
variable "git_remote" {
|
||||
|
|
@ -52,6 +53,13 @@ variable "remote_user" {
|
|||
default = "mc"
|
||||
}
|
||||
|
||||
variable "fleet_pubkey" {
|
||||
type = string
|
||||
default = ""
|
||||
# The fleet SSH public key, baked into the build user's authorized_keys so the
|
||||
# dispatch can ssh in without relying on cloud-init's (snapshot-flaky) key copy.
|
||||
}
|
||||
|
||||
locals {
|
||||
ts = formatdate("YYYYMMDDhhmmss", timestamp())
|
||||
}
|
||||
|
|
@ -73,6 +81,7 @@ build {
|
|||
"GIT_REMOTE=${var.git_remote}",
|
||||
"GIT_REF=${var.git_ref}",
|
||||
"BUILD_USER=${var.remote_user}",
|
||||
"FLEET_PUBKEY=${var.fleet_pubkey}",
|
||||
]
|
||||
execute_command = "chmod +x {{ .Path }}; env {{ .Vars }} bash {{ .Path }}"
|
||||
script = "${path.root}/provision.sh"
|
||||
|
|
|
|||
|
|
@ -17,8 +17,11 @@ REPO_PATH="Code/@projects/@magic-civilization" # relative to the build user's HO
|
|||
|
||||
echo "=== [1/7] base packages (incl. software-render stack: weston + Mesa llvmpipe) ==="
|
||||
export DEBIAN_FRONTEND=noninteractive
|
||||
apt-get update -y
|
||||
apt-get install -y --no-install-recommends \
|
||||
# Wait for cloud-init's own boot-time apt to finish, then take the dpkg lock with
|
||||
# a timeout (avoids the first-boot lock race that aborts provisioning).
|
||||
cloud-init status --wait >/dev/null 2>&1 || true
|
||||
apt-get -o DPkg::Lock::Timeout=600 update -y
|
||||
apt-get -o DPkg::Lock::Timeout=600 install -y --no-install-recommends \
|
||||
git curl ca-certificates build-essential pkg-config libssl-dev \
|
||||
unzip sudo python3-pip flatpak rsync \
|
||||
weston libgl1-mesa-dri libegl1 libgles2 libwayland-egl1 \
|
||||
|
|
@ -26,10 +29,29 @@ apt-get install -y --no-install-recommends \
|
|||
# So every worker can render proof scenes (opengl3/gl_compatibility) under a
|
||||
# headless weston with no GPU — see tools/capture-proof.sh + tools/autoplay-batch.sh --weston.
|
||||
|
||||
echo "=== [1b/7] swapfile (8 GB box needs headroom for the Rust link step) ==="
|
||||
if ! swapon --show 2>/dev/null | grep -q /swapfile; then
|
||||
fallocate -l 4G /swapfile && chmod 600 /swapfile && mkswap /swapfile >/dev/null && swapon /swapfile
|
||||
grep -q '/swapfile' /etc/fstab || echo '/swapfile none swap sw 0 0' >> /etc/fstab
|
||||
fi
|
||||
|
||||
echo "=== [2/7] build user '$BUILD_USER' ==="
|
||||
if ! id "$BUILD_USER" >/dev/null 2>&1; then
|
||||
useradd --create-home --shell /bin/bash "$BUILD_USER"
|
||||
fi
|
||||
# Passwordless sudo: scripts/dev-setup/linux.sh installs system packages (node, etc.)
|
||||
# via sudo apt-get; the build user must be able to run them non-interactively.
|
||||
echo "$BUILD_USER ALL=(ALL) NOPASSWD:ALL" > "/etc/sudoers.d/90-${BUILD_USER}"
|
||||
chmod 440 "/etc/sudoers.d/90-${BUILD_USER}"
|
||||
# Authorize the fleet key for the build user directly — the dispatch ssh's in as
|
||||
# this user, and cloud-init's key-copy runcmd is unreliable on snapshot-booted
|
||||
# droplets. Baking it means ssh works the moment sshd is up.
|
||||
if [ -n "${FLEET_PUBKEY:-}" ]; then
|
||||
install -d -m 700 -o "$BUILD_USER" -g "$BUILD_USER" "/home/$BUILD_USER/.ssh"
|
||||
printf '%s\n' "$FLEET_PUBKEY" > "/home/$BUILD_USER/.ssh/authorized_keys"
|
||||
chmod 600 "/home/$BUILD_USER/.ssh/authorized_keys"
|
||||
chown "$BUILD_USER:$BUILD_USER" "/home/$BUILD_USER/.ssh/authorized_keys"
|
||||
fi
|
||||
BUILD_UID="$(id -u "$BUILD_USER")"
|
||||
# Enable lingering so /run/user/$UID (and the user D-Bus flatpak needs for
|
||||
# headless --import) exists without an interactive login.
|
||||
|
|
@ -70,4 +92,10 @@ as_user "mkdir -p ~/bin && cp ~/$REPO_PATH/scripts/autoplay/run_ap3.sh ~/bin/run
|
|||
as_user "cd ~/$REPO_PATH && flatpak run --user org.godotengine.Godot --path src/game --headless --import" || \
|
||||
echo "WARN: headless --import did not complete cleanly — validate in the live smoke"
|
||||
|
||||
echo "=== reset cloud-init so fleet workers boot FRESH ==="
|
||||
# Snapshot-booted droplets otherwise carry this build box's 'already ran' state and
|
||||
# skip runcmd (key copy + git pull). Cleaning makes the new worker run cloud-init
|
||||
# fully on first boot.
|
||||
cloud-init clean --logs 2>/dev/null || true
|
||||
|
||||
echo "=== golden image provisioned OK ==="
|
||||
|
|
|
|||
|
|
@ -2,9 +2,10 @@
|
|||
# No persistent volume: workers are stateless. The golden image carries the warm
|
||||
# clone + toolchain + prebuilt .so; results leave via the dispatch layer (scp).
|
||||
|
||||
resource "digitalocean_ssh_key" "fleet" {
|
||||
name = "${var.name}-key"
|
||||
public_key = file(pathexpand(var.ssh_public_key_path))
|
||||
# Reuse the already-registered MC fleet key (shared with the forge + packer) rather
|
||||
# than creating a duplicate — DO rejects a second key with the same public_key.
|
||||
data "digitalocean_ssh_key" "fleet" {
|
||||
name = var.ssh_key_name
|
||||
}
|
||||
|
||||
# Resolve the newest golden image by name substring. Skipped entirely when
|
||||
|
|
@ -34,7 +35,7 @@ resource "digitalocean_droplet" "worker" {
|
|||
size = var.size
|
||||
region = var.region
|
||||
image = local.image
|
||||
ssh_keys = [digitalocean_ssh_key.fleet.id]
|
||||
ssh_keys = [data.digitalocean_ssh_key.fleet.id]
|
||||
|
||||
# Thin cloud-init: copy the injected key to the build user and fast-forward
|
||||
# the warm clone to the requested ref. The golden image already holds the
|
||||
|
|
|
|||
|
|
@ -15,9 +15,8 @@ mock_provider "digitalocean" {
|
|||
}
|
||||
|
||||
variables {
|
||||
do_token = "mock-token-unused"
|
||||
git_remote = "https://example.com/magic-civilization.git"
|
||||
ssh_public_key_path = "./tests/fixtures/id_test.pub"
|
||||
do_token = "mock-token-unused"
|
||||
git_remote = "https://example.com/magic-civilization.git"
|
||||
}
|
||||
|
||||
# base_image set -> golden data source is skipped (count 0); fleet expands to N.
|
||||
|
|
|
|||
|
|
@ -37,7 +37,7 @@ variable "size" {
|
|||
c-8 = 8 vCPU / 16 GB, c-16 = 16 vCPU / 32 GB
|
||||
EOT
|
||||
type = string
|
||||
default = "s-8vcpu-16gb"
|
||||
default = "s-8vcpu-16gb-amd"
|
||||
}
|
||||
|
||||
variable "base_image" {
|
||||
|
|
@ -58,10 +58,10 @@ variable "golden_name_match" {
|
|||
default = "mc-golden"
|
||||
}
|
||||
|
||||
variable "ssh_public_key_path" {
|
||||
description = "Public key authorised on every worker (and used by the dispatch scripts to ssh in)."
|
||||
variable "ssh_key_name" {
|
||||
description = "Name of the pre-registered DigitalOcean SSH key the fleet reuses (shared with forge + packer; the dispatch ssh's in with its private half)."
|
||||
type = string
|
||||
default = "~/.ssh/id_ed25519.pub"
|
||||
default = "mc-fleet"
|
||||
}
|
||||
|
||||
variable "name" {
|
||||
|
|
|
|||
58
scripts/cloud-bringup.sh
Normal file
58
scripts/cloud-bringup.sh
Normal file
|
|
@ -0,0 +1,58 @@
|
|||
#!/usr/bin/env bash
|
||||
# One-shot DigitalOcean bring-up + smoke. Run it yourself so the cloud build and
|
||||
# repo clone happen under your authority (the agent can't auto-clone private
|
||||
# source onto cloud boxes; you can). It:
|
||||
# 1. builds the golden image from the forge,
|
||||
# 2. spins 1 worker, runs the test suite (timed) + a render proof,
|
||||
# 3. tears the worker down (trap, even on failure).
|
||||
#
|
||||
# Launch and walk away:
|
||||
# nohup bash scripts/cloud-bringup.sh > ~/cloud-bringup.log 2>&1 &
|
||||
# # ...sleep... then on waking: less ~/cloud-bringup.log ; open ~/Desktop/mc-do-proof.png
|
||||
#
|
||||
# Reads all secrets from ~/.vault/ — nothing sensitive is hardcoded here.
|
||||
set -uo pipefail
|
||||
|
||||
REPO="$HOME/Code/@projects/@magic-civilization"
|
||||
cd "$REPO" || exit 1
|
||||
|
||||
# --- auth (from vault) ---
|
||||
export DIGITALOCEAN_TOKEN; DIGITALOCEAN_TOKEN="$(cat ~/.vault/do_pat_mc)"
|
||||
export TF_VAR_do_token="$DIGITALOCEAN_TOKEN"
|
||||
# shellcheck disable=SC1090
|
||||
. ~/.vault/mc_forge_creds # FORGE_IP ADMIN_USER ADMIN_PASS ...
|
||||
GITR="http://${ADMIN_USER}:${ADMIN_PASS}@${FORGE_IP}:3000/mcadmin/magicciv.git"
|
||||
export TF_VAR_git_remote="$GITR" # workers pull latest from the forge
|
||||
export PKR_VAR_git_remote="$GITR" # packer reads the creds from env, not argv
|
||||
PKR_VAR_fleet_pubkey="$(cat ~/.ssh/id_mc_fleet.pub)"; export PKR_VAR_fleet_pubkey # baked into worker authorized_keys
|
||||
# fleet reuses the pre-registered DO key 'mc-fleet' (var ssh_key_name default); just load its private half
|
||||
ssh-add ~/.ssh/id_mc_fleet 2>/dev/null || true # so the dispatch ssh (mc@worker) authenticates
|
||||
|
||||
echo "########## $(date) — DO cloud bring-up starting ##########"
|
||||
|
||||
_teardown() {
|
||||
echo "########## teardown: ./run dist:down ##########"
|
||||
./run dist:down 2>&1 | tail -3 || true
|
||||
echo "forge left UP for inspection — './run forge:down' to park it (~\$0.30/mo idle)."
|
||||
}
|
||||
trap _teardown EXIT
|
||||
|
||||
echo "=== [1/4] packer build golden image (~20-40 min) ==="
|
||||
( cd infra/packer && packer init golden-image.pkr.hcl >/dev/null && \
|
||||
packer build golden-image.pkr.hcl ) \
|
||||
|| { echo "!!! PACKER BUILD FAILED — see above. Stopping."; exit 1; }
|
||||
|
||||
echo "=== [2/4] dist:up 1 worker (s-8vcpu-16gb-amd — beefy, from golden snapshot) ==="
|
||||
./run dist:up 1 s-8vcpu-16gb-amd || { echo "!!! dist:up FAILED"; exit 1; }
|
||||
echo " waiting 75s for worker cloud-init (key + git pull) to settle ..."
|
||||
sleep 75
|
||||
|
||||
echo "=== [3/4] dist:test on the worker (TIMED — the DX-win proof) ==="
|
||||
time ./run dist:test || echo " (dist:test returned nonzero — see output above)"
|
||||
|
||||
echo "=== [4/4] dist:render proof scene -> ~/Desktop/mc-do-proof.png ==="
|
||||
./run dist:render res://engine/scenes/tests/city_proof.tscn "$HOME/Desktop/mc-do-proof.png" 240 \
|
||||
|| echo " (render returned nonzero — try another scene from src/game/engine/scenes/tests/*_proof.tscn)"
|
||||
|
||||
echo "########## $(date) — bring-up done. Worker will be torn down on exit. ##########"
|
||||
echo "Review: this log + ~/Desktop/mc-do-proof.png"
|
||||
|
|
@ -27,6 +27,26 @@ _dist_read_hosts() {
|
|||
grep -vE '^\s*(#|$)' "$inv" 2>/dev/null || true
|
||||
}
|
||||
|
||||
_dist_wait_ready() {
|
||||
# Block until each worker's cloud-init finishes — it copies the fleet key to the
|
||||
# build user and git-pulls. DO's boot agent install delays runcmd 1-3 min, so the
|
||||
# build user isn't ssh-able until then. We ssh as root (authorized immediately) to wait.
|
||||
local root inv host ip
|
||||
root="$(_dist_repo_root)"; inv="$root/.local/fleet/inventory"
|
||||
[ -f "$inv" ] || return 0
|
||||
while IFS= read -r host; do
|
||||
ip="${host#*@}"
|
||||
printf ' waiting for %s cloud-init... ' "$ip"
|
||||
local _i
|
||||
for _i in $(seq 1 36); do
|
||||
ssh -n -o StrictHostKeyChecking=accept-new -o ConnectTimeout=8 -o BatchMode=yes -i ~/.ssh/id_mc_fleet "root@$ip" true 2>/dev/null && break
|
||||
sleep 5
|
||||
done
|
||||
ssh -n -o BatchMode=yes -i ~/.ssh/id_mc_fleet "root@$ip" 'cloud-init status --wait >/dev/null 2>&1 || true' 2>/dev/null
|
||||
echo "ready"
|
||||
done < <(_dist_read_hosts "$inv")
|
||||
}
|
||||
|
||||
cmd_dist() {
|
||||
cat <<'EOF'
|
||||
Distributed test/train fleet (DigitalOcean). Set TF_VAR_do_token first.
|
||||
|
|
@ -67,8 +87,10 @@ cmd_dist_up() {
|
|||
[ -n "${2:-}" ] && args+=(-var "size=$2")
|
||||
[ -n "${3:-}" ] && args+=(-var "region=$3")
|
||||
_dist_tf init -input=false >/dev/null
|
||||
_dist_tf apply "${args[@]}"
|
||||
echo "fleet up: $n worker(s). inventory: $(_dist_repo_root)/.local/fleet/inventory"
|
||||
_dist_tf apply "${args[@]}" || { echo "dist:up FAILED — terraform apply errored (see above)" >&2; return 1; }
|
||||
echo "fleet up: $n worker(s) — waiting for cloud-init before they're usable..."
|
||||
_dist_wait_ready
|
||||
echo "fleet ready. inventory: $(_dist_repo_root)/.local/fleet/inventory"
|
||||
}
|
||||
|
||||
cmd_dist_down() {
|
||||
|
|
|
|||
|
|
@ -56,9 +56,20 @@ cmd_forge() {
|
|||
Forgejo origin lifecycle (DigitalOcean). Needs ~/.vault/do_pat_mc.
|
||||
./run forge:down stop + snapshot + destroy (~$6/mo -> ~$0.30/mo idle)
|
||||
./run forge:up restore from newest snapshot, refresh vault creds
|
||||
./run forge:dns point the 'mcforge' hostname at the current forge IP (sudo; macOS /etc/hosts)
|
||||
EOF
|
||||
}
|
||||
|
||||
cmd_forge_dns() {
|
||||
# Map a friendly hostname to the current forge IP in /etc/hosts (macOS).
|
||||
# Re-run after forge:up (the IP changes). Browse the forge at http://mcforge:3000.
|
||||
local name="${1:-mcforge}" ip
|
||||
ip="$(grep -E '^FORGE_IP=' "$_VAULT_CREDS" 2>/dev/null | cut -d= -f2)"
|
||||
[ -n "$ip" ] || { echo "no FORGE_IP in $_VAULT_CREDS" >&2; return 1; }
|
||||
sudo sh -c "sed -i '' '/[[:space:]]${name}\$/d' /etc/hosts 2>/dev/null; printf '%s\t%s\n' '$ip' '$name' >> /etc/hosts"
|
||||
echo "/etc/hosts: $name -> $ip → http://$name:3000"
|
||||
}
|
||||
|
||||
cmd_forge_down() {
|
||||
local id ip aid snap
|
||||
id="$(_forge_droplet_id)"
|
||||
|
|
|
|||
|
|
@ -1,8 +1,9 @@
|
|||
#!/usr/bin/env bash
|
||||
# `./run verify` — full regression-gate pipeline.
|
||||
#
|
||||
# Split out of dev.sh. The pipeline is 15 steps covering data validation,
|
||||
# i18n, objectives dashboard freshness, Rust build/test/clippy/machete/deny/docs,
|
||||
# Split out of dev.sh. The pipeline covers data validation, i18n, objectives
|
||||
# dashboard freshness, the Rail-1 (no sim logic in GDScript) and Rail-2 (no
|
||||
# hardcoded game content in Rust) gates, Rust build/test/clippy/machete/deny/docs,
|
||||
# file-size cap, TS typecheck, GDScript lint (3 trees), and a headless Godot
|
||||
# boot check. Each step times itself; failures abort and print a summary.
|
||||
|
||||
|
|
@ -85,7 +86,7 @@ cmd_verify() {
|
|||
echo -e "${BLUE}─────────────────────────────────────────────────${NC}"
|
||||
}
|
||||
|
||||
local TOTAL=21
|
||||
local TOTAL=22
|
||||
|
||||
# Step 0 — Game data schema validation
|
||||
_verify_step 0 $TOTAL "game data JSON schemas" \
|
||||
|
|
@ -108,6 +109,14 @@ cmd_verify() {
|
|||
_verify_step 18 $TOTAL "Rail-1: no sim logic in GDScript" \
|
||||
python3 "$REPO_ROOT/tools/check-no-gdscript-sim-logic.py"
|
||||
|
||||
# Step 19 — Rail-2 gate: Rust loads canonical game content from JSON, never
|
||||
# hardcodes it. Registry-driven (low false-positive): registered content
|
||||
# files must stay include_str!-loaded by their owning module, and tombstoned
|
||||
# hardcodes (e.g. promotions XP_THRESHOLDS) must not resurrect. Stops the
|
||||
# two-path divergence (in-game DataLoader vs headless include_str copy).
|
||||
_verify_step 19 $TOTAL "Rail-2: no hardcoded game content in Rust" \
|
||||
python3 "$REPO_ROOT/tools/check-no-rust-hardcoded-content.py"
|
||||
|
||||
# Step 16 — "Build output never under src/" invariant.
|
||||
# Rule source: .claude/instructions/build-output-locations.md.
|
||||
_verify_step 16 $TOTAL "no build output under src/" \
|
||||
|
|
|
|||
175
tools/check-no-rust-hardcoded-content.py
Executable file
175
tools/check-no-rust-hardcoded-content.py
Executable file
|
|
@ -0,0 +1,175 @@
|
|||
#!/usr/bin/env python3
|
||||
"""Rail-2 guard: Rust loads canonical game content from JSON, never hardcodes it.
|
||||
|
||||
Rail-2 — "JSON game packs are the canonical content store; neither Rust nor
|
||||
GDScript hardcodes game content." The failure mode this gate exists to stop is
|
||||
the *two-path divergence*: content reaches the sim two ways — in-game the
|
||||
GDScript `DataLoader` reads the JSON at runtime, headless (tests, CI, AI
|
||||
self-play, the WASM guide) Rust falls back to a compile-time copy. If a balance
|
||||
table is also hardcoded in a Rust crate, the two copies drift apart silently,
|
||||
and the headless path is where the AI trains. (See instruction module
|
||||
`rust-source-of-truth.md` → "the two-path divergence", and p3-28 for the
|
||||
structural endgame: one host-fed `ContentRegistry` both paths read.)
|
||||
|
||||
This is a REGISTRY-DRIVEN gate, deliberately low-false-positive — it does NOT
|
||||
blind-grep for "balance-looking constants" (that flags legit sim-tuning consts
|
||||
like `MIGRATION_RATE`). It enforces two things per registered content file:
|
||||
|
||||
Check A — the JSON file exists and its owning module(s) actually `include_str!`
|
||||
it (the headless Rust path reads the canonical JSON, not a private
|
||||
copy). Catches a module that stops loading its content.
|
||||
|
||||
Check B — "tombstones": const/static names that previously held now-externalized
|
||||
content must NOT reappear in the owning module. Exact-name match →
|
||||
zero false positives. Catches a re-introduced hardcode (the exact
|
||||
promotions regression: XP_THRESHOLDS / HEAL_ON_PROMOTE_FRACTION).
|
||||
|
||||
Coverage is opt-in: a file is guarded once it's added to REGISTRY below. This is
|
||||
intentional — the gate makes a precise promise (registered content stays loaded;
|
||||
known-deleted hardcodes stay dead), not the unkeepable one of catching every
|
||||
possible future hardcode. That blanket guarantee is the ContentRegistry's job
|
||||
(p3-28). Grow REGISTRY as content modules are identified.
|
||||
|
||||
Escape hatch: none needed — Check B uses exact tombstone names, so it only ever
|
||||
fires on a deliberate resurrection of a deleted hardcode.
|
||||
|
||||
Usage:
|
||||
tools/check-no-rust-hardcoded-content.py # report; exit 1 if violations
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import sys
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parent.parent
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ContentEntry:
|
||||
"""One canonical JSON content file and the Rust module(s) that load it."""
|
||||
|
||||
json: str # repo-relative path to the canonical JSON content file
|
||||
modules: tuple[str, ...] # repo-relative Rust module(s) that must load it
|
||||
# const/static names that PREVIOUSLY hardcoded this content and were removed;
|
||||
# the gate fails if any reappears in an owning module (a divergence regression).
|
||||
tombstones: tuple[str, ...] = field(default=())
|
||||
|
||||
|
||||
# The registry. Each entry was verified (file:line) at authoring time against the
|
||||
# `include_str!` content sites in src/simulator/crates/*/src/.
|
||||
REGISTRY: tuple[ContentEntry, ...] = (
|
||||
ContentEntry(
|
||||
json="public/resources/promotions/promotions.json",
|
||||
modules=("src/simulator/crates/mc-combat/src/promotions.rs",),
|
||||
# Removed 2026-06-27 when promotion tuning moved to promotions.json
|
||||
# (the divergence that motivated this gate). Must not come back.
|
||||
tombstones=("XP_THRESHOLDS", "HEAL_ON_PROMOTE_FRACTION"),
|
||||
),
|
||||
ContentEntry(
|
||||
json="public/resources/diplomacy/treaty_rules.json",
|
||||
modules=(
|
||||
"src/simulator/crates/mc-trade/src/rules.rs",
|
||||
"src/simulator/crates/mc-trade/src/tribute.rs",
|
||||
"src/simulator/crates/mc-trade/src/renewal.rs",
|
||||
),
|
||||
),
|
||||
ContentEntry(
|
||||
json="public/resources/ai/freepeople/freepeople.json",
|
||||
modules=("src/simulator/crates/mc-trade/src/tribute.rs",),
|
||||
),
|
||||
ContentEntry(
|
||||
json="public/games/age-of-dwarves/data/score.json",
|
||||
modules=("src/simulator/crates/mc-score/src/lib.rs",),
|
||||
),
|
||||
ContentEntry(
|
||||
json="public/resources/ecology/traits/biome_trait_weights.json",
|
||||
modules=("src/simulator/crates/mc-ecology/src/generation.rs",),
|
||||
),
|
||||
ContentEntry(
|
||||
json="public/resources/ecology/traits/flavor.json",
|
||||
modules=("src/simulator/crates/mc-ecology/src/generation.rs",),
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def _json_suffix(json_path: str) -> str:
|
||||
"""Last two path segments, e.g. 'promotions/promotions.json' — robust to the
|
||||
differing `../` depths used across include_str! sites."""
|
||||
parts = json_path.split("/")
|
||||
return "/".join(parts[-2:])
|
||||
|
||||
|
||||
def _const_decl(name: str) -> re.Pattern[str]:
|
||||
"""Match a `const NAME:` / `static NAME:` (optionally `pub`) declaration."""
|
||||
return re.compile(rf"^\s*(?:pub\s+)?(?:const|static)\s+{re.escape(name)}\s*:")
|
||||
|
||||
|
||||
def main() -> int:
|
||||
violations: list[str] = []
|
||||
|
||||
for entry in REGISTRY:
|
||||
json_path = REPO_ROOT / entry.json
|
||||
suffix = _json_suffix(entry.json)
|
||||
|
||||
# Check A.0 — the canonical content file must exist.
|
||||
if not json_path.is_file():
|
||||
violations.append(
|
||||
f"[A] missing canonical content file: {entry.json}\n"
|
||||
f" registered as game content but not present on disk"
|
||||
)
|
||||
continue
|
||||
|
||||
for mod in entry.modules:
|
||||
mod_path = REPO_ROOT / mod
|
||||
if not mod_path.is_file():
|
||||
violations.append(
|
||||
f"[A] owning module not found: {mod}\n"
|
||||
f" registered as loader of {entry.json}"
|
||||
)
|
||||
continue
|
||||
text = mod_path.read_text(encoding="utf-8", errors="replace")
|
||||
|
||||
# Check A — the module must include_str! its registered content.
|
||||
if "include_str!" not in text or suffix not in text:
|
||||
violations.append(
|
||||
f"[A] {mod} no longer loads {entry.json}\n"
|
||||
f" expected an include_str!(... {suffix}) — content must be\n"
|
||||
f" LOADED from the canonical JSON, not hardcoded (Rail-2)."
|
||||
)
|
||||
|
||||
# Check B — no tombstoned hardcode may reappear.
|
||||
for name in entry.tombstones:
|
||||
rx = _const_decl(name)
|
||||
for lineno, line in enumerate(text.splitlines(), start=1):
|
||||
if rx.search(line):
|
||||
violations.append(
|
||||
f"[B] {mod}:{lineno} resurrects deleted hardcode `{name}`\n"
|
||||
f" {line.strip()}\n"
|
||||
f" This content lives in {entry.json}; load it, don't hardcode (Rail-2)."
|
||||
)
|
||||
|
||||
if violations:
|
||||
print(f"Rail-2 violation: {len(violations)} content-divergence issue(s):\n")
|
||||
for v in violations:
|
||||
print(f" {v}")
|
||||
print(
|
||||
"\nRail-2: JSON game packs are the canonical content store. A Rust crate\n"
|
||||
"must LOAD balance content from public/resources/** (OnceLock+include_str!,\n"
|
||||
"WASM/gdext-safe), never hold a second hardcoded copy that drifts from the\n"
|
||||
"JSON. See .claude/instructions/rust-source-of-truth.md and p3-28."
|
||||
)
|
||||
return 1
|
||||
|
||||
n_files = len(REGISTRY)
|
||||
n_tombstones = sum(len(e.tombstones) for e in REGISTRY)
|
||||
print(
|
||||
f"OK: {n_files} registered content file(s) loaded from JSON; "
|
||||
f"{n_tombstones} tombstoned hardcode(s) stay dead (Rail-2)"
|
||||
)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Loading…
Add table
Reference in a new issue