magicciv

applications/magicciv

Fork 0

Commit graph

Author	SHA1	Message	Date
Natalie	273a7c71f8	feat(infra): auto-cull orphaned packer build droplets to prevent zombies Some checks are pending ci / regression gate (push) Waiting to run Details Packer destroys its build droplet on a clean finish, but a killed/slept/ network-dropped run leaves the s-8vcpu-16gb-amd builder alive (~$192/mo). This happened once already (.project/handoffs/20260629_packer-cross-account-leak.md). Two defense layers: - scripts/cull-orphan-builders.sh reaps leftover builders by name prefix (mc-packer-* / legacy packer-*) with a size guard and an optional age guard; pins the MC token via --access-token. - cloud-bringup.sh calls it in its EXIT trap, so a failed/Ctrl-C'd build reaps its own builder. - infra/launchd/com.uvlava.mc.cull-builders.plist sweeps every 30m with --min-age-min 90 to catch SIGKILL/power-loss cases no trap can. golden-image.pkr.hcl names the builder mc-packer-<ts> for deterministic matching. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-30 00:05:59 -04:00
Natalie	6332d47011	fix(infra): make the DO fleet actually work on real hardware + render host Real-DO testing surfaced bugs the mocked tests couldn't: - ssh key: reference shared 'mc-fleet' key via data source, not a duplicate (DO 422s on dup pubkeys). - cmd_dist_up: fail loudly on failed apply; dist:up waits for cloud-init readiness. - snapshot cloud-init skips runcmd -> bake authorized_keys (FLEET_PUBKEY) + 'cloud-init clean' before snapshot. - build user passwordless sudo; apt dpkg-lock race fixed (cloud-init --wait + Lock::Timeout). - size s-8vcpu-16gb-amd (tier max); creds via PKR_VAR env not argv. - render host: weston+Mesa baked; ./run dist:render proven (Godot->PNG on DO, no GPU). forge:dns shortcut. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-27 12:45:29 -04:00

Author

SHA1

Message

Date

Natalie

273a7c71f8

feat(infra): auto-cull orphaned packer build droplets to prevent zombies

ci / regression gate (push) Waiting to run

Details

Packer destroys its build droplet on a clean finish, but a killed/slept/
network-dropped run leaves the s-8vcpu-16gb-amd builder alive (~$192/mo).
This happened once already (.project/handoffs/20260629_packer-cross-account-leak.md).

Two defense layers:
- scripts/cull-orphan-builders.sh reaps leftover builders by name prefix
  (mc-packer-* / legacy packer-*) with a size guard and an optional age guard;
  pins the MC token via --access-token.
- cloud-bringup.sh calls it in its EXIT trap, so a failed/Ctrl-C'd build reaps
  its own builder.
- infra/launchd/com.uvlava.mc.cull-builders.plist sweeps every 30m with
  --min-age-min 90 to catch SIGKILL/power-loss cases no trap can.

golden-image.pkr.hcl names the builder mc-packer-<ts> for deterministic matching.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

2026-06-30 00:05:59 -04:00

Natalie

6332d47011

fix(infra): make the DO fleet actually work on real hardware + render host

Real-DO testing surfaced bugs the mocked tests couldn't:
- ssh key: reference shared 'mc-fleet' key via data source, not a duplicate (DO 422s on dup pubkeys).
- cmd_dist_up: fail loudly on failed apply; dist:up waits for cloud-init readiness.
- snapshot cloud-init skips runcmd -> bake authorized_keys (FLEET_PUBKEY) + 'cloud-init clean' before snapshot.
- build user passwordless sudo; apt dpkg-lock race fixed (cloud-init --wait + Lock::Timeout).
- size s-8vcpu-16gb-amd (tier max); creds via PKR_VAR env not argv.
- render host: weston+Mesa baked; ./run dist:render proven (Godot->PNG on DO, no GPU). forge:dns shortcut.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

2026-06-27 12:45:29 -04:00

2 commits