docs(infra): handoff — mc packer leaked into cocotte DO account

mc golden-image build ran with the cocotte DIGITALOCEAN_TOKEN, leaving 3
mc-golden-* images + 2 orphaned s-8vcpu-16gb-amd build VMs (~$192/mo) in the
ct account. Fix: always use ~/.vault/do_pat_mc; tear down build VMs every run.
Includes cleanup IDs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Natalie 2026-06-29 17:55:39 -04:00
parent 57a2d83e2d
commit a0428fc950

View file

@ -0,0 +1,41 @@
# Handoff: ct-infra → magicciv simulator-infra
- Date: 2026-06-29
- From: ct-infra (cocotte CI/CD work)
- To: magicciv simulator-infra / cloud-dx owner
---
## Use your own token, and stop making zombies.
While provisioning ct-forge CI runners we found **magic-civilization's golden-image
packer build is running in the COCOTTE DigitalOcean account**, not mc's. It has
leaked artifacts + orphaned droplets into the wrong account.
### Evidence (in the `ct` / cocotte DO account, queried with `do_pat_cocotte`)
- **3 stray `mc-golden-*` images** — IDs `234574121`, `234574942`, `234698723`
(2026-06-27/28). These belong in the mc account.
- **2 orphaned build droplets** (the zombies) — `packer-6a4130d1-...` (id `580870251`)
and `packer-6a413161-...` (id `580870438`), both **`s-8vcpu-16gb-amd`** = your packer
worker size. ~$192/mo bleeding from the wrong account. Packer destroys its build VM
on success; these survived a failed/interrupted run and were never cleaned up.
Root cause: the build ran with `DIGITALOCEAN_TOKEN` set to the cocotte token.
`infra/packer/golden-image.pkr.hcl` takes `do_token = env("DIGITALOCEAN_TOKEN")`, so
whatever account that token belongs to is where the image + VM land.
### Fix (two rules)
1. **Use your own token.** Always export the mc token before any mc packer/terraform:
`export DIGITALOCEAN_TOKEN="$(cat ~/.vault/do_pat_mc)"`. Never the cocotte token.
This is already the documented rule — `tooling/.../instructions/cloud-dx-do.md:30`
names `~/.vault/do_pat_mc`; the build just didn't follow it.
2. **No zombies.** Confirm Packer tears down its build droplet every run; on a failed
build, delete the leftover `packer-*` VM immediately (16 GB AMD is not cheap). Don't
leave 8-vCPU boxes idling.
### Cleanup owed (in the cocotte account — ask ct/quinn to run, or whoever holds the PAT)
```
DIGITALOCEAN_TOKEN=$(cat ~/.vault/do_pat_cocotte) doctl compute droplet delete 580870251 580870438 --force
DIGITALOCEAN_TOKEN=$(cat ~/.vault/do_pat_cocotte) doctl compute image delete 234574121 234574942 234698723
```
Then rebuild `mc-golden` in the **mc** account so your test-fleet auto-discovers it there.