magicciv/tools/sprite-generation/docs/PIPELINE.md

Sprite Generation Pipeline

Overview

A single long-running process (cli.py run) generates sprites one at a time, scores each immediately, and cycles through the roster until all pass. The human's job is reviewing passing sprites in the Theater GUI and approving winners.

  game JSON ──scan──▶ spritegen.db (sprites table — what to generate)
                            │
                     ┌──────┴──────┐
                     │ ORCHESTRATOR │  cli.py run
                     │   (daemon)   │
                     └──────┬──────┘
                            │
               ┌────────────┼────────────┐
               ▼            ▼            ▼
          ┌─────────┐ ┌─────────┐ ┌──────────┐
          │GENERATE │ │  RANK   │ │  STATUS  │
          │model-boss│ │ Sonnet  │ │  CHECK   │
          │1 sprite │ │ vision  │ │          │
          │4 variants│ │ 7 dims  │ │ pass?    │
          └────┬────┘ └────┬────┘ └────┬─────┘
               │           │           │
               ▼           ▼           ▼
          raw/*.png   spritegen.db   status=review (pass)
                      variant.notes  status=needed (fail, retry)
                      variant.rating max 5 attempts → review anyway
                            │
                     ┌──────┴──────┐
                     │ THEATER GUI │  localhost:5850
                     │ human picks │
                     │ approve/skip│
                     └──────┬──────┘
                            │
                     ┌──────┴──────┐
                     │   INSTALL   │  cli.py approve <id>
                     │ chroma key  │
                     │ resize 256² │
                     │ → game dir  │
                     └─────────────┘

Data Model (spritegen.db)

sprites (51)                    One row per sprite to generate
  id          TEXT PK           "units/spearmen_dwarves_m"
  category    TEXT              "units"
  entity_id   TEXT              "spearmen_dwarves_m"
  status      TEXT              needed → review → approved → installed
  prompt      TEXT              Current prompt template (mutable)
  negative_prompt TEXT          Current negative (mutable)
  install_path TEXT             Game asset destination path
  gen_width/height              Generation resolution (1024×1024)
  target_width/height           Final sprite size (256×256)
        │
        │ 1:N
        ▼
variants (431+)                 One row per generated image
  id            INTEGER PK
  sprite_id     TEXT FK → sprites
  seed          INTEGER         Reproducible seed
  job_status    TEXT            submitted → completed
  raw_path      TEXT            raw/{sprite_id}_{variant_id}.png
  processed_path TEXT           variants/{...}.png (after chroma key)
  is_approved   INTEGER         0 or 1
  rating        INTEGER         1-5 (from Sonnet), -1 = rejected/skipped
  notes         TEXT            JSON scores {"composition": 0.85, ...}
  ── immutable generation record ──
  model         TEXT            "juggernaut-xi-v11"
  prompt_used   TEXT            Exact prompt sent to model
  negative_used TEXT            Exact negative sent
  guidance_scale REAL           9.0
  steps         INTEGER         25

sprite_dimensions (41)          Quality/race/gender permutations
  sprite_id   FK → sprites
  dimension_type  TEXT          "quality"
  dimension_value TEXT          "q1"
  prompt_modifier TEXT          Added to base prompt

generation_runs (617)           Batch tracking
  total_jobs / completed / failed

Orchestrator Loop

cli.py run --category units --variants 4

Each loop iteration:

1. Pick ONE sprite with status=needed (skip if hit 5 regen attempts)
2. Generate 4 variants via model-boss (MAX_CONCURRENT=1, sequential)
3. Rank ALL unscored variants for that sprite via Sonnet vision (7 dimensions)
4. If ≥3 variants pass 70% threshold → status=review
5. If <3 pass and attempts < 5 → keep as needed (retry next loop)
6. If attempts = 5 → force to status=review (human picks from best available)
7. Sleep 5s, next sprite

Scoring (7 dimensions, 70% threshold)

Dimension	What it checks
facing_direction	Character oriented toward lower-left (southwest)
composition	Single character, clean framing, no clutter
unit_identity	Correct unit type with appropriate equipment
race_accuracy	Correct racial proportions (dwarf = short/stocky)
gender_accuracy	Correct gender presentation
pose_quality	Full body visible, natural stance
background_compliance	Clean green background, no base/tile/ground
art_style	2D hand-painted fantasy game art, not photorealistic

Confidence = average of all 8 dimensions. Variant passes at ≥70%.

CLI Commands

# Full pipeline (generate → rank → regen loop + GUI server)
python3 cli.py run --category units --variants 4

# GUI server only
python3 cli.py start --port 5850

# Scan game data → populate sprite registry
python3 cli.py --demo scan

# Test prompts without touching DB (rapid iteration)
python3 cli.py test-prompt --entity spearmen --race dwarves --gender male --seeds 42 123 777

# Rebuild all stored prompts from current templates
python3 cli.py refresh-prompts --category units --clear-scores

# Manual operations
python3 cli.py generate --sprite units/spearmen_dwarves_m --variants 8
python3 cli.py rank --sprite units/spearmen_dwarves_m
python3 cli.py approve 129
python3 cli.py status
python3 cli.py reset --sprite units/spearmen_dwarves_m

File Flow

game JSON data (demo-data/units/*.json)
    │
    ▼ scan
spritegen.db ──── sprites table (what to generate)
    │
    ▼ generate (model-boss → juggernaut-xi-v11)
raw/{sprite_id}_{variant_id}.png ──── 1024×1024, green bg
    │
    ▼ rank (Sonnet vision, 7 dimensions)
spritegen.db ──── variants table (scores in notes JSON)
    │
    ▼ human approve (Theater GUI or cli.py approve)
spritegen.db ──── variant.is_approved = 1
    │
    ▼ process (chroma key removal + resize)
variants/{sprite_id}_{variant_id}.png ──── 256×256, transparent
    │
    ▼ install
public/games/age-of-dwarves/assets/sprites/units/{name}.png

Configuration

// sprite-config.json
{
  "model": "juggernaut-xi-v11",
  "api_base": "http://localhost:8210",
  "defaults": {
    "steps": 25,
    "guidance_scale": 9.0
  }
}

Prompt templates live in engine/prompts.py. The compose_prompt() function builds the full prompt from: style prefix + race/gender + entity description + style tail.