docs: handover for new chat session Mar 29 2026

SCRATCH/handover-mar29-2026.md

@@ -0,0 +1,213 @@
+# Handover — March 29, 2026
+
+## Context
+This is a fresh chat handover. Read this file first, then read:
+- `OPEN_THREADS.md` — current platform status and pre-launch checklist
+- `PROJECT_CONTEXT.md` — infrastructure, stack, naming conventions
+- `SCRATCH/migration-new-host.md` — migration checklist (in progress)
+- `SCRATCH/pricing-structure.md` — pricing decisions
+
+---
+
+## What's Happening Right Now
+
+A new dedicated server has been purchased from GTHost Detroit and is active.
+The platform is being migrated from the old Denver server to the new Detroit server.
+The user is preparing to begin the migration today.
+
+**New server specs:**
+- Supermicro 2029TP-HTR
+- Intel Xeon Gold 6152 — 22c/44t, 2.1-3.7GHz
+- 192GB DDR4
+- 2x1.92TB SSD
+- Proxmox 9 — already installed by GTHost
+- $99/month Detroit (vs old $103/month Denver)
+
+**Old server (still running, do not cancel yet):**
+- Denver, Silver 4116 12c/24t, 192GB, 2x1.92TB SSD, $103/month
+- Cancel AFTER migration is confirmed working
+
+---
+
+## Migration Approach
+
+**NOT a PBS restore of everything** — deliberate fresh rebuild:
+- Fresh LXC/VM installs for all services
+- Copy project folders for API and portal (not restore)
+- Copy jars/plugins for Velocity
+- Export/import OPNsense config (not VM restore)
+- rsync artifacts content
+- Only restore PBS where config is too complex to rebuild
+
+**Why fresh rebuild:**
+- Eliminates accumulated cruft (Tailscale orphans, stale configs, old packages)
+- Proxmox 9 native LXC templates from the start
+- Clean cgroup v2 resource management
+- Only bring over what is intentionally chosen
+
+---
+
+## PBS Status (CRITICAL — Do This First)
+
+- Old backups only go to November 2025 — disk was full
+- Disk has been resized to 1.1TB
+- Plan: delete old datastore, recreate fresh, run full backup of everything
+- DO NOT start migration until fresh backups are confirmed green
+
+**Fresh backup priority order:**
+1. OPNsense core router (105)
+2. OPNsense zpack router (1006)
+3. zlh-dns (1001)
+4. zpack-api (1005)
+5. zpack-portal (1100)
+6. zlh-artifacts (1003)
+7. zlh-velocity (300)
+8. Everything else
+
+---
+
+## Current VM/LXC Inventory (old host)
+
+### DO MIGRATE — New ID assignments:
+
+| Old ID | Name | New ID | Type | How |
+|--------|------|--------|------|-----|
+| 105 | zlh-router (core OPNsense) | 9001 | VM | Fresh 26.1 + config import |
+| 1006 | zlh-zpack-router (game/dev OPNsense) | 9002 | VM | Fresh 26.1 + config import |
+| 1001 | zlh-dns | 9010 | LXC | Fresh + Technitium export/import |
+| 1002 | zlh-proxy (Traefik core) | 9011 | LXC | Fresh + config |
+| 1004 | zlh-zpack-proxy (Traefik game/dev) | 9012 | LXC | Fresh + config |
+| 9000 | zlh-connect (Twingate) | 9013 | LXC | Fresh install |
+| 1003 | zlh-artifacts (Caddy file server) | 9014 | LXC | Fresh Caddy + rsync content |
+| 300 | zlh-velocity | 9015 | LXC | Fresh + copy jar + plugin |
+| 104 | zlh-monitor (Prometheus/Grafana) | 9016 | LXC | Fresh install |
+| 2001 | zlh-back (PBS) | 9017 | VM | Fresh + PBS config |
+| 1005 | zpack-api | 9020 | VM | Fresh Ubuntu VM + copy project folder |
+| 1100 | zpack-portal | 9021 | VM | Fresh Ubuntu VM + copy project folder |
+| 4000 | aimeesites | 9030 | LXC | Migrate |
+
+### DO NOT MIGRATE (legacy/unused):
+- 100 (zlh-panel) — old Pterodactyl
+- 101 (zlh-wings) — old Pterodactyl
+- 102 (zlh-portal) — old Pterodactyl
+- 103 (zlh-api) — old Pterodactyl
+- 1000 (zlh-router) — original Pterodactyl router
+- 810/890 (zlh-base templates)
+- 5000 (pup) — replaced by Twingate
+- 1007 (zlh-bastion) — on hold
+- 2000 (zlh-ctl) — no Ansible in stack
+
+### Dev/test containers (not production, can recreate if needed):
+- 6050 (zpack-dev-velocity) — used to develop Velocity plugin
+- 6051 (zpack-agent-dev) — used to develop agent
+
+### Active game/dev containers (will be reprovisioned by platform):
+- 5117 (mc-neoforge-5117)
+- 5119 (mc-forge-5119)
+- 5120 (mc-fabric-5120)
+- 6071 (dev-6071)
+
+---
+
+## New Host ID Scheme
+
+| Range | Purpose |
+|-------|---------|
+| 9000s | Core infrastructure (routers, DNS, proxy, monitoring, PBS, API, portal) |
+| 5000+ | Game server containers (provisioned by platform) |
+| 6000+ | Dev containers (provisioned by platform) |
+
+---
+
+## Key Service Notes
+
+**zlh-artifacts (9014):**
+- Runs Caddy as a file server
+- Hosts all runtime binaries (Node, Python, Go, Java, .NET)
+- Hosts Minecraft server jars
+- Hosts code-server binary
+- API pulls from it during provisioning — CRITICAL SERVICE
+- Migration: fresh Caddy install + rsync entire content tree from old server
+
+**OPNsense routers:**
+- Two routers: core (105→9001) and zpack/game/dev (1006→9002)
+- Installing OPNsense 26.1 fresh (upgrade from 25.7.10)
+- Import 25.7.10 config — upgrade path is supported
+- DHCP plugin auto-installs during upgrade
+- Firewall rules migration assistant available but not urgent
+- Interface reassignment after config import may be needed
+- GTHost MAC binding required for WAN public IPs — check GTHost panel for registered MACs
+
+**zpack-api / zpack-portal:**
+- Both kept as VMs (not LXC) — active development, always been VMs
+- Node 22.21.0 (already current LTS — no upgrade needed)
+- Next.js 16.1.1 (current — no upgrade needed)
+- Copy project folder + npm install on new VM
+
+**zlh-velocity:**
+- Velocity 3.5.0-SNAPSHOT (latest)
+- Has a custom dynamic game server routing plugin built by the team
+- Fresh LXC + copy Velocity jar + copy plugin jar
+- Check current version on old server before migrating
+
+---
+
+## Architecture Reminders
+
+- Everything internal to Proxmox except Velocity TCP port (Minecraft players)
+- Portal is only public-facing web surface
+- API runs on private IP — portal calls it internally
+- Minecraft player traffic proxied through Velocity VM
+- Twingate for admin remote access
+- WireGuard on OPNsense as fallback admin access
+- Agent is sole filesystem authority — API never duplicates filesystem logic
+- Portal never calls agents directly — all traffic through API
+- Upload transport: raw http.request piping only, never fetch()
+- VMs 100, 101, 102, 103, 1000 are legacy — do not touch
+
+---
+
+## Network Notes
+
+- All services run on private IPs internally
+- No hardcoded public IP dependencies in application code
+- DNS cutover (Cloudflare A/SRV records for Velocity) is the only external change needed at cutover
+- New host will have different physical NIC names — check with `ip link show` before configuring bridges
+- Mirror the vmbr layout from current host exactly
+- GTHost MAC binding: OPNsense WAN virtual NIC MAC must match what GTHost has registered for the public IPs
+
+---
+
+## Proxmox API Setup (for migration scripting)
+
+When ready to script VM/LXC creation:
+1. Create user `zlh-automation@pve` in Proxmox
+2. Create role with VM.Allocate, VM.Config.*, Datastore.AllocateSpace, Sys.Console
+3. Assign role to user at path `/`
+4. Create API token — save it, only shown once
+5. Use `Authorization: PVEAPIToken=zlh-automation@pve!migration=<secret>` header
+
+---
+
+## Platform Status (from OPEN_THREADS.md)
+
+**Pre-launch blockers:**
+1. Billing / Stripe integration
+2. Game server world backup / restore
+3. User onboarding flow
+4. Password reset flow — verify wired up
+5. Usage limits / quota enforcement
+6. Email notifications
+7. Upload testing
+8. OPNsense audit
+
+**Portal copy — DONE:**
+Landing, features, FAQ, about, pricing all rewritten and updated by Codex.
+Pricing: Vanilla $8/mo, Modded $20/mo, Heavy $35/mo — Minecraft only launch.
+
+---
+
+## Source of Truth
+
+`git.zerolaghub.com/jester/zlh-grind` — always read this before making decisions.
+`git.zerolaghub.com/jester/knowledge-base` — older docs, mostly stale (Dec 2025), leave as historical reference.