Update open threads — hosted IDE flow curl-verified, Traefik wildcard complete

OPEN_THREADS.md

@@ -44,16 +44,16 @@ Outstanding:
 
 ### Code Server Addon
 
-Status: ✅ Installed, running, and proxied through API
+Status: ✅ Installed, running, and reachable through API proxy
 
 Confirmed:
 
 - pulled from artifact server (tar.gz)
 - installed to `/opt/zlh/services/code-server`
-- binds to `0.0.0.0:8080`
+- binds to `0.0.0.0:6000`
 - lifecycle endpoints: `POST /dev/codeserver/start|stop|restart`
 - detection via `/proc/*/cmdline` scan
-- browser IDE fully working end-to-end via API proxy
+- hosted browser IDE flow verified with curl end-to-end through Traefik + API proxy
 
 ---
 
@@ -80,52 +80,43 @@ Completed:
 
 ## Dev IDE Access
 
-### Browser IDE ✅ Working (path-based)
+### Browser IDE ✅ Working (host-based through Traefik + API)
 
 ```
-Browser → Portal → API (bootstrap) → /__ide/:id/* → container:8080
+Browser → dev-<vmid>.zerolaghub.dev → Traefik → API → container:6000
 ```
 
-Working flow:
+Verified flow:
 
 1. frontend calls `POST /api/dev/:id/ide-token`
-2. API returns `/api/dev/:id/ide?token=...`
-3. frontend opens that URL in new tab
-4. bootstrap route validates token, sets HTTP-only IDE cookie, redirects to `/__ide/:id/`
-5. all live code-server HTTP + WS traffic proxied through `/__ide/:id/*`
-6. API proxies to `http://<container-ip>:8080`
+2. API returns `https://dev-<vmid>.zerolaghub.dev/?token=...`
+3. browser opens hosted URL
+4. Traefik wildcard router forwards to API at `http://10.60.0.245:4000`
+5. API validates token, sets `zlh_dev_ide_token`, redirects to clean host URL
+6. subsequent cookie-backed request redirects to `/?folder=/home/dev/workspace`
+7. final response is `200` with code-server HTML
+8. API remains HTTP + WS proxy boundary to the container
 
-### Host-based IDE URL — deferred
+Curl-verified response chain:
 
-Goal: open IDE on `dev-<vmid>.zerolaghub.dev` instead of raw API IP.
+- `GET /?token=...` → `302` + `Set-Cookie: zlh_dev_ide_token`
+- `GET /` with cookie → `302` to `/?folder=/home/dev/workspace`
+- `GET /?folder=/home/dev/workspace` → `200` code-server HTML
 
-State: reverted — Caddy removed, back to working path-based flow.
+### Remaining Work
 
-Root cause understood: Express resolves relative redirects using the `Host`
-header. Without `header_up Host {host}` in Caddy, the bootstrap redirect fires
-as `http://10.60.0.245:4000/__ide/6070/` instead of staying on the clean hostname.
+- verify full browser behavior beyond curl
+- verify WebSocket behavior in-browser under hosted flow
+- reduce legacy `/__ide/:id` compatibility paths once host-based is fully canonical
+- confirm "Open IDE" button in portal uses hosted URL in production path
 
-When revisiting, the full Caddyfile block needed is:
+### Wildcard Edge (Traefik)
 
-```
-{
-    auto_https off
-}
-
-http://dev-*.zerolaghub.dev {
-    @dev host dev-*.zerolaghub.dev
-    reverse_proxy @dev 127.0.0.1:4000 {
-        header_up Host {host}
-    }
-}
-```
-
-`header_up Host {host}` is the critical line — without it Express loses the
-hostname on every redirect.
-
-API env vars already set and working:
-- `DEV_IDE_HOST_SUFFIX=zerolaghub.dev`
-- `DEV_IDE_RETURN_HOSTED_URL=true`
+- Traefik on `zlh-zpack-proxy` (10.70.0.242) handles wildcard TLS via DNS challenge
+- wildcard cert `*.zerolaghub.dev` issued via Let's Encrypt + Cloudflare DNS-01
+- Traefik routes `dev-*.zerolaghub.dev` → API at `http://10.60.0.245:4000`
+- `passHostHeader: true` preserves original hostname through to API
+- no Caddy, no `:8081`, no per-container DNS/Traefik side effects from API
 
 ### Local Dev Access (Headscale/Tailscale — Future)
 
@@ -148,15 +139,18 @@ Completed:
 - runtime/version fields
 - enable_code_server flag
 - `GET /api/servers/:id/status` — server status endpoint
-- `POST /api/dev/:id/ide-token` — IDE token generation
+- `POST /api/dev/:id/ide-token` — IDE token generation + hosted URL
 - `GET /api/dev/:id/ide` — bootstrap route (validates token, sets cookie, redirects)
 - `/__ide/:id/*` — live tunnel proxy (HTTP + WS, target-bound)
 - dev routing experiment removed (`devRouting.js`, `devDePublisher.js` deleted)
 - host-based URL generation (`DEV_IDE_HOST_SUFFIX`, `DEV_IDE_RETURN_HOSTED_URL`)
 - `handleHostedProxy` — host-based routing via `Host` header vmid extraction
+- token bootstrap → cookie handoff working under hosted flow
+- hosted flow proxies to container successfully
 
 Outstanding:
 
+- simplify and harden host-native `devProxy` — remove stale path-based assumptions
 - dev runtime catalog endpoint for portal
 - Headscale auth key generation
 
@@ -173,7 +167,8 @@ Completed:
 
 Outstanding:
 
-- "Open IDE" button — calls `POST /api/dev/:id/ide-token`, opens returned URL in new tab
+- confirm "Open IDE" button fully uses hosted URL flow
+- browser validation against hosted wildcard model
 - Headscale setup instructions
 
 ---
@@ -197,9 +192,11 @@ Future work:
 - ✅ Dev container filesystem model
 - ✅ Code-server artifact fix
 - ✅ API status endpoint for frontend agent-state consumption
-- ✅ Dev DNS/Traefik routing experiment — removed
 - ✅ Game server crash recovery with backoff
 - ✅ Crash observability (classification, log tail, exit metadata)
 - ✅ Code-server lifecycle endpoints (start/stop/restart)
 - ✅ Code-server process detection via /proc scan
-- ✅ Dev IDE proxy — browser IDE fully working end-to-end (path-based)
+- ✅ Dev IDE proxy — path-based browser IDE working end-to-end
+- ✅ Hosted wildcard Traefik → API → container dev IDE flow (curl-verified)
+- ✅ Per-container dev IDE edge publish/unpublish removed from API
+- ✅ Wildcard TLS cert `*.zerolaghub.dev` via Let's Encrypt + Cloudflare DNS-01