zlh-grind/SESSION_LOG/2026-03-01-upload-pipeline-filesystem-consolidation.md

2026-03-01 – Upload Pipeline + Filesystem Consolidation

Type: Implementation + Architecture Consolidation Status: Upload implemented, transport tuning in progress

---

Completed

Upload Model

• Implemented direct runtime upload model
• Removed staging and symlink exploration entirely
• Uploads write atomically to runtime root
• Enforced by strict allowlist (mods/*.jar, world/datapacks/*.zip)
• No directory creation, no staging folders, no symlinks

Provenance

• Added .zlh_metadata.json provenance tracking
• Extended stat response with source field ("user" | null)
• No curated inference implemented (intentional)

API Transport

• Identified fetch() streaming as source of upload proxy timing issues
• Switched to raw http.request piping:

  req.pipe(proxyReq)
  proxyRes.pipe(res)
  

• Confirmed agent upload via direct curl
• Upload timeout tuning identified as remaining work

Documentation

• Consolidated filesystem architecture docs (filesystem-and-file-browser.md, mod-deployment-safety.md)
• Full grind repo consolidation pass (README, CONSTRAINTS, OPEN_THREADS, SESSION_START, ANTI_DRIFT, UPSTREAMS)

---

System State

• Read: Stable
• Write: Stable with shadow backup
• Delete: Constrained and safe
• Upload: Implemented, transport tuning in progress
• Console: Stable

---

Known Architectural Decisions Locked In

• Direct runtime upload — no staging, no symlinks
• .zlh_metadata.json is the provenance layer; stat returns source
• Raw http.request piping for upload proxy — no fetch() streaming
• Upload allowlist enforced by agent, not API

---

Remaining Issues

Upload Transport Timeout

• Upload route timeout not yet increased
• Need to distinguish:
  • client abort
  • upstream timeout
  • socket reset
• Upload route should log explicit error categories

Dev Server Filesystem Model

• Not yet designed
• Game server model is complete and stable
• Dev runtime layout, artifact promotion strategy, and file editing scope all TBD

---

Next Steps

1. Increase upload route timeout
2. Add explicit error category logging to upload route
3. Begin dev server filesystem model planning