docs: replace with runtime-aligned upload architecture (direct writes, metadata provenance, API streaming)

docs/architecture/filesystem-and-file-browser.md

@@ -1,84 +1,181 @@
-# ZeroLagHub – File System & File Browser Strategy
+# Filesystem and File Browser Architecture
 
-**Date:** 2026-02-28
+**Updated:** 2026-03-01
-**Status:** Planning — not yet implemented
+**Status:** Active — reflects implemented upload model
-**Next Action:** Stub file endpoints in existing agent, prove end-to-end, extract later
 
 ---
 
-## Context
+## Overview
 
-This document captures the architectural decisions and UX direction for file system access in ZeroLagHub game and dev containers. It is a planning document, not an implementation spec.
+ZeroLagHub uses a strict, runtime-aligned filesystem model for game containers.
+The file browser reflects the real runtime layout and does not use staging directories, shadow folders, or symlink-based indirection.
+
+Uploads are written directly into the runtime directory tree with strict policy enforcement and provenance tracking.
 
 ---
 
-## Use Cases
+## Runtime Root
 
-### Game Server Owners (non-technical)
+Each game container has a single runtime root:
-- Edit `server.properties` and config files
-- Upload mod `.jar` files to `/mods`
-- Restore deleted mods from `/mods-removed`
-- Download log files for debugging
-
-### Developers / BYOS (technical)
-- Full shell access
-- File transfer (upload/download)
-- SFTP access to dev container filesystem
-
-These are two distinct personas with different needs and different solutions.
-
----
-
-## Architecture Decision
-
-### Game Server File Access
-Handled via **agent file endpoints + portal file browser UI**.
-
-No SSH required. The agent exposes REST file management endpoints. The API proxies them behind auth + ownership enforcement (same pattern as all other agent endpoints). The portal renders a file browser panel.
-
-### Dev Container File Access
-Handled via **WebSSH2 + SFTP**, proxied through the API.
-
-Developers get a real SSH2 session with SFTP channel. No direct container access from the browser — API proxy maintains the security boundary (DEC-008).
-
----
-
-## Agent File Endpoints (Planned)
 
 ```
-GET    /game/files?path=          — list directory
+/opt/zlh/minecraft/<runtime>/<world>/
-GET    /game/files/download?path= — download file
-POST   /game/files/upload?path=   — upload file
-DELETE /game/files?path=          — delete file
-PATCH  /game/files                — rename file
 ```
 
-Mirrored in API under:
+All file operations are resolved relative to this root.
-```
-/api/game/servers/:id/files/*
-```
 
-### Security Requirements
+The agent is the only component allowed to perform filesystem mutations.
-- Hard-rooted to `serverRoot` — no path traversal outside container root
-- HTTPS only
-- Auth + ownership enforced at API layer
-- Upload size limits enforced
-- No execution of uploaded files
 
 ---
 
-## Implementation Sequencing
+## Hidden Internal Files
 
-**Do not split the agent yet.**
+The following files are internal and never exposed through the file API:
 
-Land file endpoints in the existing agent first. Prove the feature end-to-end. Once the surface area is clear and stable, extract if needed.
+- `.zlh-shadow`
+- `.zlh_metadata.json`
 
-SFTP is the exception — if SFTP access for dev containers is implemented, it warrants its own separate process from day one due to SSH server complexity. It does not belong in the main game agent.
+These are filtered centrally inside the agent `internal/files` package, not in route handlers.
 
-### Phased approach
+---
-1. File endpoints in existing agent (stub → prove → harden)
+
-2. Portal file browser UI wired to API proxy
+## Upload Model
-3. SFTP as separate agent process for dev containers (separate binary, separate port, separate systemd unit)
+
+### No Staging
+
+Uploads are written directly into their final runtime location.
+
+There is:
+- ❌ No temporary upload folder
+- ❌ No symlink-based deployment
+- ❌ No background relocation job
+
+Uploads are atomic:
+
+1. Write to temp file in same directory
+2. `os.Rename()` to final path
+
+---
+
+## Upload Allowlist
+
+Uploads are restricted to:
+
+| Type | Path | Extension |
+|------|------|-----------|
+| Mods | `mods/<file>.jar` | `.jar` |
+| Datapacks | `world/datapacks/<file>.zip` | `.zip` |
+
+All other upload paths are rejected.
+
+Parent directory must already exist. No directory creation is performed.
+
+---
+
+## Security Constraints
+
+Upload is rejected if:
+
+- Path traversal (`../`)
+- Absolute path
+- Control characters
+- Target exists as directory
+- Target is a symlink
+- Target resolves outside runtime root
+- Path not allowlisted
+- Parent directory missing
+
+---
+
+## Provenance Metadata
+
+Uploads create or update `.zlh_metadata.json`, stored at runtime root.
+
+**Example:**
+
+```json
+{
+  "mods/sodium.jar": {
+    "source": "user",
+    "uploaded_at": "2026-03-01T22:37:01Z"
+  }
+}
+```
+
+### stat Response Extension
+
+File stat now includes:
+
+```
+"source": "user" | null
+```
+
+Only `"user"` is currently written by the agent. No curated inference is performed.
+
+---
+
+## File Browser Responsibilities
+
+### Portal (Frontend)
+- Displays files
+- Displays `source: "user"` badge
+- Enforces extension pre-validation
+- Sends multipart upload
+
+### API
+- Auth + ownership enforcement
+- Streams upload to agent
+- Does **NOT** inspect file contents
+- Does **NOT** re-implement upload policy
+
+### Agent
+- Enforces filesystem rules
+- Writes metadata
+- Performs atomic writes
+- Returns authoritative status
+
+---
+
+## Why No Symlinks
+
+Symlink-based deployment was rejected because:
+
+- It complicates mod loading behavior
+- Breaks compatibility with server-side mod loaders
+- Introduces unexpected runtime indirection
+- Makes provenance ambiguous
+
+Direct writes + metadata is simpler and safer.
+
+---
+
+## Data Flow
+
+```
+Browser → API → Agent → Filesystem
+```
+
+The API uses raw `http.request` piping to stream uploads. It does not buffer large files in memory.
+
+---
+
+## API Transport Considerations
+
+The API acts strictly as a streaming proxy.
+
+Implementation uses raw `http.request` piping:
+
+```js
+req.pipe(proxyReq)
+proxyRes.pipe(res)
+```
+
+This avoids:
+- Fetch streaming incompatibilities
+- Duplex locking issues
+- Multipart buffering problems
+
+Upload timeout for this route should be substantially larger than normal file routes.
 
 ---
 
@@ -91,16 +188,16 @@ Split-pane panel — directory tree left, file detail/actions right. Slides in a
 Breadcrumb-based. Flat navigation within each directory rather than deep tree expansion. Click folder → replace view. Not expand-in-place.
 
 ### File Listing
-Columns: name, size, modified date, type icon. For `.jar` files: status badge (enabled / disabled / removed).
+Columns: name, size, modified date, type icon, source badge (`user` for user-uploaded files). For `.jar` files: enabled / disabled / removed status.
 
 ### Actions
 Context menu or per-row three-dot menu. Actions: download, delete, rename. For `.jar` files: enable / disable toggle. Drag-to-upload supported, file picker fallback.
 
 ### In-Browser Editor
-Plain textarea or Monaco for text files (`server.properties`, `.json`, `.txt`). Binary files get download link only. Not required for launch.
+Plain textarea or Monaco for text files (`server.properties`, `.json`, `.toml`, `.yml`, `.txt`). Binary files get download link only. Not required for launch.
 
 ### `mods-removed` Surface
-"Recently removed" section or toggle to show `/mods-removed` alongside active mods. This makes soft delete visible and gives users a restore path without knowing the underlying filesystem layout.
+"Recently removed" section or toggle to show `/mods-removed` alongside active mods. Makes soft delete visible and gives users a restore path without knowing the underlying filesystem layout.
 
 ### What to Avoid
 - Deep expand/collapse tree for mod directory (use flat list + filter)
@@ -115,7 +212,15 @@ Plain textarea or Monaco for text files (`server.properties`, `.json`, `.txt`).
 
 The agent already runs an HTTP server. Serving static file browser assets from the agent directly keeps per-container footprint minimal. No additional process, no config management, no extra memory overhead.
 
-Caddy or Nginx per container would make sense if you needed per-container SSL termination or direct browser access without the API proxy. ZLH's architecture routes everything through the API proxy (`zlh-proxy` handles SSL at the edge), so a local web server adds a layer without adding capability.
+`zlh-proxy` handles SSL at the edge. A local web server per container adds a layer without adding capability.
+
+---
+
+## Dev Container File Access
+
+Handled via **WebSSH2 + SFTP**, proxied through the API.
+
+Developers get a real SSH2 session with SFTP channel. No direct container access from the browser — API proxy maintains the security boundary (DEC-008). SFTP warrants its own separate process from day one due to SSH server complexity. It does not belong in the main game agent.
 
 ---
 
@@ -129,14 +234,20 @@ The file agent (when extracted) should follow the same binary resilience pattern
 - Systemd watchdog flips `current` back to previous on health check failure
 - No dependency on artifact server for rollback — local fallback only
 
-This keeps the file service self-healing without operator intervention, consistent with ZLH's overall design goal.
+---
+
+## Design Philosophy
+
+The runtime directory is the source of truth.
+
+No abstraction layer should diverge from Minecraft's real file structure.
 
 ---
 
 ## Related Documents
 
-- `docs/architecture/mod-deployment-safety.md` — mod lifecycle and rollback model
+- `docs/architecture/mod-deployment-safety.md` — mod lifecycle, upload safety, rollback model
 - `docs/architecture/dev-to-game-artifact-pipeline.md` — dev container promotion pipeline
+- `docs/reference/minecraft-file-locations.md` — known files and directories by loader
 - `OPEN_THREADS.md` — file browser listed as next major feature
-- `Frontend/TerminalView_Component.md` (knowledge-base) — terminal implementation reference
 - WebSSH2: `https://github.com/billchurch/webssh2` — SFTP + SSH2 for dev containers