diff --git a/docs/architecture/filesystem-and-file-browser.md b/docs/architecture/filesystem-and-file-browser.md index 35295c8..42f4485 100644 --- a/docs/architecture/filesystem-and-file-browser.md +++ b/docs/architecture/filesystem-and-file-browser.md @@ -1,84 +1,181 @@ -# ZeroLagHub – File System & File Browser Strategy +# Filesystem and File Browser Architecture -**Date:** 2026-02-28 -**Status:** Planning — not yet implemented -**Next Action:** Stub file endpoints in existing agent, prove end-to-end, extract later +**Updated:** 2026-03-01 +**Status:** Active — reflects implemented upload model --- -## Context +## Overview -This document captures the architectural decisions and UX direction for file system access in ZeroLagHub game and dev containers. It is a planning document, not an implementation spec. +ZeroLagHub uses a strict, runtime-aligned filesystem model for game containers. +The file browser reflects the real runtime layout and does not use staging directories, shadow folders, or symlink-based indirection. + +Uploads are written directly into the runtime directory tree with strict policy enforcement and provenance tracking. --- -## Use Cases +## Runtime Root -### Game Server Owners (non-technical) -- Edit `server.properties` and config files -- Upload mod `.jar` files to `/mods` -- Restore deleted mods from `/mods-removed` -- Download log files for debugging - -### Developers / BYOS (technical) -- Full shell access -- File transfer (upload/download) -- SFTP access to dev container filesystem - -These are two distinct personas with different needs and different solutions. - ---- - -## Architecture Decision - -### Game Server File Access -Handled via **agent file endpoints + portal file browser UI**. - -No SSH required. The agent exposes REST file management endpoints. The API proxies them behind auth + ownership enforcement (same pattern as all other agent endpoints). The portal renders a file browser panel. - -### Dev Container File Access -Handled via **WebSSH2 + SFTP**, proxied through the API. - -Developers get a real SSH2 session with SFTP channel. No direct container access from the browser — API proxy maintains the security boundary (DEC-008). - ---- - -## Agent File Endpoints (Planned) +Each game container has a single runtime root: ``` -GET /game/files?path= — list directory -GET /game/files/download?path= — download file -POST /game/files/upload?path= — upload file -DELETE /game/files?path= — delete file -PATCH /game/files — rename file +/opt/zlh/minecraft/// ``` -Mirrored in API under: -``` -/api/game/servers/:id/files/* -``` +All file operations are resolved relative to this root. -### Security Requirements -- Hard-rooted to `serverRoot` — no path traversal outside container root -- HTTPS only -- Auth + ownership enforced at API layer -- Upload size limits enforced -- No execution of uploaded files +The agent is the only component allowed to perform filesystem mutations. --- -## Implementation Sequencing +## Hidden Internal Files -**Do not split the agent yet.** +The following files are internal and never exposed through the file API: -Land file endpoints in the existing agent first. Prove the feature end-to-end. Once the surface area is clear and stable, extract if needed. +- `.zlh-shadow` +- `.zlh_metadata.json` -SFTP is the exception — if SFTP access for dev containers is implemented, it warrants its own separate process from day one due to SSH server complexity. It does not belong in the main game agent. +These are filtered centrally inside the agent `internal/files` package, not in route handlers. -### Phased approach -1. File endpoints in existing agent (stub → prove → harden) -2. Portal file browser UI wired to API proxy -3. SFTP as separate agent process for dev containers (separate binary, separate port, separate systemd unit) +--- + +## Upload Model + +### No Staging + +Uploads are written directly into their final runtime location. + +There is: +- ❌ No temporary upload folder +- ❌ No symlink-based deployment +- ❌ No background relocation job + +Uploads are atomic: + +1. Write to temp file in same directory +2. `os.Rename()` to final path + +--- + +## Upload Allowlist + +Uploads are restricted to: + +| Type | Path | Extension | +|------|------|-----------| +| Mods | `mods/.jar` | `.jar` | +| Datapacks | `world/datapacks/.zip` | `.zip` | + +All other upload paths are rejected. + +Parent directory must already exist. No directory creation is performed. + +--- + +## Security Constraints + +Upload is rejected if: + +- Path traversal (`../`) +- Absolute path +- Control characters +- Target exists as directory +- Target is a symlink +- Target resolves outside runtime root +- Path not allowlisted +- Parent directory missing + +--- + +## Provenance Metadata + +Uploads create or update `.zlh_metadata.json`, stored at runtime root. + +**Example:** + +```json +{ + "mods/sodium.jar": { + "source": "user", + "uploaded_at": "2026-03-01T22:37:01Z" + } +} +``` + +### stat Response Extension + +File stat now includes: + +``` +"source": "user" | null +``` + +Only `"user"` is currently written by the agent. No curated inference is performed. + +--- + +## File Browser Responsibilities + +### Portal (Frontend) +- Displays files +- Displays `source: "user"` badge +- Enforces extension pre-validation +- Sends multipart upload + +### API +- Auth + ownership enforcement +- Streams upload to agent +- Does **NOT** inspect file contents +- Does **NOT** re-implement upload policy + +### Agent +- Enforces filesystem rules +- Writes metadata +- Performs atomic writes +- Returns authoritative status + +--- + +## Why No Symlinks + +Symlink-based deployment was rejected because: + +- It complicates mod loading behavior +- Breaks compatibility with server-side mod loaders +- Introduces unexpected runtime indirection +- Makes provenance ambiguous + +Direct writes + metadata is simpler and safer. + +--- + +## Data Flow + +``` +Browser → API → Agent → Filesystem +``` + +The API uses raw `http.request` piping to stream uploads. It does not buffer large files in memory. + +--- + +## API Transport Considerations + +The API acts strictly as a streaming proxy. + +Implementation uses raw `http.request` piping: + +```js +req.pipe(proxyReq) +proxyRes.pipe(res) +``` + +This avoids: +- Fetch streaming incompatibilities +- Duplex locking issues +- Multipart buffering problems + +Upload timeout for this route should be substantially larger than normal file routes. --- @@ -91,16 +188,16 @@ Split-pane panel — directory tree left, file detail/actions right. Slides in a Breadcrumb-based. Flat navigation within each directory rather than deep tree expansion. Click folder → replace view. Not expand-in-place. ### File Listing -Columns: name, size, modified date, type icon. For `.jar` files: status badge (enabled / disabled / removed). +Columns: name, size, modified date, type icon, source badge (`user` for user-uploaded files). For `.jar` files: enabled / disabled / removed status. ### Actions Context menu or per-row three-dot menu. Actions: download, delete, rename. For `.jar` files: enable / disable toggle. Drag-to-upload supported, file picker fallback. ### In-Browser Editor -Plain textarea or Monaco for text files (`server.properties`, `.json`, `.txt`). Binary files get download link only. Not required for launch. +Plain textarea or Monaco for text files (`server.properties`, `.json`, `.toml`, `.yml`, `.txt`). Binary files get download link only. Not required for launch. ### `mods-removed` Surface -"Recently removed" section or toggle to show `/mods-removed` alongside active mods. This makes soft delete visible and gives users a restore path without knowing the underlying filesystem layout. +"Recently removed" section or toggle to show `/mods-removed` alongside active mods. Makes soft delete visible and gives users a restore path without knowing the underlying filesystem layout. ### What to Avoid - Deep expand/collapse tree for mod directory (use flat list + filter) @@ -115,7 +212,15 @@ Plain textarea or Monaco for text files (`server.properties`, `.json`, `.txt`). The agent already runs an HTTP server. Serving static file browser assets from the agent directly keeps per-container footprint minimal. No additional process, no config management, no extra memory overhead. -Caddy or Nginx per container would make sense if you needed per-container SSL termination or direct browser access without the API proxy. ZLH's architecture routes everything through the API proxy (`zlh-proxy` handles SSL at the edge), so a local web server adds a layer without adding capability. +`zlh-proxy` handles SSL at the edge. A local web server per container adds a layer without adding capability. + +--- + +## Dev Container File Access + +Handled via **WebSSH2 + SFTP**, proxied through the API. + +Developers get a real SSH2 session with SFTP channel. No direct container access from the browser — API proxy maintains the security boundary (DEC-008). SFTP warrants its own separate process from day one due to SSH server complexity. It does not belong in the main game agent. --- @@ -129,14 +234,20 @@ The file agent (when extracted) should follow the same binary resilience pattern - Systemd watchdog flips `current` back to previous on health check failure - No dependency on artifact server for rollback — local fallback only -This keeps the file service self-healing without operator intervention, consistent with ZLH's overall design goal. +--- + +## Design Philosophy + +The runtime directory is the source of truth. + +No abstraction layer should diverge from Minecraft's real file structure. --- ## Related Documents -- `docs/architecture/mod-deployment-safety.md` — mod lifecycle and rollback model +- `docs/architecture/mod-deployment-safety.md` — mod lifecycle, upload safety, rollback model - `docs/architecture/dev-to-game-artifact-pipeline.md` — dev container promotion pipeline +- `docs/reference/minecraft-file-locations.md` — known files and directories by loader - `OPEN_THREADS.md` — file browser listed as next major feature -- `Frontend/TerminalView_Component.md` (knowledge-base) — terminal implementation reference - WebSSH2: `https://github.com/billchurch/webssh2` — SFTP + SSH2 for dev containers