Public dry run dryrun-a43e907efd9d

Source commit: a43e907efd9d1561c23fe73499478e881f868355

Public tree identity: sha256:453dea30195485dd8939f575a69b39f4bb7acd84c4df23f8aa29b55d044f2673

docs/architecture.md

@@ -0,0 +1,55 @@
+# Architecture
+
+Clusterflux separates a control plane from execution nodes.
+
+## Coordinator
+
+The coordinator owns identity scope, one-active-process admission, task
+placement, attempt history, Debug Epoch coordination, VFS metadata, artifact
+metadata, quotas, and secure relay reservations. Its async main may park and wake
+without consuming node compute.
+
+The coordinator does not execute arbitrary native commands or hosted
+containers. It also does not become a durable artifact store merely because it
+coordinates a download.
+
+## Nodes
+
+A node is an enrolled public-key identity. It reports capabilities and periodic
+heartbeats. The coordinator derives whether it is live from the last accepted
+heartbeat; a client-supplied "online" field is not placement authority.
+
+Nodes resolve bundle environments, execute Wasm tasks, run native commands, and
+retain artifact bytes. A node must prove the tenant, project, process, task, and
+key scope on every signed request.
+
+## Virtual process and tasks
+
+A Coordinator Project admits one active virtual process. The process contains:
+
+- one coordinator-hosted async main;
+- logical task instances created by that main;
+- one or more attempts for each logical task;
+- joins tied to the logical task, not to an individual attempt.
+
+"FailFast" makes a terminal failure visible to the join immediately.
+"AwaitOperator" keeps the join pending while an operator accepts, cancels, or
+restarts the failed task. Restart creates a distinct attempt but preserves the
+logical task identity.
+
+A terminal process releases the active slot automatically. A task parked for an
+operator is not terminal and continues to occupy the slot.
+
+## Durable and live state
+
+Projects, identities, credentials, permissions, and hosted policy records may
+be stored in Postgres. Active processes, task placement, Debug Epochs, transient
+VFS state, relay reservations, and live node state are bounded in memory and are
+not reconstructed as running after a coordinator restart.
+
+## Protocol lanes
+
+Human clients use scoped sessions. Agents and nodes use separate signed
+public-key identities. Operator actions use a separate administrative
+credential. Authority comes from the authenticated lane and server-side scope,
+never from identity fields in an untrusted request body.

docs/artifacts.md

@@ -0,0 +1,36 @@
+# Artifacts
+
+Clusterflux separates artifact metadata from artifact bytes.
+
+## Publish metadata
+
+"flush()" makes a VFS artifact visible to downstream tasks and records its
+digest, size, producer task and attempt, and retaining locations. It does not
+upload bytes to the coordinator by default.
+
+Your node retains artifact bytes on a best-effort basis. If every retaining node
+is lost, stale, revoked, or garbage-collects the bytes, the artifact is
+unavailable. Clusterflux reports that state instead of inventing durable
+storage.
+
+## Move bytes explicitly
+
+Use "sync()" or an explicit export when you need another node or your own storage
+system to hold the bytes. Same-node reuse avoids a coordinator transfer.
+
+## Download
+
+~~~bash
+clusterflux artifact list --process <process-id>
+clusterflux artifact download <artifact-id>   --to ./output.bin   --max-bytes 67108864
+~~~
+
+Before returning a link, the service checks authorization, current retention,
+source-node liveness, live size, project quota, and relay capacity. It reserves
+the affordable ingress and egress budget atomically. The scoped link is
+unguessable, expires, can be revoked, and is bound to the actor, tenant, project,
+process, artifact, and policy context.
+
+The reverse stream counts framing, base64 expansion, failed bytes, and abandoned
+transfers. The CLI verifies the final digest. Hosted policy may impose stricter
+size, concurrency, and period limits or disable relay traffic globally.

docs/debugging.md

@@ -0,0 +1,42 @@
+# Debugging
+
+The VS Code extension uses the Debug Adapter Protocol and the coordinator's
+authoritative process, task, attempt, and Debug Epoch APIs.
+
+## Launch
+
+Open your project and start "Clusterflux: Launch Virtual Process". The adapter
+builds the bundle, launches the selected entrypoint, and replaces its thread
+view with coordinator task snapshots. Product mode does not infer threads from
+completion events or demo fixtures.
+
+Each task view includes its logical task, attempt ID, definition, state, node,
+environment, arguments, typed handles, command status, VFS checkpoint, probe
+source when known, and restart compatibility. Unknown source remains unknown.
+
+## Breakpoints and Debug Epochs
+
+When a breakpoint is hit, the coordinator asks every active participant to
+freeze. A fully frozen epoch is a consistent all-participant view.
+
+If one or more participants cannot freeze within five seconds, the epoch becomes
+partially frozen when at least one participant did freeze. The adapter warns
+that running and frozen tasks do not form a consistent global snapshot. You may
+inspect the acknowledged participants and resume them cleanly. Missing or failed
+participants remain explicit.
+
+Stack frames, Wasm locals, task arguments, handles, command status, and output
+come from runtime acknowledgements. No frame or value is fabricated when the
+runtime did not report it.
+
+## Continue and restart
+
+Continue resumes only participants that acknowledged the freeze.
+
+Restarting a terminal task rebuilds the bundle and checks its clean entry
+checkpoint. A compatible restart creates a new attempt under the same logical
+task identity. Earlier attempt history remains inspectable, and the logical join
+continues. An active task or an incompatible boundary requires a whole-process
+restart.
+
+Source stepping is not simulated. Unsupported stepping fails explicitly.

docs/environments.md

@@ -0,0 +1,34 @@
+# Environments
+
+Declare environments in the bundle under:
+
+~~~text
+envs/<name>/Containerfile
+envs/<name>/Dockerfile
+~~~
+
+Reference one by logical name:
+
+~~~rust
+use clusterflux::env;
+
+let linux = env!("linux");
+~~~
+
+Bundle inspection reports every discovered environment and its digest:
+
+~~~bash
+clusterflux bundle inspect --project .
+~~~
+
+The bundle definition is authoritative for every spawn. The coordinator passes
+the declared environment identity and digest in the TaskSpec, and the node
+resolves that exact definition. A same-named local recipe with different bytes
+is rejected rather than substituted.
+
+On Linux, container-backed environments use rootless Podman. Clusterflux does
+not enable privileged containers by default.
+
+A task may use a bind-mounted local checkout for speed. That source path is
+non-hermetic. Choose a source snapshot when you need a reproducible input
+identity independent of the current working tree.

docs/getting-started.md

@@ -0,0 +1,110 @@
+# Getting started
+
+This guide uses the hosted coordinator. For your own coordinator, complete
+[Self-hosting](self-hosting.md) first and then return to the node and run steps.
+
+## 1. Install the commands
+
+~~~bash
+cargo install --path crates/clusterflux-cli --bin clusterflux
+cargo install --path crates/clusterflux-node --bin clusterflux-node
+cargo install --path crates/clusterflux-dap --bin clusterflux-debug-dap
+~~~
+
+Install rootless Podman on each Linux node that will execute container-backed
+environments.
+
+## 2. Sign in and create a project
+
+~~~bash
+clusterflux login --browser
+clusterflux auth status
+clusterflux project init   --new-project my-project   --name "My Project"   --yes
+clusterflux project select my-project
+~~~
+
+The browser flow is owned by the configured Authentik identity provider. The CLI
+stores an opaque Clusterflux session, not provider authorization codes or
+provider tokens.
+
+## 3. Enroll and start a node
+
+Create a short-lived grant:
+
+~~~bash
+clusterflux node enroll --project-id my-project --json
+~~~
+
+Generate a node key pair using your normal secret-management tooling. Exchange
+the grant once:
+
+~~~bash
+clusterflux node attach   --project-id my-project   --node workstation   --enrollment-grant "$ENROLLMENT_GRANT"   --public-key "$NODE_PUBLIC_KEY"   --json
+~~~
+
+Start the worker from the project directory with its private key supplied
+through protected environment or credential storage:
+
+~~~bash
+CLUSTERFLUX_NODE_PRIVATE_KEY="$NODE_PRIVATE_KEY" clusterflux-node   --coordinator https://clusterflux.michelpaulissen.com   --tenant "$TENANT"   --project-id my-project   --node workstation   --public-key "$NODE_PUBLIC_KEY"   --project-root "$PWD"   --worker   --emit-ready
+~~~
+
+The enrollment grant is not needed again. Stop and restart the worker with the
+same key identity. Check server-derived liveness with:
+
+~~~bash
+clusterflux node list
+clusterflux node status workstation
+~~~
+
+## 4. Inspect and run a bundle
+
+A project contains "clusterflux.toml", Rust workflow source, and any declared
+environments under "envs/".
+
+~~~bash
+clusterflux bundle inspect --project .
+clusterflux run --project . build
+~~~
+
+Choose another entrypoint by replacing "build". Clusterflux rejects an
+oversized or invalid bundle before it creates the virtual process.
+
+## 5. Inspect tasks and output
+
+~~~bash
+clusterflux process list
+clusterflux process status
+clusterflux task list
+clusterflux logs
+clusterflux artifact list
+~~~
+
+A failed task configured with "AwaitOperator" remains visible as awaiting
+action. Restart it as a new attempt under the same logical task identity:
+
+~~~bash
+clusterflux task restart <task-id> --process <process-id> --yes
+~~~
+
+## 6. Debug in VS Code
+
+Open the project in VS Code and start "Clusterflux: Launch Virtual Process".
+Set a breakpoint on a generated probe location and use the normal Threads,
+Stack, Variables, Continue, Pause, and Restart controls.
+
+A fully frozen Debug Epoch gives a consistent all-participant view. If a
+participant cannot freeze within five seconds, the adapter reports a partial
+epoch. You may inspect frozen participants, but values across running and frozen
+tasks are not a consistent global snapshot.
+
+## 7. Download an artifact
+
+~~~bash
+clusterflux artifact list --process <process-id>
+clusterflux artifact download <artifact-id>   --to ./output.bin   --max-bytes 67108864
+~~~
+
+The command opens a scoped, expiring download and verifies the artifact digest.
+It fails if the retaining node is stale, the bytes were garbage collected, the
+digest or size changed, or policy cannot reserve the transfer.

docs/nodes.md

@@ -0,0 +1,53 @@
+# Nodes
+
+Your node runs real commands and retains real output bytes. Enroll it once, then
+restart it with the same public-key identity.
+
+## Enroll
+
+~~~bash
+clusterflux node enroll --project-id my-project --json
+clusterflux node attach   --project-id my-project   --node workstation   --enrollment-grant "$ENROLLMENT_GRANT"   --public-key "$NODE_PUBLIC_KEY"
+~~~
+
+Treat the enrollment grant as a short-lived secret. It is exchanged once and is
+not a worker credential.
+
+## Run the worker
+
+~~~bash
+CLUSTERFLUX_NODE_PRIVATE_KEY="$NODE_PRIVATE_KEY" clusterflux-node   --coordinator https://clusterflux.michelpaulissen.com   --tenant "$TENANT"   --project-id my-project   --node workstation   --public-key "$NODE_PUBLIC_KEY"   --project-root "$PWD"   --worker   --emit-ready
+~~~
+
+Start the worker from the project directory when tasks need a local checkout.
+The worker reports detected command, container, source, VFS, environment, OS,
+and architecture capabilities. Use "clusterflux node attach --cap <capability>"
+only when detection needs an explicit override.
+
+## Liveness
+
+~~~bash
+clusterflux node list
+clusterflux node status workstation
+~~~
+
+The coordinator marks a node stale after its accepted heartbeat age exceeds the
+configured threshold. Stale nodes are excluded before placement and before a
+retained-node download link is created.
+
+## Local source
+
+A bind-mounted local checkout is fast and avoids transferring the repository,
+but it is non-hermetic: uncommitted changes, ignored files, and concurrent
+editor writes can affect the command. Use a content-addressed source snapshot
+when reproducibility matters.
+
+## Revoke
+
+~~~bash
+clusterflux node revoke workstation
+~~~
+
+Revocation removes the node identity and its live descriptor. Subsequent signed
+requests fail. Artifacts retained only by that node become unavailable unless
+you explicitly synchronized them elsewhere.

docs/security.md

@@ -0,0 +1,48 @@
+# Security
+
+## Authority
+
+Clusterflux separates four authority lanes:
+
+- Client sessions for people.
+- Agent public-key signatures for non-interactive workflows.
+- Node public-key signatures for enrolled workers.
+- Operator credentials for hosted administrative actions.
+
+The server derives tenant, project, identity, and permission scope from the
+authenticated lane. Request-body identity fields are not authority.
+
+## Sessions and keys
+
+Human login uses the configured identity provider and an opaque, nonce- and
+PKCE-bound transaction. The CLI never accepts provider claims or authorization
+codes as a session.
+
+Enrollment grants are short-lived and single-use. Node and agent signatures bind
+the canonical request body, timestamp, nonce, key fingerprint, and scope.
+Forged, replayed, expired, revoked, cross-tenant, and body-modified requests
+fail.
+
+Keep ".clusterflux/session.json", node private keys, agent private keys, and
+operator credentials out of source control. Use protected environment files or
+your secret manager.
+
+## Execution
+
+The coordinator does not run arbitrary native user commands. Nodes execute
+commands within their reported capabilities. Linux container environments use
+rootless Podman and avoid privileged defaults.
+
+Bundle size, canonical argument size, handle count, logs, task history, Debug
+Epoch state, relay state, and coordinator collections are bounded. Resource and
+relay reservations happen before unaffordable work or transfer begins.
+
+## Artifacts
+
+Artifact links are scoped, expiring, revocable, and bound to live metadata.
+Digest, size, producer, task attempt, and retaining source are checked. The
+coordinator retains metadata and bounded relay spool state, not durable artifact
+bytes by default.
+
+Report security issues privately to the repository owner rather than opening a
+public issue with credentials, keys, or exploit details.

docs/self-hosting.md

@@ -0,0 +1,38 @@
+# Self-hosting
+
+Run the public coordinator and node runtime without the hosted website.
+
+## Start a strict local coordinator
+
+Supply one scoped bootstrap session through protected service configuration:
+
+~~~bash
+CLUSTERFLUX_SELF_HOSTED_SESSION_SECRET="$SELF_HOSTED_SESSION_SECRET" CLUSTERFLUX_SELF_HOSTED_TENANT=my-team CLUSTERFLUX_SELF_HOSTED_PROJECT=my-project CLUSTERFLUX_SELF_HOSTED_USER=me clusterflux-coordinator --listen 127.0.0.1:7999
+~~~
+
+Connect the CLI without placing the secret in a process argument:
+
+~~~bash
+printf '%s\n' "$SELF_HOSTED_SESSION_SECRET" | clusterflux auth connect-self-hosted   --coordinator 127.0.0.1:7999   --tenant my-team   --project-id my-project   --user me   --session-secret-stdin
+~~~
+
+The CLI verifies the scope before writing ".clusterflux/session.json". On Unix,
+the session file uses mode "0600".
+
+## Attach nodes
+
+Use the same enrollment and worker flow described in [Nodes](nodes.md), with
+"--coordinator 127.0.0.1:7999".
+
+## Network boundary
+
+The native coordinator transport is plaintext and refuses non-loopback
+listeners. For another machine, keep the coordinator on loopback and use an
+authenticated SSH tunnel or deploy a trusted TLS reverse proxy that enforces the
+same client boundary. Do not expose the native port directly.
+
+## Administration
+
+Project, node, process, task, log, artifact, debug, quota, and self-hosted admin
+operations remain available through the public CLI/API. Authentik is one hosted
+identity deployment, not a requirement for your coordinator.

docs/task-abi.md

@@ -0,0 +1,49 @@
+# Task ABI
+
+Clusterflux tasks cross a canonical, versioned Wasm boundary.
+
+## Values
+
+Small arguments and results use canonical JSON-compatible values. Large or
+capability-bearing values use typed handles. A structured boundary value carries
+both its JSON structure and the complete typed handle table required to
+materialize it.
+
+Supported handles include VFS and artifact identities with full tenant, project,
+process, producer, digest, size, and path provenance. The SDK encodes handles as
+explicit placeholders and the runtime replaces only placeholders that match the
+declared type and metadata.
+
+A malformed, missing, extra, cross-scope, or type-mismatched handle fails closed.
+The runtime does not reconstruct authority from a string path or caller-supplied
+display field.
+
+## Tasks
+
+Use the macros and SDK types:
+
+~~~rust
+#[clusterflux::task]
+async fn compile(input: clusterflux::Artifact) -> Result<clusterflux::Artifact, String> {
+    // Task body
+}
+
+#[clusterflux::main]
+async fn build() -> Result<(), String> {
+    // Spawn and join tasks
+}
+~~~
+
+Each spawn produces a TaskSpec containing the logical task instance,
+definition, environment identity and digest, canonical arguments, handles, VFS
+epoch, bundle identity, dispatch ABI, and failure policy.
+
+Use distinct logical task instances when you spawn the same definition twice.
+A restart preserves that logical identity and creates a distinct attempt.
+
+## Failure policy
+
+"FailFast" is the default. "AwaitOperator" leaves the logical join pending after
+failure so you can inspect, accept, cancel, or restart the attempt. Side effects
+outside the VFS boundary are at-least-once across retries; make them idempotent
+or add your own deduplication key.