Public release release-98d969b26488

Source commit: 98d969b26488b4cda8b2381fa870276a00ca98ea

Public tree identity: sha256:d02dd1e8d3547a489bb300741bed544bdc60bb8fe0bd541fd43ce9bfa7129a3e

docs/architecture.md

@@ -0,0 +1,57 @@
+# Architecture
+
+Clusterflux separates a control plane from execution nodes.
+
+## Coordinator
+
+The coordinator owns identity scope, one-active-process admission, task
+placement, attempt history, Debug Epoch coordination, VFS metadata, artifact
+metadata, quotas, and secure relay reservations. Its async main may park and wake
+without consuming node compute.
+
+The coordinator does not execute arbitrary native commands or hosted
+containers. It also does not become a durable artifact store merely because it
+coordinates a download.
+
+## Nodes
+
+A node is an enrolled public-key identity. It reports capabilities and periodic
+heartbeats. The coordinator derives whether it is live from the last accepted
+heartbeat; a client-supplied "online" field is not placement authority.
+
+Nodes resolve bundle environments, execute Wasm tasks, run native commands, and
+retain artifact bytes. A node must prove the tenant, project, process, task, and
+key scope on every signed request.
+
+## Virtual process and tasks
+
+A Coordinator Project admits one active virtual process. The process contains:
+
+- one coordinator-hosted async main;
+- logical task instances created by that main;
+- one or more attempts for each logical task;
+- joins tied to the logical task, not to an individual attempt.
+
+"FailFast" makes a terminal failure visible to the join immediately.
+"AwaitOperator" keeps the join pending while an operator accepts, cancels, or
+restarts the failed task. Restart creates a distinct attempt but preserves the
+logical task identity.
+
+A terminal process releases the active slot automatically. A task parked for an
+operator is not terminal and continues to occupy the slot.
+
+## Durable and live state
+
+Projects, identities, credentials, permissions, hosted policy records, relay
+period usage, and relay reservation accounting may be stored in Postgres.
+Active processes, task placement, Debug Epochs, transient VFS state, relay byte
+streams, and live node state are bounded in memory and are not reconstructed as
+running after a coordinator restart. Pre-restart relay reservations are
+reconciled as abandoned before new links can be issued.
+
+## Protocol lanes
+
+Human clients use scoped sessions. Agents and nodes use separate signed
+public-key identities. Operator actions use a separate administrative
+credential. Authority comes from the authenticated lane and server-side scope,
+never from identity fields in an untrusted request body.

docs/artifacts.md

@@ -0,0 +1,38 @@
+# Artifacts
+
+Clusterflux separates artifact metadata from artifact bytes.
+
+## Publish metadata
+
+`flush()` makes a VFS artifact visible to downstream tasks and records its
+digest, size, producer task and attempt, and retaining locations. It does not
+upload bytes to the coordinator by default.
+
+Your node retains artifact bytes on a best-effort basis. If every retaining node
+is lost, stale, revoked, or garbage-collects the bytes, the artifact is
+unavailable. Clusterflux reports that state instead of inventing durable
+storage.
+
+## Move bytes explicitly
+
+Use `sync()` or an explicit export when you need another node or your own storage
+system to hold the bytes. Same-node reuse avoids a coordinator transfer.
+
+## Download
+
+~~~bash
+clusterflux artifact list --process <process-id>
+clusterflux artifact download <artifact-id> --to ./output.bin --max-bytes 67108864
+~~~
+
+Before returning a link, the service checks authorization, current retention,
+source-node liveness, live size, per-project and per-account policy, and relay
+capacity. It reserves the affordable ingress and egress budget atomically. The
+scoped link is unguessable, expires, can be revoked, and is bound to the actor,
+tenant, project, process, artifact, and policy context.
+
+The reverse stream counts framing, base64 expansion, failed bytes, and abandoned
+transfers. The CLI verifies the final digest. Hosted policy may impose
+per-project, per-tenant, and per-account size, concurrency, and period limits.
+Operators may disable relay traffic as an emergency safety control; one tenant's
+period usage does not consume a shared customer byte quota.

docs/debugging.md

@@ -0,0 +1,42 @@
+# Debugging
+
+The VS Code extension uses the Debug Adapter Protocol and the coordinator's
+authoritative process, task, attempt, and Debug Epoch APIs.
+
+## Launch
+
+Open your project and start "Clusterflux: Launch Virtual Process". The adapter
+builds the bundle, launches the selected entrypoint, and replaces its thread
+view with coordinator task snapshots. Product mode does not infer threads from
+completion events or demo fixtures.
+
+Each task view includes its logical task, attempt ID, definition, state, node,
+environment, arguments, typed handles, command status, VFS checkpoint, probe
+source when known, and restart compatibility. Unknown source remains unknown.
+
+## Breakpoints and Debug Epochs
+
+When a breakpoint is hit, the coordinator asks every active participant to
+freeze. A fully frozen epoch is a consistent all-participant view.
+
+If one or more participants cannot freeze within five seconds, the epoch becomes
+partially frozen when at least one participant did freeze. The adapter warns
+that running and frozen tasks do not form a consistent global snapshot. You may
+inspect the acknowledged participants and resume them cleanly. Missing or failed
+participants remain explicit.
+
+Stack frames, Wasm locals, task arguments, handles, command status, and output
+come from runtime acknowledgements. No frame or value is fabricated when the
+runtime did not report it.
+
+## Continue and restart
+
+Continue resumes only participants that acknowledged the freeze.
+
+Restarting a terminal task rebuilds the bundle and checks its clean entry
+checkpoint. A compatible restart creates a new attempt under the same logical
+task identity. Earlier attempt history remains inspectable, and the logical join
+continues. An active task or an incompatible boundary requires a whole-process
+restart.
+
+Source stepping is not simulated. Unsupported stepping fails explicitly.

docs/environments.md

@@ -0,0 +1,34 @@
+# Environments
+
+Declare environments in the bundle under:
+
+~~~text
+envs/<name>/Containerfile
+envs/<name>/Dockerfile
+~~~
+
+Reference one by logical name:
+
+~~~rust
+use clusterflux::env;
+
+let linux = env!("linux");
+~~~
+
+Bundle inspection reports every discovered environment and its digest:
+
+~~~bash
+clusterflux bundle inspect --project .
+~~~
+
+The bundle definition is authoritative for every spawn. The coordinator passes
+the declared environment identity and digest in the TaskSpec, and the node
+resolves that exact definition. A same-named local recipe with different bytes
+is rejected rather than substituted.
+
+On Linux, container-backed environments use rootless Podman. Clusterflux does
+not enable privileged containers by default.
+
+A task may use a bind-mounted local checkout for speed. That source path is
+non-hermetic. Choose a source snapshot when you need a reproducible input
+identity independent of the current working tree.

docs/getting-started.md

@@ -0,0 +1,125 @@
+# Getting started
+
+This guide uses the hosted coordinator. For your own coordinator, complete
+[Self-hosting](self-hosting.md) first and then return to the node and run steps.
+
+## 1. Install the commands
+
+~~~bash
+cargo install --path crates/clusterflux-cli --bin clusterflux
+cargo install --path crates/clusterflux-node --bin clusterflux-node
+cargo install --path crates/clusterflux-dap --bin clusterflux-debug-dap
+~~~
+
+Install rootless Podman on each Linux node that will execute container-backed
+environments.
+
+## 2. Sign in and select your hosted project
+
+~~~bash
+clusterflux login --browser
+clusterflux auth status
+clusterflux project list
+clusterflux project select <hosted-project-id>
+~~~
+
+The browser flow is owned by the configured Authentik identity provider. The CLI
+stores an opaque Clusterflux session, not provider authorization codes or
+provider tokens. Hosted login creates or links your single project; hosted
+admission rejects additional project creation.
+
+## 3. Enroll and start a node
+
+Create a short-lived grant:
+
+~~~bash
+clusterflux node enroll --project-id <hosted-project-id> --json
+~~~
+
+Create the local node identity and exchange the grant once:
+
+~~~bash
+clusterflux node attach \
+  --project-id <hosted-project-id> \
+  --node workstation \
+  --enrollment-grant "$ENROLLMENT_GRANT" \
+  --json
+~~~
+
+Clusterflux creates and stores the node key locally with restricted permissions.
+The enrollment grant is not needed again. Stop and restart the worker with the
+same stored identity:
+
+~~~bash
+clusterflux-node \
+  --coordinator https://clusterflux.michelpaulissen.com \
+  --tenant "$TENANT" \
+  --project-id <hosted-project-id> \
+  --node workstation \
+  --project-root "$PWD" \
+  --worker \
+  --emit-ready
+~~~
+
+Supplying `--public-key` and `CLUSTERFLUX_NODE_PRIVATE_KEY` is an advanced option
+for nodes whose key material is managed externally. Check server-derived
+liveness with:
+
+~~~bash
+clusterflux node list
+clusterflux node status workstation
+~~~
+
+## 4. Inspect and run a bundle
+
+A project contains `clusterflux.toml`, Rust workflow source, and any declared
+environments under `envs/`. Task arguments and handles use portable canonical
+representations; they are not host pointers or shared process memory.
+
+~~~bash
+clusterflux bundle inspect --project .
+clusterflux run --project . build
+~~~
+
+Choose another entrypoint by replacing `build`. Clusterflux rejects an oversized
+or invalid bundle before it creates the virtual process.
+
+## 5. Inspect tasks and output
+
+~~~bash
+clusterflux process list
+clusterflux process status
+clusterflux task list
+clusterflux logs
+clusterflux artifact list
+~~~
+
+A failed task configured with `AwaitOperator` remains visible as awaiting action.
+Restart it as a new attempt under the same logical task identity:
+
+~~~bash
+clusterflux task restart <task-id> --process <process-id> --yes
+~~~
+
+## 6. Debug in VS Code
+
+Open the project in VS Code and start `Clusterflux: Launch Virtual Process`.
+Set a breakpoint on a generated probe location and use Threads, Stack,
+Variables, Continue, Pause, and Restart.
+
+A fully frozen Debug Epoch gives a consistent all-participant view. If a
+participant cannot freeze within five seconds, the adapter reports a partial
+epoch. You may inspect frozen participants, but values across running and frozen
+tasks are not a consistent global snapshot. Continue resumes only participants
+that acknowledged the freeze.
+
+## 7. Download an artifact
+
+~~~bash
+clusterflux artifact list --process <process-id>
+clusterflux artifact download <artifact-id> --to ./output.bin --max-bytes 67108864
+~~~
+
+The command opens a scoped, expiring download and verifies the artifact digest.
+It fails if the retaining node is stale, the bytes were garbage collected, the
+digest or size changed, or policy cannot reserve the transfer.

docs/nodes.md

@@ -0,0 +1,66 @@
+# Nodes
+
+Your node runs real commands and retains real output bytes. Enroll it once, then
+restart it with the same public-key identity.
+
+## Enroll
+
+~~~bash
+clusterflux node enroll --project-id <project-id> --json
+clusterflux node attach \
+  --project-id <project-id> \
+  --node workstation \
+  --enrollment-grant "$ENROLLMENT_GRANT"
+~~~
+
+Treat the enrollment grant as a short-lived secret. It is exchanged once and is
+not a worker credential. By default, `clusterflux node attach` creates and stores
+a local node key with restricted permissions. Use an explicit `--public-key`
+with `CLUSTERFLUX_NODE_PRIVATE_KEY` only when external secret management owns the
+key pair.
+
+## Run the worker
+
+~~~bash
+clusterflux-node \
+  --coordinator https://clusterflux.michelpaulissen.com \
+  --tenant "$TENANT" \
+  --project-id <project-id> \
+  --node workstation \
+  --project-root "$PWD" \
+  --worker \
+  --emit-ready
+~~~
+
+Start the worker from the project directory when tasks need a local checkout.
+The worker reports detected command, container, source, VFS, environment, OS,
+and architecture capabilities. Use `clusterflux node attach --cap <capability>`
+only when detection needs an explicit override.
+
+## Liveness
+
+~~~bash
+clusterflux node list
+clusterflux node status workstation
+~~~
+
+The coordinator marks a node stale after its accepted heartbeat age exceeds the
+configured threshold. Stale nodes are excluded before placement and before a
+retained-node download link is created.
+
+## Local source
+
+A bind-mounted local checkout is fast and avoids transferring the repository,
+but it is non-hermetic: uncommitted changes, ignored files, and concurrent
+editor writes can affect the command. Use a content-addressed source snapshot
+when reproducibility matters.
+
+## Revoke
+
+~~~bash
+clusterflux node revoke workstation
+~~~
+
+Revocation removes the node identity and its live descriptor. Subsequent signed
+requests fail. Artifacts retained only by that node become unavailable unless
+you explicitly synchronized them elsewhere.

docs/security.md

@@ -0,0 +1,49 @@
+# Security
+
+## Authority
+
+Clusterflux separates four authority lanes:
+
+- Client sessions for people.
+- Agent public-key signatures for non-interactive workflows.
+- Node public-key signatures for enrolled workers.
+- Operator credentials for hosted administrative actions.
+
+The server derives tenant, project, identity, and permission scope from the
+authenticated lane. Request-body identity fields are not authority.
+
+## Sessions and keys
+
+Human login uses the configured identity provider and an opaque, nonce- and
+PKCE-bound transaction. The CLI never accepts provider claims or authorization
+codes as a session. Browser-login requests use a dedicated proxy route with
+per-client rate limiting. The hosted service binds only to loopback and does not
+use client-supplied forwarding headers as identity or authorization.
+
+Enrollment grants are short-lived and single-use. Node and agent signatures bind
+the canonical request body, timestamp, nonce, key fingerprint, and scope.
+Forged, replayed, expired, revoked, cross-tenant, and body-modified requests
+fail.
+
+Keep `.clusterflux/session.json`, node private keys, agent private keys, and
+operator credentials out of source control. Use protected environment files or
+your secret manager.
+
+## Execution
+
+The coordinator does not run arbitrary native user commands. Nodes execute
+commands within their reported capabilities. Linux container environments use
+rootless Podman and avoid privileged defaults.
+
+Bundle size, canonical argument size, handle count, logs, task history, Debug
+Epoch state, relay state, and coordinator collections are bounded. Resource and
+relay reservations happen before unaffordable work or transfer begins.
+
+## Artifacts
+
+Artifact links are scoped, expiring, revocable, and bound to live metadata.
+Digest, size, producer, task attempt, and retaining source are checked. The
+coordinator retains metadata and bounded relay spool state, not durable artifact
+bytes by default.
+
+Report security issues privately using [SECURITY.md](../SECURITY.md).

docs/self-hosting.md

@@ -0,0 +1,38 @@
+# Self-hosting
+
+Run the public coordinator and node runtime without the hosted website.
+
+## Start a strict local coordinator
+
+Supply one scoped bootstrap session through protected service configuration:
+
+~~~bash
+CLUSTERFLUX_SELF_HOSTED_SESSION_SECRET="$SELF_HOSTED_SESSION_SECRET" CLUSTERFLUX_SELF_HOSTED_TENANT=my-team CLUSTERFLUX_SELF_HOSTED_PROJECT=my-project CLUSTERFLUX_SELF_HOSTED_USER=me clusterflux-coordinator --listen 127.0.0.1:7999
+~~~
+
+Connect the CLI without placing the secret in a process argument:
+
+~~~bash
+printf '%s\n' "$SELF_HOSTED_SESSION_SECRET" | clusterflux auth connect-self-hosted   --coordinator 127.0.0.1:7999   --tenant my-team   --project-id my-project   --user me   --session-secret-stdin
+~~~
+
+The CLI verifies the scope before writing ".clusterflux/session.json". On Unix,
+the session file uses mode "0600".
+
+## Attach nodes
+
+Use the same enrollment and worker flow described in [Nodes](nodes.md), with
+"--coordinator 127.0.0.1:7999".
+
+## Network boundary
+
+The native coordinator transport is plaintext and refuses non-loopback
+listeners. For another machine, keep the coordinator on loopback and use an
+authenticated SSH tunnel or deploy a trusted TLS reverse proxy that enforces the
+same client boundary. Do not expose the native port directly.
+
+## Administration
+
+Project, node, process, task, log, artifact, debug, quota, and self-hosted admin
+operations remain available through the public CLI/API. Authentik is one hosted
+identity deployment, not a requirement for your coordinator.

docs/task-abi.md

@@ -0,0 +1,49 @@
+# Task ABI
+
+Clusterflux tasks cross a canonical, versioned Wasm boundary.
+
+## Values
+
+Small arguments and results use canonical JSON-compatible values. Large or
+capability-bearing values use typed handles. A structured boundary value carries
+both its JSON structure and the complete typed handle table required to
+materialize it.
+
+Supported handles include VFS and artifact identities with full tenant, project,
+process, producer, digest, size, and path provenance. The SDK encodes handles as
+explicit placeholders and the runtime replaces only placeholders that match the
+declared type and metadata.
+
+A malformed, missing, extra, cross-scope, or type-mismatched handle fails closed.
+The runtime does not reconstruct authority from a string path or caller-supplied
+display field.
+
+## Tasks
+
+Use the macros and SDK types:
+
+~~~rust
+#[clusterflux::task]
+async fn compile(input: clusterflux::Artifact) -> Result<clusterflux::Artifact, String> {
+    // Task body
+}
+
+#[clusterflux::main]
+async fn build() -> Result<(), String> {
+    // Spawn and join tasks
+}
+~~~
+
+Each spawn produces a TaskSpec containing the logical task instance,
+definition, environment identity and digest, canonical arguments, handles, VFS
+epoch, bundle identity, dispatch ABI, and failure policy.
+
+Use distinct logical task instances when you spawn the same definition twice.
+A restart preserves that logical identity and creates a distinct attempt.
+
+## Failure policy
+
+"FailFast" is the default. "AwaitOperator" leaves the logical join pending after
+failure so you can inspect, accept, cancel, or restart the attempt. Side effects
+outside the VFS boundary are at-least-once across retries; make them idempotent
+or add your own deduplication key.