michel/clusterflux-public
1
0
Fork 0
Code Issues Pull requests Projects Releases 21 Packages Wiki Activity Actions

Public dry run dryrun-309831e1e021

Browse source 
Source commit: 309831e1e021f962c118452336776fd9a94025f9

Public tree identity: sha256:6fa95c1745579bd6256dbeb3d476db0b07c2f24aa9213b0f7783ce1adfc8aca5
This commit is contained in:
Disasmer release dry run 2026-07-03 16:07:13 +02:00
commit f22d0a5791
113 changed files with 39348 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

447
README.md Normal file
View file

@ -0,0 +1,447 @@
# Disasmer
Disasmer is a distributed Wasm runtime for source-debuggable, local-first execution. The flagship workflow is a build written as Rust source code: one virtual process, many virtual threads/tasks, ordinary debugger controls, attached user nodes, and explicit artifact handling.
The MVP shape is:
```text
examples/launch-build-demo/envs/linux/Containerfile
examples/launch-build-demo/src/build.rs
```
```rust
use disasmer_sdk::env;
let linux = env!("linux");
```
`envs/<name>/Containerfile` or `envs/<name>/Dockerfile` defines an environment named `<name>`. Source code references that environment by logical name with `env!("name")`, not by runner label or machine name.
Bundle metadata is inspectable before launch:
```bash
disasmer bundle inspect --project examples/launch-build-demo
```
The inspection output includes discovered environments, selected input digests, default source-provider choices, and the bundle identity. Container images are not embedded by default.
The flagship demo lives in `examples/launch-build-demo`. Its build workflow is Rust source, not CI YAML, and its tasks return artifact/source handles rather than copying large bytes through task arguments.
## Quickstart
Prerequisites for the local MVP path are a Rust toolchain, Node.js for smoke
scripts, rootless Podman for Linux environment materialization, and VS Code when
debugging through the extension.
Build the public workspace and install the local command binaries:
```bash
cargo build --workspace
cargo install --path crates/disasmer-cli --bin disasmer
cargo install --path crates/disasmer-node --bin disasmer-node
cargo install --path crates/disasmer-coordinator --bin disasmer-coordinator
cargo install --path crates/disasmer-dap --bin disasmer-debug-dap
```
Install or run the VS Code extension from this checkout:
```bash
code --extensionDevelopmentPath "$(pwd)/vscode-extension" examples/launch-build-demo
```
For a persistent local install, copy or symlink `vscode-extension` into the VS
Code user extension directory under a versioned folder such as
`disasmer.disasmer-vscode-0.1.0`, then restart VS Code.
Inspect and run the flagship project:
```bash
disasmer bundle inspect --project examples/launch-build-demo
disasmer run --local --project examples/launch-build-demo build
```
For explicit process-boundary inspection, run the local services yourself:
```bash
disasmer-coordinator --listen 127.0.0.1:7999
disasmer node attach --coordinator 127.0.0.1:7999 --enrollment-grant local-dev --public-key local-node-key
disasmer run --local --coordinator 127.0.0.1:7999 --project examples/launch-build-demo build
```
`disasmer run --local` starts a loopback coordinator and user-attached local
node for the run when no coordinator address is supplied. `disasmer run [entry]`
selects an entrypoint such as `build`; `--project` overrides the project
directory. When the CLI has a hosted login and no local override is selected,
`disasmer run` uses the hosted coordinator. Use `--local` or
`--coordinator <host:port>` to force local coordinator mode.
In VS Code, open `examples/launch-build-demo`, start the `Disasmer: Launch
Virtual Process` configuration or press F5, inspect the Disasmer nodes,
processes, logs, artifacts, and inspector views, and use the ordinary debugger
controls for breakpoints, continue, pause, and restart. Artifact download
behavior is exercised by `node scripts/artifact-download-smoke.js`; final export
is explicit and should go through an attached receiver node or user-provided
storage integration rather than hidden coordinator storage.
Cleanup for the local quickstart is ordinary process and artifact cleanup: stop
the coordinator process, stop attached node processes, remove any temporary
artifacts under `target/acceptance/`, and remove the local VS Code extension copy
or symlink if one was installed.
## First-Run Diagnostics
Use these messages as first checks before debugging infrastructure:
- Missing nodes: attach a node with `disasmer node attach --coordinator <host:port>` or check the Disasmer Nodes view.
- Missing environments: add `envs/<name>/Containerfile` or `envs/<name>/Dockerfile`; the VS Code extension highlights unknown `env!("name")` references.
- Quota limits: hosted/community denials are returned before dispatch with a specific community tier reason.
- Unavailable artifacts: downloads fail before showing a link when retention, size, authorization, quota, or connectivity makes streaming impossible.
- Auth failures: browser login uses the device/browser flow, while agents and nodes use public-key identity and scoped enrollment grants.
- Failed debug freezes: all-stop reports the participant that could not freeze instead of claiming success.
- Source-provider capability gaps: source preparation stays pending until a node reports `SourceGit` or `SourceFilesystem`.
## Repository Shape
The public workspace contains the open-source contract layer:
```text
crates/disasmer-core shared identities, policy traits, scheduling, VFS/artifacts
crates/disasmer-coordinator local coordinator state model
crates/disasmer-dap Debug Adapter Protocol adapter for VS Code/debug clients
crates/disasmer-macros #[disasmer::main] and #[disasmer::task]
crates/disasmer-node node backend interfaces and local node runtime
crates/disasmer-cli CLI command surface
crates/disasmer-sdk user-facing Rust SDK
scripts/verify-public-split.sh
```
Hosted-only policy code lives under `private/**`. The public split is verified by copying the repo without `private/**` and running the public workspace tests.
## Public Release Dry Run
The public release dry run uses `https://disasmer.michelpaulissen.com:9443` as the
default operator endpoint for public CLI login and VS Code hosted/dry-run
configuration. The deployment should be a real externally reachable service
that can be shared with selected users, not a loopback-only, local-only, or mock
deployment. During the dry run, the name is intentionally not published in
public DNS until the `disasmer.michelpaulissen.com` DNS record is deployed; once
that record is live, no resolver override is required.
The default operator at `disasmer.michelpaulissen.com:9443` is the private hosted
coordinator from `private/hosted-policy`, with private hosted modules enabled.
The word `public` in this dry run describes the public Forgejo repository,
release assets, selected-user network reachability, and the public client
protocol used by released binaries. It does not mean deploying the standalone
open-source/public coordinator as the hosted operator. The public coordinator is
for local/self-hosted use; the dry-run operator must remain the private hosted
coordinator.
The dry-run acceptance validates both coordinator implementations. The private
hosted coordinator is validated through the live default-operator deployment,
service smoke, and public-repository e2e. The standalone public/open-source
coordinator is validated separately from the filtered public repository and
public release binaries in a local/self-hosted coordinator smoke; that validation
does not make it the hosted default operator.
Human browser login uses the nginx-served
`https://disasmer.michelpaulissen.com/auth/browser/start` path, which redirects
to the hosted Authentik account flow and returns to the CLI's localhost callback.
The `disasmer.michelpaulissen.com` website is deliberately barebones HTML with
no CSS for this dry run; layout and UX optimization belong to a later acceptance
phase.
The dry-run public repository is a real public repository on the Forgejo
instance at `git.michelpaulissen.com`, produced from the filtered public tree
without `private/**` or `experiments/**`. Its Forgejo Release publishes compiled
assets for the supported public binaries and the VS Code extension package, so a
user can download the appropriate artifacts from Forgejo and start without
building from source. Repeating the release later on a fresh public domain and a
public GitHub release should be mechanically straightforward, but that
fresh-domain/GitHub publication is outside this dry run.
Prepare the filtered public tree, release binary archive, VS Code extension VSIX,
checksums, and evidence manifest locally with:
```bash
node scripts/prepare-public-release-dryrun.js
```
To embed controlled-resolution fallback instructions in the selected-user assets,
set one of these when DNS is not live yet:
```bash
DISASMER_PUBLIC_DRYRUN_RESOLVER_INSTRUCTIONS=<instructions>
DISASMER_PUBLIC_DRYRUN_HOSTS_ENTRY="<ip-address> disasmer.michelpaulissen.com"
DISASMER_PUBLIC_RELEASE_DRYRUN_IP=<ip-address>
```
To publish the filtered snapshot to the Forgejo public repository, provide the
explicit remote and opt in to the push:
```bash
DISASMER_PUBLIC_REPO_REMOTE=ssh://git.michelpaulissen.com:2222/<owner>/<public-repo>.git \
DISASMER_PUBLISH_PUBLIC_TREE=1 \
node scripts/prepare-public-release-dryrun.js
```
The manual `Public release dry run assets` Forgejo workflow runs the same
preparation path for Linux and, when the intermittent `windows` runner is
online, for Windows. Upload the produced assets, including
`disasmer-vscode-*.vsix`, and
`public-release-manifest.json` to the Forgejo Release for the public repository.
The generated `DISASMER_PUBLIC_DRYRUN_GETTING_STARTED-*.md` asset is the
selected-user quickstart, and `DISASMER_PUBLIC_DRYRUN_INVITE-*.md` is the short
invite that can be shared with selected external users for installing the
binaries, resolving the public operator DNS name, and connecting to the default
operator.
The workflow artifacts are staging evidence; the actual Forgejo Release is only
accepted once those assets are attached to the public repository's Release.
Publish the Forgejo Release through the API after the filtered public tree has
been pushed:
```bash
DISASMER_FORGEJO_TOKEN=<token> \
node scripts/publish-public-release-dryrun.js
```
The publisher infers the Forgejo owner and repository name from
`public-release-manifest.json`. Set `DISASMER_PUBLIC_REPO_OWNER=<owner>` and
`DISASMER_PUBLIC_REPO_NAME=<public-repo>` only to override that manifest value.
Before publishing, the non-e2e preflight can verify that the manifest matches
the current commit, the filtered public branch points at the prepared tree, all
local assets exist, and checksums match:
```bash
node scripts/public-release-dryrun-preflight.js
```
Selected external users, such as friends invited to the dry run, should be able
to download those assets from Forgejo, resolve
`disasmer.michelpaulissen.com` through public DNS or the supplied fallback
instructions, and get a realistic product experience without a source build or
private repo access.
Once the real hosted service is deployed, record private deployment evidence
against that externally reachable service with:
```bash
node private/hosted-policy/scripts/prepare-public-release-dryrun-deployment.js
DISASMER_PUBLIC_RELEASE_DRYRUN_SERVICE_ADDR=disasmer.michelpaulissen.com:9443 \
DISASMER_PUBLIC_RELEASE_DRYRUN_OIDC_ISSUER_URL=<test-issuer-url> \
DISASMER_PUBLIC_RELEASE_DRYRUN_OIDC_CODE=<authorization-code> \
node private/hosted-policy/scripts/public-release-dryrun-service-smoke.js
```
The deployment bundle includes `disasmer-hosted-service`, a systemd unit that
binds `0.0.0.0:9443`, and a deployment manifest. Installing and starting that
bundle on the external host is required before the service smoke can be accepted
as real dry-run evidence.
After the public Forgejo Release, deployment, service smoke, public-operator
compatibility smoke, and standalone public-coordinator smoke have produced
evidence, run the public-repository dry-run e2e against the deployed operator:
```bash
DISASMER_PUBLIC_RELEASE_DRYRUN_E2E=1 \
DISASMER_PUBLIC_RELEASE_DRYRUN_SERVICE_ADDR=disasmer.michelpaulissen.com:9443 \
DISASMER_PUBLIC_RELEASE_DRYRUN_OIDC_ISSUER_URL=<test-issuer-url> \
DISASMER_PUBLIC_RELEASE_DRYRUN_OIDC_CODE=<authorization-code> \
node scripts/public-release-dryrun-e2e.js
```
After the public-repository e2e dry run has produced evidence, run the final
consistency verifier:
```bash
DISASMER_PUBLIC_RELEASE_DRYRUN_FINAL=1 \
node scripts/public-release-dryrun-final-evidence.js
```
The public-repository e2e evidence is
`target/acceptance/public-release-dryrun-e2e.json`. It must record the Forgejo
public repository URL, release name, source commit, filtered public tree
identity, default operator endpoint, controlled service address, commands, tool
versions, and true checks for downloaded release assets, checksum verification,
clean public checkout, build or install from the public repo/assets, default
operator selection, browser or CLI login, user node attachment, flagship
workflow run, VS Code debugger behavior, logs, artifact metadata, and artifact
download or explicit export.
## Coordinator State
The coordinator separates durable project and identity state from live runtime state. Tenants, users, projects, node identities, credentials, source-provider configuration, durable service policy records, and explicit project permissions can be stored in Postgres. Active virtual processes, live virtual threads, scheduler state, debug epochs, ephemeral VFS manifests, and transient artifact locations remain in coordinator memory for the MVP and are not represented in the Postgres schema.
## Local-First Builds
Disasmer is designed so local source checkouts and large outputs stay node-local unless user code or policy explicitly moves bytes.
`flush()` publishes metadata and visibility information. It makes produced artifacts visible to downstream tasks without implying durable coordinator storage.
`sync()` is explicit. It may move bytes to another node or user-provided storage according to code or configured policy.
User-provided storage/export integrations are ordinary project code or external
commands, such as publishing through a CLI. Disasmer records metadata and
coordinates capability, scheduling, and transfer decisions; it does not provide
or manage an explicit artifact-store feature as part of the MVP.
Artifacts are best-effort retained on nodes by default. If unsynced node-local bytes are garbage collected or the retaining node is lost, the artifact becomes unavailable instead of silently recovering from coordinator storage.
Artifact downloads are created only when current retention, size, authorization, and community tier accounting allow it. Download links are scoped to the tenant, project, process, artifact, actor, and policy context, expire after a bounded TTL, can be revoked, and streaming usage is charged before and during transfer.
On Linux nodes, Containerfile and Dockerfile environments are materialized with rootless user Podman. The node runtime has a concrete runner path for `podman build` followed by `podman run`; tests use a recording runner, while real attached nodes can use the `std::process::Command` runner. A local build from an attached local checkout uses a node-local bind mount into the selected environment; the coordinator does not create a full-repo tarball or route compiler file reads. Command output is associated with the virtual thread that started it and can be staged into the VFS artifact namespace. `node scripts/wasmtime-node-smoke.js` builds the launch example for `wasm32-unknown-unknown`, runs its exported task through the node Wasmtime runtime, and verifies a Wasmtime task can invoke a native command through the node `disasmer.cmd_run` host import without moving command execution into the hosted coordinator.
## Nodes And Trust
Users attach their own nodes for real work. The hosted coordinator is a control plane for identity, rendezvous, scheduling metadata, debug sessions, logs, artifact metadata, and operator state. The community tier does not provide arbitrary hosted native commands or hosted containers.
Agent and worker automation should use enrolled public-key identity. User OAuth or browser session tokens are not task credentials and should not be passed to nodes.
```bash
disasmer login
disasmer login --browser
disasmer agent enroll --public-key <agent-public-key>
DISASMER_AGENT_PUBLIC_KEY=<agent-public-key> disasmer run build
disasmer node attach --enrollment-grant <grant> --public-key <node-public-key>
disasmer-node --coordinator https://disasmer.michelpaulissen.com:9443 --tenant <tenant> --project-id <project> --node <node-id> --public-key <node-public-key> --enrollment-grant <grant> --worker --emit-ready
```
`disasmer login` uses a short-lived human device flow and does not ask users to paste long-lived secrets. `disasmer login --browser` opens the user's browser, sends the user through the hosted account flow, and completes through a local callback. `disasmer login --browser --plan` is the diagnostic JSON login plan path. Node attach exchanges a short-lived enrollment grant for a scoped long-lived node identity and then exits; a long-lived `disasmer-node --worker` process must be running for coordinator-side launches and DAP live-services debugging to place real command/container work on that node.
Hosted agent public keys are project-scoped records: a signed-in user can
register, list, rotate, and revoke an agent key without giving the agent browser
or OAuth credentials. An enrolled agent can run CLI commands non-interactively
with `DISASMER_AGENT_PUBLIC_KEY`; the CLI records the agent public-key
fingerprint in its run plan instead of starting a browser flow.
`disasmer node attach` auto-detects OS, architecture, command/container,
VFS/artifact, source-provider, and environment capabilities. `--cap <name>` is
available as an override for unusual local setups, not as required normal
configuration.
For local process-boundary testing, the public workspace includes a TCP JSON-line coordinator service and a node runtime binary:
```bash
cargo run -p disasmer-coordinator -- --listen 127.0.0.1:0
cargo run -p disasmer-node -- --coordinator <host:port> --project examples/launch-build-demo
```
The local smoke starts those as separate processes, keeps one node-to-coordinator JSON-line session open for attach, heartbeat, process start, reconnect, and completion, has the node run the demo test command, stages stdout as a VFS artifact, and records task completion metadata with the coordinator. The cancellation smoke keeps a node session open while a separate client asks the coordinator to cancel the task; the node polls task control, records a cancelled terminal event, and exits without reporting a false success.
For self-hosted local clouds, trusted teams, or VPN deployments, run the public
coordinator on the chosen listen address and attach team-owned nodes with their
public keys. This path uses the open-source coordinator, scheduler, node
heartbeat, task metadata, retained-node downloads, and direct node-to-node export
planning without private hosted OIDC or community tier policy modules.
`node scripts/self-hosted-coordinator-smoke.js` exercises that public path with
two trusted Linux nodes.
## Debugging
The VS Code extension contributes the normal `disasmer` debug type, refreshes bundle metadata before launch, and starts the DAP adapter. The adapter presents one virtual process with virtual threads/tasks, all-stop breakpoint and pause behavior, thread stacks, task args, handles, command status, recent output, continue, and selected-task restart controls. Compatible source edits restart the selected task from a VFS checkpoint; incompatible edits reject selected-task restart with a whole virtual-process restart message. The F5 and DAP smokes run the adapter in local-services launch mode, so the debug variables include task completion recorded through the local coordinator/node process boundary. Disasmer-specific side views expose nodes, virtual processes, logs, artifacts, and inspector state alongside the normal debugger UI.
In live-services mode, the DAP adapter starts the coordinator-side virtual
process without requiring a node at launch. When that process reaches a virtual
task with `Command` or container capability requirements, the coordinator places
that task on a capable node; an explicitly attached/running worker node polls
the coordinator for that assignment.
The built-in operator panel preview is rendered by the coordinator from live
process, task, log, and artifact metadata. It uses typed widgets only, keeps
program UI events scoped and rate-limited, and keeps control-plane actions such
as debug, cancel, restart, and artifact download available when program UI events
are disabled.
## Windows
Windows node support uses the same node protocol and capability model as Linux. MVP Windows command execution is user-attached development execution, labeled `windows-command-dev`. Production-grade managed Windows sandboxing is behind an explicit backend stub until it is implemented and validated independently. When the acceptance report shows `windows_validation: "not-run"`, Windows support is best-effort and unvalidated for that release; the public smoke only verifies protocol, placement, metadata, and download behavior for a Windows-capable node identity.
Real Windows validation runs through the manual Forgejo workflow
`.forgejo/workflows/windows-validation.yml` when the intermittent Windows runner is
online. That workflow records `windows_validation: "forgejo-windows-runner"` in
the acceptance report, runs `disasmer node attach` against a coordinator, starts a
Windows node process, executes a simple `windows-command-dev` command, publishes
artifact metadata, checks Windows placement, and verifies the debugger can show a
Windows virtual thread.
## Source Providers
Git is an optional source-provider module included by default. The VFS core depends on source-provider manifests and snapshot handles, not on Git internals. Non-Git source providers can implement the public source-provider interface and run snapshot preparation as node work instead of requiring coordinator filesystem access.
Source preparation is scheduled as node work. The coordinator can return a
pending source-preparation status while waiting for any capable node, then assign
the work after a node reports `SourceGit` or `SourceFilesystem` capability; it
does not need checkout or provider access itself.
Source-provider manifests are validated as public input and must declare a
local-first transfer policy: local source bytes stay node-local, the coordinator
does not receive source bytes by default, and remote preparation uses required
content or explicit snapshot chunks rather than a full-repo tarball.
## Verification
Run the public workspace checks:
```bash
scripts/acceptance-public.sh
```
Or run the individual public checks:
```bash
cargo test --workspace
node scripts/acceptance-report-smoke.js
node scripts/public-private-boundary-smoke.js
node scripts/release-blocker-smoke.js
node scripts/docs-smoke.js
node scripts/public-release-dryrun-contract-smoke.js
node scripts/wasmtime-node-smoke.js
node scripts/podman-backend-smoke.js
node scripts/vscode-extension-smoke.js
node scripts/vscode-f5-smoke.js
node scripts/node-attach-smoke.js
node scripts/local-services-smoke.js
node scripts/cancellation-smoke.js
node scripts/cli-local-run-smoke.js
node scripts/artifact-download-smoke.js
node scripts/artifact-export-smoke.js
node scripts/operator-panel-smoke.js
node scripts/source-preparation-smoke.js
node scripts/scheduler-placement-smoke.js
node scripts/windows-best-effort-smoke.js
node scripts/quic-smoke.js
node scripts/flagship-demo-smoke.js
node scripts/dap-smoke.js
node scripts/prepare-public-release-dryrun.js
scripts/verify-public-split.sh
```
When the Forgejo Windows runner is online, run the manual `Windows validation`
workflow as the release Windows gate. The local equivalent on a Windows machine is:
```powershell
$env:DISASMER_WINDOWS_VALIDATION = "forgejo-windows-runner"
node scripts/acceptance-report.js windows
cargo fmt --all --check
cargo test -p disasmer-node windows_backend_is_labeled_user_attached_dev_execution
node scripts/windows-runner-smoke.js
```
Run private hosted policy checks separately:
```bash
scripts/acceptance-private.sh
```
The private hosted gate includes `public-operator-compat-smoke.js`, which starts
the hosted dry-run service locally and verifies that the public CLI and public
node runtime can attach, launch work through coordinator task assignment, and
publish debug/log/artifact metadata through the hosted operator protocol bridge.
Both acceptance scripts write an environment report under `target/acceptance/`
with the commit SHA, toolchain versions, OS/kernel, Podman/Postgres discovery,
browser/VS Code harness metadata, and Windows-validation status. The report
records Podman as `available` only when rootless Podman is discoverable; if it is
missing or not rootless, Linux Podman backend behavior is marked `incomplete`
with the reason and the Podman backend smoke blocks acceptance with the same
incomplete reason.