clusterflux-public/MVP.md

Disasmer MVP Engineering Plan

Status: draft MVP execution plan
Audience: engineering / founder-engineer
Goal: build a public-launchable MVP that proves Disasmer can replace CI/build systems with a general distributed Wasm runtime whose flagship use case is distributed, local-first, source-debuggable builds.

---

0. Executive summary

Disasmer should launch as a general distributed Wasm runtime with a build-system experience so strong that it is the obvious wedge. The MVP is not a toy local scheduler and not merely a CI runner with nicer YAML. It must demonstrate the core thesis:

A build can be a normal Rust program, running as one virtual process across multiple machines, with build steps represented as virtual threads/tasks, artifacts represented by a low-latency virtual filesystem, and debugging represented as one normal VS Code debug session.

The launchable MVP should include:

• Open-source local runtime, SDK, CLI, node runtime, protocol definitions, and VS Code extension.
• Hosted coordinator from day one, with persistent project/identity state in Postgres and private hosted Authentik/OIDC/community-policy modules; the community tier coordinates but does not provide arbitrary hosted compute.
• User-attached Linux and Windows nodes.
• Containerfile-based environment discovery for MVP user projects.
• Rootless Podman backend for Linux command execution.
• Windows command backend sufficient for executing commands unsandboxed, with the Windows sandboxing backend explicitly stubbed (to be implemented later) and isolated behind an interface.
• Wasmtime-based execution for Disasmer guest/orchestration code.
• Native command execution only through node runtimes, never through the hosted coordinator control plane.
• Small serializable task arguments and results.
• Large inputs/outputs passed through handles: SourceSnapshot, Artifact, Blob, and VFS manifests.
• Local-first VFS where flush() publishes metadata and sync() triggers explicit user-chosen export/transfer/durability behavior; artifacts are otherwise ephemeral node-local data.
• Git support as an optional source-provider module that is included by default, with non-Git providers pluggable and dependencies avoidable when unused.
• All-stop debugging with Debug Epochs as a mandatory MVP capability.
• A Disasmer DAP adapter that targets VS Code first while staying usable by other DAP clients.
• Task restart from filesystem/artifact checkpoint boundaries, not live stack migration or hot-swapping.
• Abuse controls designed into the object model and control plane, not bolted on later.
• Operator panel foundations in the protocol and runtime-state model, with the full immediate-mode panel as the second product milestone.

The flagship demo should make people immediately understand why this is not “yet another CI system”:

1. The repo has envs/linux/Containerfile and envs/windows/Containerfile.
2. The build is one Rust source file using #[disasmer::main], #[disasmer::task], spawn::task, env!("linux"), env!("windows"), cmd!(...), fs::flush(), and fs::sync() / artifact::export_to_node().
3. A Linux node and a Windows node can be attached to the hosted coordinator.
4. Pressing F5 in VS Code starts one virtual process.
5. VS Code shows main, compile linux, compile windows, and package as threads of one debug target.
6. A breakpoint in the Windows build task stops the whole virtual process using a Debug Epoch.
7. The user edits the build logic, restarts only the failed task from a clean VFS checkpoint, and continues.
8. Artifacts appear through the VFS without coordinator-routed bulk data.

---

1. Product stance and launch bar

1.1 Positioning

Disasmer should be positioned as:

A distributed Wasm runtime that makes distributed execution feel like ordinary source-level-debuggable multithreaded programming, with build systems as the first killer use case.

It is not primarily:

• a hosted CI SaaS;
• a YAML syntax replacement;
• a remote shell wrapper;
• a Kubernetes abstraction;
• a general-purpose compute subsidy;
• a task queue with logs;
• or a build cache alone.

It should feel closer to:

“What if my build system were an actual program, and the debugger saw the whole distributed build as one process?”

1.2 Public-launch promise

The launch should be built around a concrete promise:

Add a Containerfile.
Reference it from Rust with env!("name").
Press F5.
Debug the distributed build like one normal program.

The public launch should be able to show a short video or GIF with these beats:

1. VS Code open on a Rust build program.
2. env!("linux") and env!("windows") autocomplete from envs/**/Containerfile.
3. Press F5.
4. Hosted coordinator shows two attached nodes.
5. Linux and Windows build tasks start.
6. VS Code thread list shows virtual threads.
7. Breakpoint hits in the Windows task.
8. Both tasks stop.
9. User inspects args/locals/artifact handles.
10. User restarts only the failed task.
11. Final artifacts are explicitly exported.

1.3 What must be true before public launch

The MVP is launchable only if these are true:

• A user can use the hosted coordinator without you granting them compute.
• A user can attach their own Linux node and run a build task on it.
• A user can attach a Windows node and run a Windows command task on it.
• Source code references environments by logical name, not by runner label or machine name.
• Local-first Linux builds do not upload the repo, tar the checkout, route file reads through the coordinator, or automatically upload build outputs.
• flush() is cheap and metadata-first.
• sync() is explicit and may move bytes.
• The debugger shows one process and many virtual threads.
• A Debug Epoch stops all Wasm tasks and controlled native command tasks for the virtual process.
• Task restart works from task entrypoint or last published VFS checkpoint.
• Community hosted users cannot run arbitrary containers or native commands on your infrastructure.
• Every public object has tenant ownership, quota accounting, and authorization checks.

1.4 What can be rough at launch

These can be rough, provided the rough edges are honest and isolated:

• Windows sandboxing can be a stubbed backend with a clear windows-command-dev capability, as long as hosted managed Windows compute is not offered to untrusted users.
• Nix can be included as a default plugin but not required for the flagship Containerfile demo.
• The operator panel can be scaffolded and visible as a preview, with the full immediate-mode UI in the second milestone.
• Debugger variable inspection can prioritize task args, Disasmer handles, top-level locals around Disasmer API calls, stack frames, and command status before supporting every possible Rust compiler local perfectly.
• NAT traversal can be coordinator-aided without relay. If direct node-to-node connectivity fails, scheduling and artifact flows should avoid requiring peer transfer rather than silently relaying bulk data through the coordinator.

---

2. Hard MVP decisions from the project constraints

2.1 Product scope

The MVP includes both:

• open-source local-first runtime, because credibility depends on not hiding the core runtime; and
• hosted coordinator, because the GitHub Actions replacement story needs an easy entry point, public rendezvous, auth, node registry, debug sessions, and a web surface.

The hosted coordinator is a control plane, not a general-purpose compute platform.

2.2 First target user

The first target user is:

A developer or small team that currently uses GitHub Actions, Forgejo Actions, or similar CI, and wants something more programmable, local-first, debuggable, and self-hostable.

The first user does not need managed compute from Disasmer. They need:

• local and self-hosted nodes;
• a hosted coordinator for convenience;
• VS Code debugging;
• artifact metadata, ephemeral artifact handles, and explicit output export;
• a better mental model than YAML pipelines;
• and a credible path to replacing CI.

2.3 Host platforms

MVP node platforms:

Linux    required
Windows  required
macOS    not required for launch, but kept in the model
BSD-like not required for launch, but kept in the model

Linux should be the most complete backend. Windows should be complete enough to run a real attached worker and prove cross-platform task placement. macOS remains a design-supported later node type.

2.4 Environment support

MVP user-facing environment support:

Containerfile / Dockerfile discovery: required
Nix plugin: included by default, but not launch-critical for all flows
Windows sandbox backend: interface + stub, not trusted managed compute

The MVP should discover:

envs/<name>/Containerfile
envs/<name>/Dockerfile

It may also discover:

envs/<name>/flake.nix
envs/<name>/flake.lock

But the flagship launch demo should not depend on Nix unless it is already stable in the implementation.

2.5 Execution support

The MVP must run both:

• Wasm task entrypoints, using Wasmtime on node runtimes; and
• native commands, through cmd!(...), executed only by node runtimes inside declared environments.

The coordinator must not run native commands or containers. The hosted control loop may run zero-capability Wasm, but only with strict fuel, memory, log, state, and wall-clock limits.

2.6 Wasm model

Use the simplest Wasmtime model that is secure and functional:

Core Wasm module + explicit host imports first.
Component Model support later or behind an internal abstraction.

Reasoning:

• Core Wasm embedding is enough for task entrypoints, host imports, memory access, and debug probes.
• The MVP does not need to expose a stable cross-language component ABI yet.
• The runtime should keep an internal GuestAbi trait so Component Model support can replace or augment this later.

2.7 Data model

MVP task boundaries allow only:

small serializable values
immutable content-addressed objects
Disasmer runtime handles

Do not allow:

raw pointers
borrowed references
local file handles
sockets
process handles
MutexGuards
unbounded byte vectors
native closures
live stack migration

Large data moves through:

SourceSnapshot
Blob
Artifact
VirtualFile
VfsManifest

2.8 Checkpoint model

MVP checkpointing means:

filesystem/artifact checkpoints for task restart

It does not mean:

live Wasm stack checkpointing
native process checkpointing
socket checkpointing
transparent migration of running processes
hot-swapping code into existing stack frames

A restartable task must be recreated from:

task name
new module hash
original serialized args
same environment handle
same source snapshot or selected dirty overlay
latest allowed VFS checkpoint

2.9 Debugging model

All-stop debugging is mandatory.

The debugger must present:

one virtual process
many virtual threads/tasks
normal breakpoints in Disasmer Rust task code
normal thread list
stacks
selected locals / task args / handle inspection
stdout/stderr per task
artifacts and VFS state in optional views
restart selected task

The MVP should provide a Disasmer-owned DAP adapter. VS Code is the first supported client, but the adapter should speak ordinary DAP so other IDEs or tools can integrate later without Disasmer providing those integrations. LLDB/lldb-dap can be used internally later for richer DWARF support, but the virtual-process illusion should remain owned by Disasmer.

2.10 Transport

Use Quinn for MVP QUIC transport, behind a transport abstraction:

node-to-node data/control links: QUIC over Quinn
node-to-coordinator persistent session: QUIC preferred, HTTPS/WebSocket allowed for auth/bootstrap
bulk transfer: node-to-node only, no coordinator bulk relay

Quinn is the MVP default because it is Rust-native and async-friendly, which keeps packaging and dogfooding simpler than an external C runtime dependency. Keep the Transport trait strict enough that quiche, TQUIC, or another implementation can replace it later if benchmarks, platform packaging, or protocol features justify a change.

The coordinator can help with rendezvous and endpoint candidate exchange. It should not relay bulk traffic. MVP can avoid cross-node bulk transfer if direct connectivity fails by scheduling tasks where the data already is or requiring explicit export to a receiver node or user-configured storage integration.

2.11 Storage

Use an abstract coordinator store with a strict split between persistent product state and ephemeral virtual-process state.

Persistent state should use Postgres in the hosted deployment and should be limited to things that must survive coordinator restart:

users / tenants / memberships where applicable
projects
node identities and enrollment records
node public keys / credentials / revocation state
bundle registry and environment metadata worth reusing
source-provider configuration metadata
hosted/community policy configuration
billing/plan flags later
longer-lived audit events where needed

Ephemeral runtime state should live in coordinator memory for the MVP:

active virtual processes
virtual threads and task placement
Debug Epochs
pending spawn queues
scheduler decisions
live node sessions and heartbeats
recent logs/events
VFS manifest cache for active processes
artifact metadata for active processes
quota counters for active processes, except hosted/community abuse ledgers that the private service wants to persist

If the coordinator restarts during the MVP, active virtual processes may be lost and should restart cleanly. That is acceptable. Do not overbuild durable process recovery before the runtime/debugging thesis is proven.

The store abstraction still matters because later the coordinator may need durable process recovery, Raft, sharding, or a different store. The MVP should not spend engineering effort on distributed consensus before the product is proven.

2.12 Auth

Support two auth modes from the start:

hosted/community service auth: Authentik through OAuth2/OIDC, implemented in a private hosted module
CLI human auth: browser callback against the hosted service
CLI/agent auth: public-key identity with explicit enrollment/authorization
node attach: short-lived enrollment token -> long-lived node key/certificate
service-to-service: scoped service tokens where needed
open-source/private coordinator auth: simple local/team trust model with pluggable auth hooks, not the hosted OIDC stack by default

User OAuth tokens must not be passed to nodes. Nodes authenticate with node identity, and tasks receive only explicit, scoped, short-lived task tokens or secrets.

The CLI should be comfortable for both humans and agents. Browser login is fine for a developer at a workstation; agents should be able to use a generated keypair, an enrollment challenge, and a scoped project/node credential without opening a browser.

2.13 Community hosted abuse stance

The community hosted tier should prove the platform, not subsidize builds. The word “community” is intentional: it should be useful for serious users, while enterprise/team use cases should naturally need more than one active process, longer retention, stronger identity, or managed capacity.

Community hosted mode supports:

sign in
create project (may be implicitly created, max 1)
create virtual process (may be implicitly created, max 1)
open coordinator UI
attach own node (max 4 nodes)
spawn tasks onto own node (max 8 tasks in flight)
debug from VS Code
see logs/artifact metadata for active processes (logs can't be bigger than 16KiB)

Community hosted mode does not support:

hosted containers
hosted native commands
default hosted artifact storage
public artifact hosting
network access from hosted Wasm
long-running compute
relay bandwidth

2.14 Operator panel

The operator panel is second milestone, but the MVP should design toward it:

• runtime-state structures for panel state and events;
• protocol namespace for typed UI updates;
• control-plane UI slot for a panel;
• built-in widgets only;
• no custom HTML/JS;
• read-only behavior when the virtual process is stopped in a debugger.
• The panel has very tight size/state constraints for community mode

For public launch, a fixed coordinator dashboard is enough if it clearly hints at the immediate-mode panel API.

---

3. MVP architecture

3.1 High-level architecture

VS Code
  ├─ Disasmer VS Code extension
  └─ Disasmer DAP adapter
        │
        ▼
Hosted or local coordinator
  ├─ Auth / tenants / projects
  ├─ node registry and rendezvous
  ├─ virtual process registry
  ├─ scheduler
  ├─ debug epoch coordinator
  ├─ VFS/artifact metadata for active processes
  ├─ hosted/community policy hooks
  ├─ operator panel state/events scaffold
  ├─ in-memory active runtime state
  └─ Postgres persistent project/identity store
        │
        ├────────────── QUIC / control session ───────────────┐
        ▼                                                     ▼
Linux node runtime                                      Windows node runtime
  ├─ Wasmtime task runner                               ├─ Wasmtime task runner
  ├─ command runner                                     ├─ command runner
  ├─ Podman environment backend                         ├─ Windows command backend
  ├─ Nix plugin backend                                 ├─ Windows sandbox stub backend
  ├─ VFS overlay store                                  ├─ VFS overlay store
  ├─ ephemeral artifact/blob cache                      ├─ ephemeral artifact/blob cache
  ├─ debug agent                                        ├─ debug agent
  ├─ node cache                                         ├─ node cache
  └─ Quinn peer transport                               └─ Quinn peer transport

3.2 Control plane vs data plane

The coordinator owns:

identity
membership
node capabilities
bundle metadata
virtual process metadata
task placement decisions
debug epochs
VFS manifest metadata for active processes
artifact metadata for active processes
quota accounting
operator panel metadata
rendezvous

All of these need constraints so that the community service can't be abused.

The nodes own:

Wasm execution
native command execution
environment materialization
source checkout access
build caches
VFS bytes
artifact bytes
large blob transfer
debug stop/resume implementation

User-attached nodes are the users' responsibility and can run broad local capabilities. The hosted coordinator must still validate what capabilities are accepted for a hosted/community process, but the open-source/private coordinator should not block normal local-node capabilities by default.

The coordinator must not become:

a network filesystem
a blob proxy
a remote compiler filesystem
a hidden build runner
a relay by default

3.3 Recommended repository layout

disasmer/
  crates/
    disasmer-sdk/              # guest-facing Rust API
    disasmer-macros/           # #[disasmer::main], #[disasmer::task], env!()
    disasmer-abi/              # guest/host ABI structs and serialization
    disasmer-cli/              # user CLI
    disasmer-coordinator/      # hosted/local coordinator service
    disasmer-node/             # node runtime daemon
    disasmer-dap/              # custom Debug Adapter Protocol server
    disasmer-vscode-protocol/  # extension <-> CLI/DAP shared types
    disasmer-transport/        # Quinn-backed QUIC wrapper and protocol framing
    disasmer-store/            # store traits + Postgres persistent-store impl
    disasmer-runtime-state/    # in-memory active process/thread/debug state
    disasmer-vfs/              # VFS manifests, overlays, blob/chunk references
    disasmer-env/              # env discovery, normalization, caching
    disasmer-source/           # source-provider traits and manifests
    disasmer-git/              # default optional Git source provider module
    disasmer-debug-model/      # breakpoints, epochs, stack/locals schema
    disasmer-ui-model/         # immediate-mode UI schema scaffold
    disasmer-auth/             # node enrollment, pubkey identities, pluggable auth hooks
    disasmer-protocol/         # coordinator/node/DAP wire messages
  private/
    hosted-oidc/               # Authentik/OIDC integration for the public service
    hosted-policy/             # community quotas, abuse controls, zero-cap hosted-loop rules
    hosted-admin/              # service admin tools, kill switches, internal dashboards
  vscode/
    extension/                 # TypeScript extension
  examples/
    launch-build/              # flagship Linux + Windows build demo
    local-one-node/            # minimal spawn/join example
    operator-panel-preview/    # second-milestone preview
  docs/
    MVP.md
    architecture.md
    protocol.md
    security.md
    debugging.md

The private folders are normal repository folders during development, but must be easy to exclude from the public source release. The public crates should expose policy/auth interfaces; the private modules implement hosted-service-specific OIDC, community quota enforcement, hosted zero-capability Wasm limits, abuse tooling, and admin affordances.

3.4 Process model

MVP process types:

disasmer coordinator
  Hosted or local control-plane service.

disasmer node
  Long-running node runtime attached to a coordinator.

disasmer dap
  Debug Adapter Protocol process launched by VS Code.

disasmer cli
  User/agent CLI for login, init, build, run, node attach, artifact export.

3.5 Primary user commands

# Authenticate a human against the hosted coordinator.
disasmer login https://disasmer.example.com

# Create or enroll a non-browser CLI identity for an agent.
disasmer auth keygen --name forgejo-worker

disasmer auth enroll-key   --coordinator https://disasmer.example.com   --project my-project   --public-key ~/.config/disasmer/keys/forgejo-worker.pub

# Initialize project metadata; optional because defaults should work.
disasmer init

# Show discovered environments, source providers, and snapshot policy.
disasmer inspect

# Build the virtual binary/bundle.
disasmer build

# Attach a node. OS, architecture, container backend, Git availability, and most
# capabilities are auto-detected; --cap remains available for explicit overrides.
disasmer node attach   --coordinator https://disasmer.example.com   --token dis_node_enroll_...

# Override or add an auto-detected capability only when needed.
disasmer node attach   --coordinator https://disasmer.example.com   --token dis_node_enroll_...   --cap env.windows-command-dev=true

# Run without VS Code. Hosted coordinator is the default when logged in;
# project defaults to the current directory. Entry point is optional.
disasmer run

disasmer run build-linux

disasmer run --project ../other-repo package-release

disasmer run --local smoke-test

# Export final artifacts by asking an attached/local receiver node to pull them.
# There is no default coordinator artifact store.
disasmer artifact export --process <id> --to ./dist

disasmer run should accept an optional entry point. This is a good idea because one Disasmer project may contain several useful virtual-process entrypoints: build, test, package, release, watch, operator, or project-specific workflows. The default remains the #[disasmer::main] entrypoint when no entry is supplied.

VS Code should hide most of this behind:

Disasmer: Login
Disasmer: Attach Local Node
Disasmer: Build Bundle
Disasmer: Run
Disasmer: Debug
Disasmer: Restart Failed Task
Disasmer: Export Artifacts

---

4. The flagship launch example

4.1 Project layout

launch-build-demo/
  Cargo.toml
  Cargo.lock
  src/
    main.rs
  envs/
    linux/
      Containerfile
    windows/
      Containerfile
  inputs/
    build-config.json
  disasmer.toml          # optional; only used for launch polish/overrides

4.2 Example disasmer.toml

The file should be optional, but the demo can include it to make platform constraints obvious.

[project]
name = "launch-build-demo"

[envs.linux]
path = "envs/linux/Containerfile"
kind = "container"
requires = { os = "linux", arch = "x86_64" }

[envs.windows]
path = "envs/windows/Containerfile"
kind = "container"
requires = { os = "windows", arch = "x86_64" }
backend_hint = "windows-command-dev"

[vfs]
include = [
  "src/**",
  "Cargo.toml",
  "Cargo.lock",
  "inputs/**"
]
exclude = [
  "target/**",
  ".git/**"
]

[git]
enabled = true
dirty_overlay = true
submodules = true

4.3 Example Rust build program

use disasmer::prelude::*;

#[derive(Serialize, Deserialize, Clone, Debug, DisasmerArg)]
enum Target {
    LinuxX64,
    WindowsX64,
}

impl Target {
    fn env(&self) -> Env {
        match self {
            Target::LinuxX64 => env!("linux"),
            Target::WindowsX64 => env!("windows"),
        }
    }

    fn name(&self) -> &'static str {
        match self {
            Target::LinuxX64 => "linux-x64",
            Target::WindowsX64 => "windows-x64",
        }
    }

    fn command(&self) -> CommandSpec {
        match self {
            Target::LinuxX64 => cmd!("cargo", "build", "--release"),
            Target::WindowsX64 => cmd!("cargo", "build", "--release"),
        }
    }
}

#[disasmer::main]
async fn main() -> Result<()> {
    // The hosted coordinator control loop is intentionally capless. Source access
    // happens on a user-attached node through the default Git source-provider
    // module, not inside the coordinator process.
    let src = spawn::task(prepare_source)
        .placement(placement::any_node().with_cap("source.git"))
        .queue_until_node_available()
        .start()
        .await?
        .join()
        .await?;

    let completed = DAtomicU32::new(0).await?;

    let linux = spawn::task(compile)
        .arg(BuildArgs::new(Target::LinuxX64, src.clone(), completed.clone()))
        .env(env!("linux"))
        .start()
        .await?;

    let windows = spawn::task(compile)
        .arg(BuildArgs::new(Target::WindowsX64, src.clone(), completed.clone()))
        .env(env!("windows"))
        .start()
        .await?;

    let linux_artifact = linux.join().await?;
    let windows_artifact = windows.join().await?;

    let dist = package(vec![linux_artifact, windows_artifact]).await?;

    assert_eq!(completed.load(Ordering::SeqCst).await?, 2);

    // Export is explicit code. The coordinator does not provide a default
    // artifact store; this asks an attached receiver/local node to pull bytes.
    artifact::export_to_node(dist, receiver::local("dist")).await?;
    Ok(())
}

#[disasmer::task]
async fn prepare_source() -> Result<SourceSnapshot> {
    source::git::current_repo().snapshot().await
}

#[derive(Serialize, Deserialize, Clone, Debug, DisasmerArg)]
struct BuildArgs {
    target: Target,
    src: SourceSnapshot,
    completed: DAtomicU32,
}

impl BuildArgs {
    fn new(target: Target, src: SourceSnapshot, completed: DAtomicU32) -> Self {
        Self { target, src, completed }
    }
}

#[disasmer::task]
async fn compile(args: BuildArgs) -> Result<Artifact> {
    fs::mount("/src", args.src).await?;

    args.target.command()
        .cwd("/src")
        .env("DISASMER_TARGET", args.target.name())
        .run()
        .await?;

    let output_path = match args.target {
        Target::LinuxX64 => "/src/target/release/app",
        Target::WindowsX64 => "/src/target/release/app.exe",
    };

    let artifact_path = format!("/vfs/artifacts/{}/{}", args.target.name(), basename(output_path));
    fs::copy(output_path, &artifact_path).await?;

    fs::flush().await?;
    args.completed.fetch_add(1, Ordering::SeqCst).await?;

    Artifact::from_path(artifact_path).await
}

#[disasmer::task]
async fn package(outputs: Vec<Artifact>) -> Result<Artifact> {
    fs::create_dir_all("/vfs/artifacts/dist").await?;

    for output in outputs {
        fs::copy(output.path(), "/vfs/artifacts/dist/").await?;
    }

    fs::flush().await?;
    Artifact::from_path("/vfs/artifacts/dist").await
}

4.4 What the demo proves

The demo proves:

• The build graph is source code, not YAML.
• Environment resources live in the repo.
• Source code references env!("linux") and env!("windows"), not runner labels.
• The coordinator schedules by logical environment and node capability.
• Linux and Windows tasks can run in the same virtual process.
• cmd!(...) runs on nodes, not on the hosted coordinator.
• Outputs become artifact handles through the VFS.
• flush() publishes artifacts for downstream tasks.
• Final artifact export is explicit code and does not imply coordinator artifact storage.
• VS Code sees one virtual process and multiple virtual threads.
• A Debug Epoch stops the whole virtual process.
• Restarting a failed task after editing source is part of the normal loop.

---

5. Rust SDK and task API

5.1 MVP API surface

The MVP Rust SDK should include:

#[disasmer::main]
#[disasmer::task]
env!("name")
spawn::task(task_fn)
TaskHandle<T>
Artifact
Blob
SourceSnapshot
VirtualFile
DAtomicU32
DAtomicU64
DChannel<T>
fs::mount()
fs::copy()
fs::flush()
fs::sync() / artifact::export_to_node()
cmd!(...)
source::git::current_repo().snapshot()
ui::* scaffold only

Do not include every distributed data structure in the first API. The launch demo needs small data, atomics/counters, task handles, source snapshots, artifacts, and VFS operations more than it needs a complete distributed memory library.

5.2 Task registration

#[disasmer::task] should generate:

• a stable task symbol;
• a task ABI descriptor;
• argument/result serialization metadata;
• debug metadata for task entry, source spans, and selected local/argument values;
• capability declarations inferred from APIs used where possible;
• task restart compatibility hash;
• a registration entry in the bundle manifest.

Example generated manifest entry:

{
  "task_id": "compile@sha256:...",
  "symbol": "compile",
  "source": "src/main.rs",
  "arg_type": "BuildArgs@sha256:...",
  "result_type": "Artifact@sha256:...",
  "requires": ["vfs", "cmd", "artifact.publish"],
  "debug": {
    "spans": "debug/compile.spans.json",
    "locals_schema": "debug/compile.locals.json"
  }
}

5.3 Argument rules

Task arguments and results must implement DisasmerArg.

Allowed:

u32, u64, bool, String
small structs/enums
SourceSnapshot
Artifact
Blob
Env
DAtomicU32 / DAtomicU64
DChannel<T>
VirtualFile
small Vec<T> with configured size limit

Rejected:

&T / &mut T
raw pointers
Rc<T>
RefCell<T>
std::fs::File
TcpStream
MutexGuard
native thread/process handles
unbounded Vec<u8>
closures with implicit captures

The compiler error should explain the distributed boundary. For example:

BuildArgs cannot be passed to a Disasmer virtual thread because it contains `&Path`.
Use String, SourceSnapshot, VirtualFile, Blob, or Artifact instead.

5.4 Size limits

Set a default task argument size limit such as:

soft warning: 256 KiB
hard default: 4 MiB
project override: allowed but noisy

The hosted/community coordinator control loop should use much tighter limits, for example a 4 KiB warning and 16 KiB hard limit. User-attached nodes can keep the normal node defaults unless project policy overrides them. A user accidentally passing an 82 MB Vec<u8> should see:

Task argument is 82 MB and would be copied to the spawned virtual thread.
Use Blob::from_file, SourceSnapshot, Artifact, or VirtualFile instead.

5.5 Host imports

The MVP guest ABI should be explicit and small.

Core imports:

disasmer.spawn.start
disasmer.spawn.join
disasmer.task.yield_safepoint
disasmer.task.current
disasmer.fs.mount
disasmer.fs.copy
disasmer.fs.flush
disasmer.fs.sync
disasmer.artifact.from_path
disasmer.artifact.open
disasmer.blob.open
disasmer.cmd.start
disasmer.cmd.poll
disasmer.cmd.kill
disasmer.atomic.new_u32
disasmer.atomic.fetch_add_u32
disasmer.atomic.load_u32
disasmer.debug.probe
disasmer.log.write
disasmer.ui.emit_snapshot    # scaffold only

5.6 Native command API

cmd!(...) should build a CommandSpec rather than immediately running a process.

cmd!("cargo", "build", "--release")
    .cwd("/src")
    .env("RUSTFLAGS", "...")
    .timeout(Duration::from_secs(600))
    .run()
    .await?;

The host checks:

• current virtual thread has cmd.run capability;
• selected environment supports native commands;
• the task is running on a node, not the hosted coordinator;
• process limits are configured;
• filesystem mounts are inside the VFS/environment policy;
• network policy is explicit;
• logs are capped and streamed with backpressure.

The hosted coordinator control plane never has this capability. An open-source/private coordinator may run with an embedded local node for trusted deployments; that node can have normal local command capabilities by default. Keep this as a compact policy/runtime module boundary rather than a separate node implementation, so dogfooding still uses the same node runtime code.

5.7 Compile-time environment references

env!("linux") should validate against generated environment metadata.

Implementation:

1. disasmer build scans envs/** and optional disasmer.toml.
2. It writes generated metadata to target/disasmer/generated/envs.rs and a JSON manifest.
3. The macro consults generated metadata during build.
4. VS Code reads the same metadata for autocomplete and diagnostics.

Missing env error:

env!("windows") does not exist.
Discovered environments: linux.
Create envs/windows/Containerfile or add [envs.windows] to disasmer.toml.

---

6. Bundle and build pipeline

6.1 Bundle concept

A Disasmer bundle is a content-addressed virtual binary.

It includes:

Wasm module(s)
Disasmer manifest
registered tasks
debug metadata
environment definitions
selected environment context files
source snapshot metadata
selected VFS seed files
capability declarations
content hashes

It should not embed full container images.

6.2 Bundle layout

app.dis/
  manifest/disasmer.json
  modules/main.wasm
  debug/main.dwarf
  debug/disasmer-debug.json
  envs/linux/Containerfile
  envs/windows/Containerfile
  envs/manifest.json
  source/source-snapshot.json
  source/dirty-overlay-manifest.json
  inputs/build-config.json
  vfs/seed-manifest.json
  capabilities.json

The on-disk .dis can initially be a directory during development and later become a zip/tar-like archive with content-addressed chunks.

6.3 Build command pipeline

disasmer build
  1. discover project root
  2. load disasmer.toml if present
  3. discover envs/**/Containerfile and envs/**/Dockerfile
  4. discover Nix flakes if plugin enabled
  5. resolve source snapshot through enabled source-provider module, Git by default
  6. compute dirty overlay manifest
  7. generate env metadata for env!()
  8. compile Rust guest to Wasm
  9. run macro/task registration collection
  10. collect debug metadata
  11. build VFS seed manifest
  12. compute bundle digest
  13. write app.dis

6.4 Digest strategy

Bundle digest should include:

Wasm bytes
task ABI descriptors
environment recipe files
environment context file hashes
source snapshot provider identity and revision, such as a Git commit id
dirty overlay file hashes
selected input file hashes
capability declarations
debug metadata hash
SDK ABI version

It should not include:

node-local cache paths
absolute host checkout path
local build timestamps
unselected .git internals
unselected target directory contents

6.5 Compatibility hashes

Task restart requires compatibility checks. Generate hashes for:

task public signature
argument schema
result schema
capability declaration
environment reference set
VFS mount contract
Disasmer ABI version

If only implementation changed and public schema is compatible, allow selected task restart. If schema changed, restart the whole virtual process.

---

7. Node runtime

7.1 Node responsibilities

The node runtime is where real work happens.

It must support:

registration with coordinator
capability reporting
heartbeat and health reporting
bundle fetch or local bundle lookup
Wasmtime task instantiation
native command execution
environment materialization/caching
VFS overlay storage
ephemeral artifact/blob cache
source snapshot providers
debug epoch handling
logs/events streaming
quota enforcement
Quinn peer links

7.2 Node state directories

Linux default:

~/.local/share/disasmer/node/
  config.toml
  identity/
  bundles/
  env-cache/
  vfs/
  blobs/
  artifacts/
  logs/
  tmp/

Windows default:

%LOCALAPPDATA%\Disasmer\node\
  config.toml
  identity\
  bundles\
  env-cache\
  vfs\
  blobs\
  artifacts\
  logs\
  tmp\

7.3 Node capability model

Node capabilities should be explicit and signed by node identity plus coordinator policy.

Example:

{
  "node_id": "node_123",
  "tenant_id": "tenant_abc",
  "os": "linux",
  "arch": "x86_64",
  "capabilities": {
    "wasm": true,
    "cmd": true,
    "container.podman.rootless": true,
    "nix": true,
    "git.source_provider": true,
    "debug.suspend_process_tree": true
  },
  "limits": {
    "max_tasks": 4,
    "max_log_bytes_per_task": 10485760,
    "max_arg_bytes": 4194304
  }
}

Windows dev backend example:

{
  "node_id": "node_win_123",
  "tenant_id": "tenant_abc",
  "os": "windows",
  "arch": "x86_64",
  "capabilities": {
    "wasm": true,
    "cmd": true,
    "windows.command_dev": true,
    "windows.sandbox_stub": true,
    "debug.job_object_suspend": true
  },
  "security_note": "not a managed-untrusted sandbox backend"
}

7.4 Wasmtime task runner

Each running virtual thread/task maps to:

Wasmtime Engine
compiled module cache
Store<ThreadHostState>
Linker with Disasmer imports
task entrypoint dispatcher
fuel/epoch/resource limits
debug probe table
VFS overlay handle
handle table

ThreadHostState should contain:

struct ThreadHostState {
    tenant_id: TenantId,
    project_id: ProjectId,
    virtual_process_id: VirtualProcessId,
    virtual_thread_id: VirtualThreadId,
    task_id: TaskId,
    bundle_digest: Digest,
    debug_epoch: Option<DebugEpochId>,
    vfs_overlay: VfsOverlayHandle,
    handles: HandleTable,
    limits: ResourceLimits,
    coordinator: CoordinatorClient,
    node_store: NodeStore,
    command_runner: Arc<dyn CommandRunner>,
}

7.5 Command runner

The command runner must abstract platform differences.

trait CommandRunner {
    async fn start(&self, spec: CommandSpec, ctx: CommandContext) -> Result<CommandHandle>;
    async fn poll(&self, handle: CommandHandle) -> Result<CommandStatus>;
    async fn suspend_for_debug(&self, handle: CommandHandle, epoch: DebugEpochId) -> Result<()>;
    async fn resume_from_debug(&self, handle: CommandHandle, epoch: DebugEpochId) -> Result<()>;
    async fn terminate(&self, handle: CommandHandle) -> Result<()>;
}

Linux implementations:

PodmanRootlessCommandRunner
LocalProcessGroupCommandRunner for development

Windows implementations:

WindowsCommandDevRunner
WindowsSandboxRunnerStub

Do not offer WindowsCommandDevRunner as secure managed compute for untrusted public workloads.

7.6 Linux Podman backend

Linux MVP should use rootless Podman.

Responsibilities:

build image from Containerfile/Dockerfile
cache image by environment digest
mount /src, /work, /cache, /out or VFS paths
run command with resource limits where practical
capture stdout/stderr
pause/unpause or suspend process tree for debug epoch
publish outputs into VFS overlay

Default mounts:

/src    source snapshot or bind mount
/work   task-local writable overlay
/cache  node-local dependency cache keyed by env/project
/out    artifact staging path

7.7 Windows backend

For the public-launch MVP, Windows should support an attached node that can run build commands in a controlled workspace.

Minimum Windows backend:

create per-task temp workspace
materialize source snapshot into workspace or bind local checkout when safe
set environment variables from normalized env
run command as non-admin service user where possible
capture stdout/stderr
track process tree with Job Object-like lifecycle management
stage artifacts into VFS overlay
support best-effort debug suspend/resume

Windows sandboxing should be behind an interface but not claimed as production isolation until implemented and tested.

Potential future backend options:

Windows Sandbox CLI backend
AppContainer / Less-Privileged AppContainer backend
Hyper-V isolated runner backend
Windows container backend, if project needs it

7.8 Logs

Logs are both a UX feature and an abuse vector.

Node should:

• stream recent log chunks to coordinator;
• keep larger logs node-local unless sync() or retention policy requires upload;
• redact configured secrets before sending;
• enforce per-task and per-process log byte caps;
• include task id, virtual thread id, sequence number, timestamp, and stream name.

Log event:

{
  "tenant_id": "tenant_abc",
  "process_id": "vp_123",
  "thread_id": "vt_456",
  "task_id": "task_789",
  "stream": "stderr",
  "seq": 42,
  "bytes": "...",
  "truncated": false
}

---

8. Coordinator

8.1 Coordinator responsibilities

The coordinator should provide:

auth/session handling; hosted OIDC through private module
project registry
node enrollment and identity
node heartbeats
node capability registry
virtual process lifecycle
task placement
bundle registry
debug session registry
Debug Epoch coordination
VFS manifest metadata for active processes
artifact metadata for active processes
hosted/community policy hooks and optional private quota ledger
operator panel state/events scaffold
rendezvous for node-to-node QUIC
web dashboard
CLI API
VS Code/DAP API

It should not run arbitrary user commands.

8.2 Coordinator state split

The coordinator needs two state layers in the MVP.

Persistent Postgres state:

users / tenants / memberships if hosted module is enabled
projects
node enrollment records
node public keys / credentials / revocations
project-scoped source-provider configuration
bundle registry entries worth reusing
persistent environment metadata/cache hints
hosted/community plan or policy records in the private module
longer-lived audit records where needed

In-memory runtime state:

active virtual processes
virtual threads
pending task queues
live node sessions and heartbeats
scheduler placements
Debug Epochs
recent logs/events
VFS manifest metadata for active processes
artifact metadata and locations for active processes
operator panel snapshots/events for active processes

For the MVP, if the coordinator restarts, active virtual processes restart or fail cleanly. Do not persist every process/thread/log/VFS row merely to survive coordinator restarts. Persistence is for product identity and project continuity, not durable runtime recovery.

8.3 Store interfaces and indicative persistent tables

Define a persistent store trait for things that must survive restart:

trait CoordinatorPersistentStore {
    async fn create_tenant(...);
    async fn create_project(...);
    async fn register_node_identity(...);
    async fn revoke_node_credential(...);
    async fn create_bundle(...);
    async fn save_environment_metadata(...);
    async fn save_source_provider_config(...);
    async fn authorize_persistent_object(...);
}

Define a separate runtime-state trait that is in-memory for the MVP:

trait CoordinatorRuntimeState {
    async fn create_virtual_process(...);
    async fn create_virtual_thread(...);
    async fn update_thread_state(...);
    async fn create_debug_epoch(...);
    async fn publish_vfs_manifest(...);
    async fn publish_artifact_metadata(...);
    async fn append_recent_log_event(...);
    async fn reserve_runtime_quota(...);
}

Indicative persistent Postgres tables:

users(
  id,
  authentik_subject nullable,
  email nullable,
  created_at,
  suspended_at
)

tenants(
  id,
  slug,
  plan,
  created_at
)

tenant_members(
  tenant_id,
  user_id,
  role
)

projects(
  id,
  tenant_id,
  name,
  visibility,
  created_at
)

nodes(
  id,
  tenant_id,
  project_id nullable,
  display_name,
  os,
  arch,
  trust_label,
  created_at,
  revoked_at
)

node_capabilities_last_seen(
  node_id,
  capability,
  value,
  reported_at
)

node_credentials(
  node_id,
  public_key,
  cert_fingerprint nullable,
  expires_at nullable,
  revoked_at
)

bundles(
  id,
  tenant_id,
  project_id,
  digest,
  manifest_json,
  created_at
)

environments(
  id,
  tenant_id,
  project_id,
  bundle_id nullable,
  name,
  kind,
  digest,
  requirements_json,
  recipe_manifest_json,
  created_at
)

source_provider_configs(
  id,
  tenant_id,
  project_id,
  provider,
  config_json,
  created_at
)

audit_events(
  id,
  tenant_id,
  project_id nullable,
  actor_id nullable,
  event_kind,
  object_id nullable,
  event_json,
  created_at
)

Runtime-only structures may look like the old tables conceptually, but they should be ordinary in-memory maps/queues first: virtual_processes, virtual_threads, debug_epochs, active vfs_manifests, artifact locations, recent logs, and operator panel snapshots.

8.4 Tenant safety invariant

Every row that belongs to user content should include tenant_id. Every API should authorize by tenant and object relationship.

Required tests:

user A cannot list user B nodes
user A cannot attach to user B process
user A cannot read user B logs
user A cannot read user B artifact metadata for active processes
user A cannot request debug memory from user B process
node A cannot claim tenant B process
node A cannot publish artifact for tenant B

8.5 Node enrollment

Flow:

1. User opens coordinator UI or CLI.
2. User creates a node enrollment token for a project or tenant.
3. Token has short TTL and may include allowed capability constraints; node OS, architecture, and most capabilities are auto-detected at attach time.
4. Node CLI starts with token.
5. Node generates keypair.
6. Node calls enrollment endpoint.
7. Coordinator validates token and policy.
8. Coordinator creates node identity/certificate.
9. Node stores credential locally.
10. Node opens persistent session to coordinator.

Enrollment token example:

{
  "tenant_id": "tenant_abc",
  "project_id": "proj_123",
  "expires_at": "...",
  "allowed_capabilities": [
    "wasm",
    "cmd",
    "container.podman.rootless"
  ],
  "auto_detect": ["os", "arch", "env", "source_providers"],
  "overrides_allowed": true
}

8.6 Scheduler

The MVP scheduler should be simple, deterministic, and locality-aware. It should also be an easily replacible module in case the user wants different scheduling, or perhaps they have limitless resources (e.g. cloud backed)

Inputs:

task environment requirements
node capabilities
node health
source snapshot locations
environment cache locations
artifact locations
node current load
tenant/project policy
quota availability
debug affinity / restart affinity

Scoring:

score(node) =
  + environment compatible
  + source snapshot already local
  + environment digest cached
  + dependency cache likely warm
  + needed artifacts local
  + same node as previous failed attempt when restart is desired
  - current load
  - expected bytes to transfer
  - recent failures
  - policy warnings

Placement should prefer the local node if it can run the task and already has the repo.

8.7 State machine

Virtual process states:

CREATED
STARTING
RUNNING
FREEZE_REQUESTED
FROZEN
RESUMING
COMPLETED
FAILED
CANCELLED

Virtual thread states:

PENDING
PLACED
STARTING
RUNNING
WAITING
COMMAND_RUNNING
FREEZE_REQUESTED
FROZEN
COMPLETED
FAILED
CANCELLED
RESTARTING

Debug epoch states:

REQUESTED
FREEZING
FROZEN
RESUMING
RESUMED
FAILED

---

9. Transport and networking

9.1 Transport choices

Use Quinn for QUIC in the MVP, behind a Rust abstraction.

trait Transport {
    async fn connect_node(&self, endpoint: Endpoint, identity: NodeIdentity) -> Result<Connection>;
    async fn accept_node(&self) -> Result<Connection>;
    async fn open_stream(&self, kind: StreamKind) -> Result<Stream>;
    async fn datagram(&self, msg: Datagram) -> Result<()>;
}

The abstraction is important because Rust bindings and platform packaging may be rough. Keep QUIC-specific details in disasmer-transport.

9.2 Coordinator sessions

The coordinator session carries:

node heartbeat
capability updates
task start/stop messages
debug epoch messages
VFS manifest publication
artifact metadata for active processes publication
log event stream
rendezvous endpoint exchange
quota updates

For development, it is acceptable to bootstrap auth over HTTPS and then use QUIC for persistent sessions.

9.3 Node-to-node sessions

Node-to-node sessions carry:

blob chunk requests
artifact reads
source snapshot delta transfer
VFS chunk transfer
debug lazy memory/page reads where appropriate

They should not carry unauthenticated traffic. Every stream must include:

tenant id
project id
process id when applicable
authorized object id
fencing token / epoch where applicable
request id
quota accounting key

9.4 Rendezvous without relay

Coordinator-aided connection:

1. Node A needs artifact from Node B.
2. A asks coordinator for artifact locations.
3. Coordinator authorizes A and returns candidate endpoints for B.
4. Coordinator notifies B to expect A.
5. A and B attempt direct QUIC connection.
6. If connection succeeds, data moves node-to-node.
7. If connection fails, scheduler avoids this transfer or asks user to sync/export through configured durable storage.

No silent bulk relay through the coordinator.

9.5 Failure handling

If direct node-to-node connection fails:

• do not fall back to coordinator bulk transfer by default;
• mark the object location as unreachable from the requesting node;
• choose a different placement if possible;
• if impossible, fail with a clear action:

Artifact is on node windows-1, but linux-1 cannot establish direct QUIC.
Run an artifact receiver node, configure an explicit external storage/export task, or run the dependent task on windows-1.

---

10. Environment system

10.1 Discovery

Scan default paths:

envs/<name>/Containerfile
envs/<name>/Dockerfile
envs/<name>/flake.nix
envs/<name>/flake.lock

Normalize to:

{
  "name": "linux",
  "kind": "container",
  "digest": "sha256:...",
  "recipe": "envs/linux/Containerfile",
  "context": "envs/linux",
  "requires": {
    "os": "linux",
    "arch": "x86_64"
  },
  "capabilities": {
    "cmd": true,
    "network": false
  }
}

10.2 Containerfile MVP

Containerfile is the MVP path.

Linux:

Containerfile -> Podman build -> local image id -> run task command

Windows:

Containerfile discovered and hashed
Windows backend may treat it as a declared environment contract/stub
actual command execution uses WindowsCommandDevRunner for attached user nodes

This is intentionally not marketed as secure Windows container isolation.

10.3 Nix plugin

Nix should be present as a plugin because the architecture already expects pluggable environment resources, and Nix is useful for development reproducibility.

MVP Nix support can be:

discovery of flake.nix and flake.lock
hashing into environment digest
optional local materialization on nodes with Nix installed
cache location reporting
not required for the flagship launch demo

10.4 Environment materialization cache

Cache key:

env digest + backend kind + platform + Disasmer env ABI version

Cache record:

{
  "env_digest": "sha256:...",
  "node_id": "node_123",
  "backend": "podman-rootless",
  "materialized_ref": "localhost/disasmer/env@sha256:...",
  "status": "ready",
  "last_used_at": "..."
}

10.5 Environment security policy

Default task environment policy:

network disabled unless explicitly granted
host filesystem inaccessible except declared VFS mounts
secrets absent unless explicitly granted
no privileged containers
no Docker/Podman socket mount
read-only source mount when possible
write access only to /work and artifact staging
logs capped
process lifetime capped by policy

For user-attached nodes, Disasmer should still enforce its policy, but documentation must be honest that the node owner controls the machine and can bypass local restrictions.

---

11. VFS, artifacts, and latency

11.1 Core rule

Move metadata eagerly.
Move bytes lazily.
Run work where the bytes already are.

This should be treated as an engineering invariant, not a performance optimization later.

11.2 Mount model

Default virtual filesystem:

/src or /vfs/project
  read-only source/input snapshot

/work or /vfs/work
  task-local writable overlay

/vfs/artifacts
  published artifact namespace

/vfs/shared
  optional explicit shared namespace, probably not launch-critical

11.3 Source snapshots and source-provider modules

Source support should be pluggable. Git is the default included source-provider module, not a hard dependency of the VFS core.

A source-provider module must be able to produce a SourceSnapshot manifest such as:

provider name and version
source identity
base revision or content root
sub-resource state if applicable
changed-file / dirty overlay manifest
included untracked or generated inputs
ignored-file policy

For Git specifically, the default module should model a source snapshot as:

repo identity
commit hash
submodule state
dirty overlay manifest
included untracked files
ignored-file policy

Local node behavior:

if task runs on node with checkout:
  bind mount checkout read-only where safe
  materialize dirty overlay if needed
else:
  fetch missing content through the enabled source provider or by content hash

Do not tar and upload the entire repo by default. Do not require users who use another source system to carry Git dependencies in their runtime.

11.4 Dirty overlay

Dirty overlay manifest:

{
  "base_commit": "abc123",
  "files": [
    {
      "path": "src/main.rs",
      "mode": "100644",
      "digest": "sha256:...",
      "size": 12345
    }
  ],
  "deleted": ["old/file.rs"]
}

Dirty bytes move only if the chosen node does not already have them.

11.5 VFS overlay commits

Each task has a writable overlay. flush() creates a manifest delta:

{
  "process_id": "vp_123",
  "thread_id": "vt_456",
  "parent_epoch": 4,
  "epoch": 5,
  "created": [
    {
      "path": "/vfs/artifacts/linux-x64/app",
      "digest": "sha256:...",
      "size": 9912321,
      "locations": ["node_linux_1"]
    }
  ],
  "modified": [],
  "deleted": []
}

The coordinator stores the manifest and locations. It does not receive the bytes unless a configured durability/export policy requires it.

11.6 flush() semantics

flush() means:

close current task write epoch
hash changed files/chunks
publish VFS manifest delta
publish artifact metadata for /vfs/artifacts changes
make changes visible to downstream tasks according to VFS consistency rules
return after metadata is accepted by coordinator

flush() should not:

upload large artifacts by default
replicate to other nodes by default
export to local dist by default
route bytes through coordinator

11.7 sync() and explicit artifact movement semantics

sync() means:

apply an explicit export/transfer/durability policy chosen by the program or CLI
verify referenced bytes still exist on at least one node or explicit store
move bytes directly between nodes or to a user-configured external destination
update artifact metadata for active processes with the result

sync() should not imply a default coordinator artifact store. By default, artifacts and other large data are nearly ephemeral: they live on nodes for as long as a task, downstream consumer, export, restart, or configured cache/GC policy needs them. They may linger long enough for practical task restart, but unsynced node-local artifacts are not a durability guarantee.

Example policies:

// Ask an attached/local receiver node to pull bytes into ./dist.
artifact::export_to_node(dist, receiver::local("dist")).await?;

// Send bytes to a user-provided storage integration implemented by project code.
artifact::copy_to(outputs, storage::s3("release-bucket")).await?;

// Keep artifacts available on the producing node long enough for likely restarts.
artifact::retain(outputs, RetainPolicy::BestEffort { min_ttl: Duration::from_hours(24) }).await?;

MVP should implement at least:

metadata-only flush
node-local ephemeral artifact/blob cache
explicit export to an attached/local receiver node
best-effort retention/GC knobs
clear errors when bytes have been garbage-collected or are unreachable

Users who want artifacts on their own machine should either run a receiver node there or write/use an explicit storage/export integration. The coordinator should keep artifact metadata for active processes, not act as the default artifact data store.

11.8 Artifact lifecycle

Artifact states:

PUBLISHED_METADATA
AVAILABLE_ON_NODE
RETAINED_BEST_EFFORT
EXPORTED
DURABLE_EXTERNAL
EXPIRED
DELETED

To allow producing artifacts without much effort from the user, the control pane could provide a download button through the immediate-mode UI, which will stream the (best-effort retained) artefact from the node that retains it. "Best effort" and usage tier limits apply as normal, but it should be an error to create a download link to an artifact that can't be downloaded within the limits in the first place. Download links should be secure and not just allow anyone that guesses the correct link to download any artifact.

Artifact metadata:

{
  "artifact_id": "art_123",
  "path": "/vfs/artifacts/linux-x64/app",
  "digest": "sha256:...",
  "size": 9912321,
  "producer_thread_id": "vt_linux",
  "locations": [
    { "kind": "node", "node_id": "node_linux_1" }
  ],
  "durability_state": "PUBLISHED_METADATA",
  "retention": "best_effort_node_gc"
}

11.9 Local-first acceptance test

For a local Linux build from an existing Git checkout:

• no full-repo tarball is created;
• no source bytes are uploaded to coordinator;
• compiler file reads are local;
• build cache reads are local;
• artifact bytes remain local after flush();
• coordinator receives only metadata, logs within caps, and debug/session events;
• explicit export/transfer code is the first operation allowed to move final artifact bytes for durability or user-visible export.

---

12. Task execution lifecycle

12.1 Spawn flow

1. Guest calls spawn::task(compile).arg(...).env(env!("linux")).start().await.
2. Wasm guest invokes disasmer.spawn.start host import.
3. Node host validates current task capability.
4. Host serializes TaskSpec with task id, env id, args, handles, VFS base epoch.
5. Coordinator creates virtual_thread row in PENDING state.
6. Scheduler chooses node.
7. Coordinator sends StartTask to chosen node.
8. Node fetches or locates bundle.
9. Node materializes environment if needed.
10. Node creates VFS overlay.
11. Node instantiates Wasmtime module or command wrapper.
12. Node calls named task entrypoint.
13. Thread becomes visible in debugger and coordinator UI.

12.2 Join flow

1. Parent awaits TaskHandle.join().
2. Parent task parks or yields.
3. Child completes with serialized result or error.
4. Result may contain handles such as Artifact.
5. Coordinator records completion.
6. Parent is woken.
7. Parent receives decoded result.

12.3 Command flow inside task

1. Guest task calls cmd!(...).run().await.
2. Host import validates command capability.
3. Node command runner starts process in task environment.
4. Task enters COMMAND_RUNNING state.
5. Logs stream to node and coordinator under caps.
6. Debug Epoch requests suspend controlled process tree.
7. Command exit status is returned to guest.
8. Guest decides whether to publish artifacts.

12.4 Failure behavior

MVP rules:

Task trap -> task FAILED, process may enter debug break depending policy.
Command nonzero exit -> task receives error, can handle or fail.
Node lost with non-replicated running state -> virtual process FAILED.
Node lost after flushed but unsynced artifact -> artifact may be unavailable.
Node lost after synced artifact -> artifact survives according to sync policy.

Do not pretend non-replicated state is recoverable.

12.5 Restart flow

1. User edits source.
2. VS Code triggers disasmer build.
3. New bundle is produced.
4. User selects Restart Failed Task or Restart Selected Virtual Thread.
5. DAP/coordinator checks task compatibility hash.
6. Runtime stops old task if needed.
7. Unflushed overlay changes are discarded unless explicitly preserved.
8. New task starts with original args, same env, chosen VFS checkpoint.
9. Breakpoints are rebound.
10. Other tasks remain alive only if compatibility checks pass.

If compatibility fails:

Task signature changed; restarting whole virtual process is required.

This capability is important to advertise in any demo(s).

---

13. Debugging

13.1 Design goal

The user should experience Disasmer as one ordinary debug target:

Process: launch-build-demo
Threads:
  1 main
  2 compile linux-x64
  3 compile windows-x64
  4 package

The debugger should not force the user to understand node IDs, QUIC links, VFS manifests, or environment cache internals unless they open the Disasmer inspector.

13.2 Use a custom DAP adapter

Use disasmer-dap as the VS Code debug adapter.

Why:

• DAP is the right integration point for VS Code.
• The target is not one native process; it is a virtual process spanning nodes.
• Disasmer must control the thread list, stop reasons, restart commands, artifact views, and all-stop semantics.
• LLDB can be a future internal helper, but it should not own the UX.

13.3 Debug components

VS Code extension
  launches disasmer-dap and contributes commands/views

disasmer-dap
  speaks DAP to VS Code
  speaks Disasmer debug protocol to coordinator
  maps DAP process/thread/stack/variables to virtual model

coordinator debug service
  owns debug sessions and Debug Epochs
  broadcasts freeze/resume
  aggregates thread state

node debug agent
  sets breakpoints/probes
  freezes/resumes Wasmtime tasks
  suspends/resumes controlled command processes
  captures stacks/locals/handle state

13.4 Debug Epoch protocol

When a breakpoint is hit:

1. Node detects breakpoint/trap/probe in virtual thread T.
2. Node reports BreakHit(process, thread, location) to coordinator.
3. Coordinator creates DebugEpoch E.
4. Coordinator marks process FREEZE_REQUESTED.
5. Coordinator sends Freeze(E) to all nodes hosting threads for the process.
6. Wasm tasks stop at current probe/safepoint or next safepoint.
7. Native command tasks are suspended using backend-specific process-tree control.
8. VFS overlay manifests are pinned.
9. Memory/handle snapshots are pinned copy-on-write where applicable.
10. Nodes report Frozen(E, thread states).
11. Coordinator marks epoch FROZEN.
12. DAP emits stopped event for all threads.

Resume:

1. User presses Continue.
2. DAP sends Continue(E) to coordinator.
3. Coordinator sends Resume(E) to nodes.
4. Nodes unpin debug snapshots.
5. Wasm tasks continue.
6. Native command tasks resume.
7. Coordinator marks epoch RESUMED.

13.5 Safepoints and breakpoints

MVP should use layered breakpoint support:

Layer 1: explicit debug probes

#[disasmer::task] and #[disasmer::main] generate probes at:

task entry
before/after spawn
before/after join
before/after cmd run
before/after flush/sync
await boundaries
explicit disasmer::debug::probe!() calls

This provides reliable all-stop behavior early.

Layer 2: source-line breakpoints

The build pipeline maps source line breakpoints to probe points and, where possible, Wasm code offsets using debug metadata.

Launch acceptance should include:

breakpoint in main()
breakpoint in compile()
breakpoint before/after cmd!().run()
breakpoint before artifact publish/flush
breakpoint in package()

Layer 3: richer DWARF/LLDB integration

After the DAP skeleton works, improve locals/stacks by reading Rust/Wasm debug metadata. This is important, but it should not block the all-stop architecture.

13.6 Variables and locals

Launchable debugger variable support should include:

task arguments
return values
Target enums and simple structs
Artifact handles
SourceSnapshot handles
Blob handles
DAtomic values
DChannel status
current VFS mounts
current command spec/status
recent stdout/stderr tail

For arbitrary Rust locals, use best-effort DWARF support where available. If a local cannot be inspected, the debugger should say so clearly and still show task args/handles.

13.7 Native command tasks in debugger

A native command is debugger-visible as part of the virtual thread, but MVP does not need to debug inside cargo or the compiler itself.

The command frame should show:

command argv
working directory
environment name
node id in inspector only
start time
status
exit code when done
recent stdout/stderr
resource limits

When the virtual process is frozen, controlled native command processes should be suspended. If the backend cannot safely suspend a command, the node must report failure and the debugger must show a clear error rather than claiming all-stop succeeded.

13.8 DAP features required for launch

Required:

initialize
launch / attach
setBreakpoints
configurationDone
threads
stackTrace
scopes
variables
continue
pause
next / step-over at probe granularity
disconnect
stopped events
continued events
output events
custom restart-task request

Not required for launch:

watchpoints
reverse debugging
non-stop debugging
hot code replace
debugging arbitrary child process internals

13.9 Disasmer inspector

Default debugger view remains normal. Add an optional inspector tree:

Virtual Process
  Threads
    compile linux-x64 -> node linux-1 -> env linux -> running/frozen
    compile windows-x64 -> node windows-1 -> env windows -> running/frozen
  VFS
    epoch 4
    artifacts
  Debug Epoch
    id E
    frozen nodes
  Environments
    linux digest cached on linux-1
    windows digest cached/stub on windows-1
  Transport
    node links status

This helps explain the system without polluting the normal debug UX.

---

14. VS Code extension

14.1 Extension responsibilities

The extension should:

watch envs/**/Containerfile
watch envs/**/Dockerfile
watch envs/**/flake.nix
watch envs/**/flake.lock
watch inputs/**
watch src/**
run disasmer build before debugging
launch/attach DAP session
show virtual threads in normal debug UI
show per-task logs
show artifact tree
show node connection status
provide env!("...") autocomplete
provide disasmer.toml diagnostics
provide restart selected/failed task commands

14.2 Launch configuration

Example .vscode/launch.json:

{
  "version": "0.2.0",
  "configurations": [
    {
      "name": "Debug Disasmer Build",
      "type": "disasmer",
      "request": "launch",
      "coordinator": "https://disasmer.example.com",
      "project": "${workspaceFolder}",
      "preLaunchTask": "disasmer: build",
      "localFirst": true
    }
  ]
}

14.3 Editor diagnostics

Diagnostics:

env!("name") missing
env discovered but incompatible with available nodes
Containerfile changed; bundle rebuild needed
large task argument warning
cmd! used in task without cmd capability
fs::sync missing for final artifact export warning
Windows backend is dev/stub isolation warning

14.4 VS Code views

MVP views:

Disasmer: Nodes
Disasmer: Virtual Processes
Disasmer: Artifacts
Disasmer: Logs
Disasmer: Inspector

Operator panel preview can appear as:

Disasmer: Operator Panel Preview

But the full immediate-mode panel is second milestone.

---

15. Hosted coordinator and product UX

15.1 Hosted coordinator MVP

Hosted coordinator pages:

Sign in
Projects
Project detail
Node attach instructions
Node list
Virtual process list
Virtual process detail
Logs
Artifacts
Debug session status
Operator panel placeholder/preview
Settings / quotas

15.2 Hosted Authentik/OIDC integration

Use Authentik as the identity provider through OIDC in the private hosted module. The open-source/private coordinator should expose auth hooks and simpler local/team auth, but should not require the hosted OIDC stack.

Implementation requirements:

authorization code + PKCE for web login
strict redirect URI allowlist
minimal scopes
short session lifetime
refresh handling if needed
logout/back-channel support later
CLI token issuance after web login
node enrollment tokens separate from user sessions

15.3 Community tier

Community tier defaults:

1 active virtual process
tiny hosted Wasm control loop limit
no hosted native commands
no hosted containers
no hosted network from tasks
small logs
small metadata
small artifact retention
4 user-attached nodes allowed
operator panel access

The hosted control loop is useful for trying Disasmer and coordinating attached nodes, not for mining, scraping, compiling huge repos, or hosting files.

15.4 Abuse controls as architecture

Every resource-consuming action must pass through a hosted/community quota policy module before work starts.

Quota dimensions:

active virtual processes
hosted Wasm fuel
hosted Wasm memory
hosted process wall-clock
API calls
node heartbeats
spawn attempts
log bytes
metadata bytes
artifact bytes
artifact downloads
debug memory reads
operator panel updates
rendezvous attempts

Pattern:

quota.reserve(tenant, Resource::SpawnAttempt, 1).await?;
// perform operation
quota.commit(reservation).await?;

For streams, use incremental accounting and backpressure.

15.5 Public input threat model

Treat as hostile:

users
OAuth identities
nodes
programs
build files
Containerfiles
Git repos
logs
artifacts
operator panel text/events
debug requests
node capabilities
cache claims

15.6 Public launch security baseline

Required before public launch:

• Tenant ID on every object.
• Authorization check on every API endpoint.
• Negative cross-tenant tests.
• No hosted containers on community tier.
• No hosted native commands on community tier.
• No arbitrary network access from hosted Wasm.
• Rate limits on auth, APIs, rendezvous, logs, spawn, and UI events.
• Log and artifact size caps.
• Private artifacts by default.
• No public file hosting by default.
• Node identity signing/revocation.
• Task capabilities explicit.
• Debug attach requires project permission.
• Debug memory reads quota-limited and audited.
• Operator panel uses built-in widgets only.
• No custom HTML or JS in panels.
• No password fields in user-defined panels.
• No OAuth flows inside operator panels.

---

16. Operator panel scaffold and second milestone

16.1 MVP scaffold

Even if the full panel is second milestone, the MVP should include:

ui model crate
panel snapshot table
panel event table
coordinator API namespace
web placeholder area
DAP/debug interaction policy
security restrictions

The first visible version can be a fixed coordinator dashboard showing:

process state
tasks
progress
logs
artifacts
Debug / Restart / Cancel buttons as control-plane actions

16.2 Immediate-mode second milestone

The second milestone adds:

let panel = ui::panel("Repo builder").await?;

loop {
    let action = panel.draw(|ui| {
        ui.heading("Repo builder");
        ui.text("Build status");
        ui.task_table("Tasks", tasks.snapshot());
        ui.log_tail("Recent output", logs.tail());
        if ui.button("Restart failed").clicked() {
            ui.emit(Action::RestartFailed);
        }
    }).await?;

    if let Some(action) = action {
        handle(action).await?;
    }
}

16.3 Panel security model

MVP/second milestone rules:

built-in widgets only
no custom HTML
no custom JavaScript
all text escaped
external links restricted
no OAuth/password widgets
panel labeled as user-program-provided
rate-limited updates
state size capped
events typed and schema-checked
read-only while debugger has process frozen
control-plane actions may remain available while frozen

---

17. Security and capability model

17.1 Capability-first execution

Every task has a capability set.

Example:

{
  "task": "compile",
  "capabilities": {
    "vfs.read": ["/src"],
    "vfs.write": ["/work", "/vfs/artifacts/linux-x64"],
    "cmd.run": true,
    "network.outbound": false,
    "secrets": []
  }
}

17.2 Coordinator authz checks

Every coordinator API checks:

authenticated principal
principal tenant membership
object tenant id
project membership/role
operation permission
quota availability
object state compatibility

17.3 Node trust model

Default rule:

A user's work runs only on that user's nodes, managed paid nodes, or explicitly shared team nodes.

For MVP:

• do not mix tenants on user-provided nodes;
• do not trust user-provided nodes to produce truthful artifacts for other tenants;
• label artifacts with node provenance;
• expose trust label in inspector;
• eventually support reproducible repeated builds for higher trust.

17.4 Bundle identity

Bundles should be content-addressed and integrity-checked.

Later signing can be added, but MVP should at least ensure:

coordinator records bundle digest
node verifies bundle digest before running
manifest records environment/resource digests
task specs reference bundle digest
artifact provenance references bundle/task/node

17.5 Secrets

MVP secrets support should be minimal.

Recommended launch stance:

no secrets in hosted/community tasks
secrets not shown in logs/UI/debug by default
explicit grant required per task/environment
short-lived scoped secret materialization
secret access audited
secret redaction in logs

For the first launch demo, avoid secrets entirely.

17.6 Container/runtime hardening

Linux Podman backend should default to:

rootless
no privileged containers
no host Docker/Podman socket mount
read-only source mount
writable task workdir only
limited network by default
seccomp/AppArmor/SELinux where available
resource limits where practical

Windows dev backend should be clearly scoped as user-attached-node execution, not public managed secure isolation.

---

18. Implementation milestones

This sequence avoids building a beautiful UI on top of a runtime that cannot yet prove the thesis.

Milestone 0: Repository and protocol spine

Deliverables:

monorepo layout
shared IDs and protocol types
CoordinatorStore trait
Postgres persistent-store migrations skeleton
CLI skeleton
node skeleton
SDK skeleton
VS Code extension skeleton
DAP skeleton
example project skeleton

Acceptance criteria:

cargo workspace builds
CLI can print discovered project info
coordinator starts locally with Postgres for persistent project/node state and in-memory runtime state
node starts and registers fake heartbeat locally
VS Code extension can run a dummy DAP session

Key considerations:

• Do not overdesign protocol versioning, but add a protocol version field immediately.
• Use stable IDs everywhere.
• Add tenant_id to data models from the first migration.

Milestone 1: Local single-node Wasm task execution

Deliverables:

#[disasmer::main]
#[disasmer::task]
DisasmerArg derive
named task registry
Wasmtime runner
spawn/join on same node
small serialized args/results
basic logs
basic DAtomicU32 or equivalent demo handle

Acceptance criteria:

example local-one-node runs main task
main spawns child Wasm task
child returns serializable result
parent joins result
logs show per virtual thread

Considerations:

• Keep the first task ABI tiny.
• Ban closures for MVP.
• Fail early on invalid args.

Milestone 2: Environment discovery and bundle generation

Deliverables:

envs/** discovery
env!("name") compile/editor validation
disasmer build creates app.dis directory
bundle manifest
task manifest
environment manifest
source snapshot manifest placeholder
VFS seed manifest

Acceptance criteria:

adding envs/linux/Containerfile makes env!("linux") valid
missing env gives useful error
bundle digest changes when Containerfile changes
bundle digest changes when task ABI changes

Considerations:

• Keep disasmer.toml optional.
• Make generated metadata visible and inspectable.

Milestone 3: VFS and artifacts

Deliverables:

per-task VFS overlay
/vfs/project or /src mount
/vfs/work
/vfs/artifacts
fs::copy
fs::flush metadata publication
Artifact::from_path
fs::sync export-to-local-directory
content-addressed blob/chunk store

Acceptance criteria:

task writes artifact
flush publishes artifact metadata for active processes
parent joins Artifact handle
package task reads artifact handle
sync exports final artifact to dist/
large artifact bytes do not go through coordinator in local mode

Considerations:

• Build the metadata/byte split now.
• Ensure artifact path namespace is stable.
• Add tests proving flush() is metadata-only.

Milestone 4: Linux Podman command backend

Deliverables:

Podman rootless env materialization
Containerfile build/cache by digest
cmd!(...).run()
stdout/stderr streaming
process limits/log caps
artifact staging from command output
basic process suspend/resume for debug epoch

Acceptance criteria:

Linux task runs cargo build in envs/linux/Containerfile
logs stream to CLI/coordinator
artifact is published through VFS
local source checkout is bind-mounted when task runs locally

Considerations:

• Keep the first Containerfile expectations simple.
• Do not support arbitrary privileged containers.
• Do not mount the container engine socket.

Milestone 5: Coordinator/node split with hosted-ready persistent store

Deliverables:

coordinator service
Postgres-backed persistent store
node enrollment token
node identity
node heartbeat
bundle registration
task placement
StartTask command over persistent session
logs/events over stream
local coordinator mode still works

Acceptance criteria:

coordinator runs separately from node
node attaches with token
disasmer run schedules task onto attached node
node executes task and reports completion
Postgres contains persistent project/node/bundle metadata; active process/thread/artifact metadata lives in memory

Considerations:

• Use same code path for local and hosted as much as possible.
• Do not create a special “demo scheduler” that will be thrown away.

Milestone 6: Quinn transport and rendezvous

Deliverables:

Quinn transport wrapper
node-to-coordinator persistent session if practical
node-to-node direct connection
rendezvous endpoint exchange
authenticated stream framing
blob/artifact chunk transfer protocol
no coordinator bulk relay

Acceptance criteria:

two nodes establish direct QUIC link
node A can request authorized artifact chunk from node B
failed direct connection produces clear error
coordinator never receives bulk artifact bytes by default

Considerations:

• Keep HTTPS bootstrap if it makes auth simpler.
• Put transport behind traits for testability.
• Add trace logs for failed connectivity.

Milestone 7: Windows attached node backend

Deliverables:

Windows node build/test
WindowsCommandDevRunner
per-task workspace
process/log capture
artifact staging
best-effort process tree lifecycle
Windows sandbox backend trait + stub
Forgejo Windows worker setup docs

Acceptance criteria:

Windows node attaches to hosted/local coordinator
scheduler places env!("windows") task on Windows node
Windows task runs cargo build or a simple command
artifact is published and joined by parent
VS Code/coordinator shows Windows virtual thread

Considerations:

• Be explicit that this is not production-grade untrusted Windows sandboxing.
• Do not offer managed public Windows compute until isolation is real.

Milestone 8: All-stop debugging and DAP

Deliverables:

custom DAP adapter
launch/attach
thread list
breakpoint setting
source/probe mapping
Debug Epoch coordinator
node freeze/resume
Wasm safepoints/probes
command process suspend/resume integration
stack/scopes/variables for task args and handles
stdout/stderr output events
restart selected/failed task command

Acceptance criteria:

VS Code F5 starts a virtual process
breakpoint in main stops all virtual threads
breakpoint in Linux compile task stops Windows task too
breakpoint in Windows task stops Linux task too
thread list is coherent
user can inspect Target arg and Artifact handle
continue resumes all tasks
task restart works after source edit

Considerations:

• This is the riskiest milestone. Start early.
• Use probe-granularity stepping first.
• Make all-stop correctness more important than perfect DWARF locals.

Milestone 9: Local-first Git/source snapshots

Deliverables:

git source provider module
commit/submodule/dirty overlay manifest
local checkout detection
bind-mount local checkout for local tasks
remote missing-content transfer by hash
scheduler locality scoring
latency regression tests

Acceptance criteria:

local Linux build from local checkout does not upload repo
remote Windows build receives only required source content
changing one file transfers only that file/overlay where possible
scheduler prefers node with source checkout and env cache

Considerations:

• Keep source provider modular.
• Do not hardwire Git into the VFS core.

Milestone 10: Hosted coordinator launch hardening

Deliverables:

private hosted Authentik/OIDC login
tenant/project UI
node attach UI
hosted/community quota policy module
authz middleware and private hosted OIDC integration
cross-tenant negative tests
community tier limits
rate limiting
log/artifact caps
admin kill switch
basic plan/policy flags, even if only community exists

Acceptance criteria:

public user can sign in
public user can create project
public user can attach own node
public user cannot run hosted native command/container
public user can run task on own node
cross-tenant tests pass
quota tests pass

Considerations:

• Do not launch without abuse controls.
• Design quotas as first-class resource accounting.

Milestone 11: Launch demo polish

Deliverables:

launch-build example
README quickstart
short screencast script
hosted landing page
install script/packages
VS Code extension packaging
error message pass
first-run diagnostics
artifact export UX
fixed coordinator dashboard
operator panel preview scaffold

Acceptance criteria:

fresh developer can follow quickstart
Linux node attach works
Windows node attach works
F5 demo works
breakpoint demo works
restart failed task demo works
artifact export works
README explains community tier honestly

Considerations:

• Public launch users will try weird setups immediately.
• Error messages matter as much as features.
• Make the first demo resilient.

---

19. Testing strategy

19.1 Unit tests

Required areas:

DisasmerArg derive allowed/rejected types
environment discovery
bundle digest stability
manifest generation
VFS path normalization
flush manifest generation
sync policy selection
hosted/community quota policy module
scheduler scoring
authz checks
protocol encoding/decoding

19.2 Integration tests

Local integration tests:

single-node spawn/join
two Wasm tasks with DAtomic
task writes artifact and parent consumes it
cmd runner fake backend
Debug Epoch with two Wasm tasks
restart failed task with compatible bundle
restart fails with incompatible schema

Linux integration tests:

Podman build env
Podman run command
local checkout bind mount
artifact publish/export
process suspend/resume if supported

Windows integration tests:

node attach
command execution
log capture
artifact staging
process lifecycle cleanup

Network integration tests:

node-to-node QUIC connect
artifact chunk transfer
connection failure clear error
no coordinator bulk relay

19.3 End-to-end tests

E2E test scenarios:

F5 local single node
hosted coordinator + attached Linux node
hosted coordinator + Linux + Windows nodes
breakpoint in Linux task all-stops process
breakpoint in Windows task all-stops process
restart failed Windows task
local-first no-upload test
community tier quota enforcement
cross-tenant forbidden access

19.4 Latency regression tests

Add tests that fail if:

local source bytes are uploaded during local build
artifact bytes go to coordinator during flush
full repo tarball is created by default
large artifact is transferred before consumer/export/sync needs it
scheduler chooses remote node despite local compatible warm node without reason

19.5 Security tests

Required negative tests:

user cannot read another tenant's process
user cannot attach debugger to another tenant's process
node cannot claim another tenant's task
node cannot publish artifact for unauthorized process
community hosted task cannot request cmd.run
community hosted task cannot request network
operator panel cannot submit custom HTML/JS
log cap truncates noisy task
quota rejects spawn storm

---

20. Documentation required for MVP

20.1 User docs

README quickstart
What is Disasmer?
Install CLI
Install VS Code extension
Login to hosted coordinator
Attach a Linux node
Attach a Windows node
Create envs/<name>/Containerfile
Write first build program
Debug in VS Code
Publish artifacts with flush()
Export artifacts through an attached receiver node or explicit storage integration
Community tier limits
Security model for user-attached nodes

20.2 Engineering docs

architecture.md
protocol.md
task-abi.md
vfs.md
debugging.md
transport.md
security.md
operator-panel.md
windows-backend.md

20.3 Error messages as docs

Invest heavily in errors:

missing env
node unavailable
environment incompatible
large task arg
host-only type in DisasmerArg
cmd used on coordinator
cmd capability missing
artifact bytes unavailable
QUIC direct connection failed
sync required for export
Debug Epoch freeze failed
restart incompatible after source change
community tier quota exceeded

---

21. Key risks and mitigations

21.1 Risk: debugger is the hardest part

Mitigation:

• Build DAP skeleton early.
• Implement Debug Epochs before fancy UI.
• Use generated debug probes for reliable breakpoints first.
• Add DWARF/local richness incrementally.
• Define launch acceptance around real breakpoints in the demo code, thread list, task args, handles, stacks, all-stop, continue, and restart.

Experiment result: experiments/debugger-gate has derisked the core VS Code debugging UX. It proves a custom DAP adapter can present a virtual process with main and task threads, bind ordinary VS Code gutter breakpoints to Wasmtime-backed Rust/Wasm source locations, hit breakpoints in multiple virtual tasks, support basic stepping/watch/variables, and keep the backend boundary shaped like a replaceable virtual-process runtime. Treat that experiment as a proof and reference, not as implementation foundation: it is intentionally hacky prototype code with hardcoded example-local variable mapping and a simplified local Wasmtime scheduler.

21.2 Risk: Windows isolation is not real yet

Mitigation:

• Keep Windows as user-attached node execution only.
• Label backend as windows-command-dev.
• Do not offer public managed Windows compute in MVP.
• Keep sandbox backend interface ready.
• Investigate Windows Sandbox/AppContainer after public launch or before managed nodes.

21.3 Risk: hosted community tier becomes compute abuse target

Mitigation:

• No community hosted containers.
• No community hosted native commands.
• Zero-cap hosted Wasm control loop only.
• Fuel, wall-clock, memory, logs, and metadata limits.
• Rate-limit every API and event stream.
• Require attached user node for real work.

21.4 Risk: QUIC/NAT becomes launch blocker

Mitigation:

• Use coordinator control session for scheduling and metadata.
• Require direct node-to-node only when bytes must move.
• Prefer scheduling where bytes already are.
• Fail clearly when direct transfer is impossible.
• For launch demo, place Linux and Windows nodes in a connectivity setup you control.

21.5 Risk: local-first goal gets lost

Mitigation:

• Add tests that fail on unwanted uploads.
• Instrument bytes moved per build.
• Show “bytes moved” in logs/dashboard.
• Make flush() metadata-only by default from day one.

21.6 Risk: product sounds too ambitious

Mitigation:

• Demo one narrow workflow extremely well.
• Keep the public README focused on build systems.
• Keep architecture docs honest about non-goals.
• Do not claim managed secure compute until it exists.
• Emphasize open-source local-first runtime.

---

22. Definition of done for the public-launch MVP

The public-launch MVP is done when a clean machine can:

1. Install Disasmer CLI.
2. Install the VS Code extension.
3. Sign in to hosted coordinator through Authentik/OIDC.
4. Create or select a project.
5. Attach a Linux node.
6. Attach a Windows node.
7. Clone the launch-build demo repo.
8. Open it in VS Code.
9. See env!("linux") and env!("windows") recognized.
10. Press F5.
11. See one virtual process start.
12. See Linux and Windows virtual threads in VS Code.
13. Hit a breakpoint in a spawned task.
14. Observe all-stop Debug Epoch behavior.
15. Inspect task args, command status, logs, and artifact handles.
16. Edit source and restart only the failed/selected task.
17. Finish the build.
18. Export artifacts through an attached/local receiver node or explicit storage integration.
19. Confirm local-first behavior in logs/metrics.
20. Confirm hosted coordinator did not run arbitrary native commands.

---

23. Suggested implementation order inside each milestone

When in doubt, build vertical slices rather than broad infrastructure.

Recommended pattern:

1. Fake backend, real API.
2. Real backend, local only.
3. Real backend, coordinator split.
4. Real backend, hosted auth/quotas.
5. Polish only after the vertical slice proves the thesis.

Example for debugging:

1. DAP shows fake virtual process with fake threads.
2. DAP attaches to local one-node Wasm process.
3. Breakpoint probe freezes one Wasm task.
4. Debug Epoch freezes two Wasm tasks.
5. Debug Epoch freezes Wasm + command task.
6. Same flow across two nodes.
7. VS Code restart selected task.
8. Richer locals/stacks.

Example for VFS:

1. In-memory manifest.
2. Local disk overlay.
3. Artifact handle.
4. Metadata-only flush.
5. Export sync.
6. Node-to-node chunk fetch.
7. Explicit receiver-node export.

---

24. External technical references to keep nearby

These are useful implementation references, not product dependencies:

• Wasmtime Rust embedding API: https://docs.wasmtime.dev/api/wasmtime/
• Wasmtime Component Model API: https://docs.wasmtime.dev/api/wasmtime/component/index.html
• Quinn repository and docs: https://github.com/quinn-rs/quinn
• Debug Adapter Protocol: https://microsoft.github.io/debug-adapter-protocol/
• VS Code debugger extension guide: https://code.visualstudio.com/api/extension-guides/debugger-extension
• Authentik OAuth2/OIDC provider docs: https://docs.goauthentik.io/add-secure-apps/providers/oauth2/
• Podman: https://podman.io/
• Nix flakes: https://nix.dev/concepts/flakes.html
• Windows Sandbox CLI: https://learn.microsoft.com/en-us/windows/security/application-security/application-isolation/windows-sandbox/windows-sandbox-cli
• Windows AppContainer isolation: https://learn.microsoft.com/en-us/windows/win32/secauthz/appcontainer-isolation
• OWASP API Security Top 10 2023, unrestricted resource consumption: https://owasp.org/API-Security/editions/2023/en/0xa4-unrestricted-resource-consumption/
• OAuth 2.0 Security Best Current Practice, RFC 9700: https://datatracker.ietf.org/doc/rfc9700/
• NIST SP 800-190, Application Container Security Guide: https://csrc.nist.gov/pubs/sp/800/190/final

---

25. Final guidance

Do not shrink the ambition of the demo, but aggressively constrain the implementation semantics.

The MVP should be ambitious in what the user sees:

one Rust build program
hosted coordinator
Linux + Windows nodes
source-level VS Code debugging
all-stop Debug Epochs
task restart
local-first artifacts

It should be conservative in what the runtime promises internally:

no transparent raw remote pointers
no live stack migration
no coordinator bulk data plane
no community hosted native compute
no fake Windows sandbox claims
no arbitrary child-process source debugging
no automatic survival of non-replicated node failure

That combination is what can make the launch credible: the visible experience is striking, while the implementation model remains honest enough to actually build.