# Disasmer MVP Engineering Plan **Status:** draft MVP execution plan **Audience:** engineering / founder-engineer **Goal:** build a public-launchable MVP that proves Disasmer can replace CI/build systems with a general distributed Wasm runtime whose flagship use case is distributed, local-first, source-debuggable builds. --- ## 0. Executive summary Disasmer should launch as a **general distributed Wasm runtime** with a **build-system experience so strong that it is the obvious wedge**. The MVP is not a toy local scheduler and not merely a CI runner with nicer YAML. It must demonstrate the core thesis: > A build can be a normal Rust program, running as one virtual process across multiple machines, with build steps represented as virtual threads/tasks, artifacts represented by a low-latency virtual filesystem, and debugging represented as one normal VS Code debug session. The launchable MVP should include: - Open-source local runtime, SDK, CLI, node runtime, protocol definitions, and VS Code extension. - Hosted coordinator from day one, with persistent project/identity state in Postgres and private hosted Authentik/OIDC/community-policy modules; the community tier coordinates but does not provide arbitrary hosted compute. - User-attached Linux and Windows nodes. - Containerfile-based environment discovery for MVP user projects. - Rootless Podman backend for Linux command execution. - Windows command backend sufficient for executing commands unsandboxed, with the Windows sandboxing backend explicitly stubbed (to be implemented later) and isolated behind an interface. - Wasmtime-based execution for Disasmer guest/orchestration code. - Native command execution only through node runtimes, never through the hosted coordinator control plane. - Small serializable task arguments and results. - Large inputs/outputs passed through handles: `SourceSnapshot`, `Artifact`, `Blob`, and VFS manifests. - Local-first VFS where `flush()` publishes metadata and `sync()` triggers explicit user-chosen export/transfer/durability behavior; artifacts are otherwise ephemeral node-local data. - Git support as an optional source-provider module that is included by default, with non-Git providers pluggable and dependencies avoidable when unused. - All-stop debugging with Debug Epochs as a mandatory MVP capability. - A Disasmer DAP adapter that targets VS Code first while staying usable by other DAP clients. - Task restart from filesystem/artifact checkpoint boundaries, not live stack migration or hot-swapping. - Abuse controls designed into the object model and control plane, not bolted on later. - Operator panel foundations in the protocol and runtime-state model, with the full immediate-mode panel as the second product milestone. The flagship demo should make people immediately understand why this is not “yet another CI system”: 1. The repo has `envs/linux/Containerfile` and `envs/windows/Containerfile`. 2. The build is one Rust source file using `#[disasmer::main]`, `#[disasmer::task]`, `spawn::task`, `env!("linux")`, `env!("windows")`, `cmd!(...)`, `fs::flush()`, and `fs::sync() / artifact::export_to_node()`. 3. A Linux node and a Windows node can be attached to the hosted coordinator. 4. Pressing F5 in VS Code starts one virtual process. 5. VS Code shows `main`, `compile linux`, `compile windows`, and `package` as threads of one debug target. 6. A breakpoint in the Windows build task stops the whole virtual process using a Debug Epoch. 7. The user edits the build logic, restarts only the failed task from a clean VFS checkpoint, and continues. 8. Artifacts appear through the VFS without coordinator-routed bulk data. --- ## 1. Product stance and launch bar ### 1.1 Positioning Disasmer should be positioned as: > A distributed Wasm runtime that makes distributed execution feel like ordinary source-level-debuggable multithreaded programming, with build systems as the first killer use case. It is not primarily: - a hosted CI SaaS; - a YAML syntax replacement; - a remote shell wrapper; - a Kubernetes abstraction; - a general-purpose compute subsidy; - a task queue with logs; - or a build cache alone. It should feel closer to: > “What if my build system were an actual program, and the debugger saw the whole distributed build as one process?” ### 1.2 Public-launch promise The launch should be built around a concrete promise: ```text Add a Containerfile. Reference it from Rust with env!("name"). Press F5. Debug the distributed build like one normal program. ``` The public launch should be able to show a short video or GIF with these beats: ```text 1. VS Code open on a Rust build program. 2. env!("linux") and env!("windows") autocomplete from envs/**/Containerfile. 3. Press F5. 4. Hosted coordinator shows two attached nodes. 5. Linux and Windows build tasks start. 6. VS Code thread list shows virtual threads. 7. Breakpoint hits in the Windows task. 8. Both tasks stop. 9. User inspects args/locals/artifact handles. 10. User restarts only the failed task. 11. Final artifacts are explicitly exported. ``` ### 1.3 What must be true before public launch The MVP is launchable only if these are true: - A user can use the hosted coordinator without you granting them compute. - A user can attach their own Linux node and run a build task on it. - A user can attach a Windows node and run a Windows command task on it. - Source code references environments by logical name, not by runner label or machine name. - Local-first Linux builds do not upload the repo, tar the checkout, route file reads through the coordinator, or automatically upload build outputs. - `flush()` is cheap and metadata-first. - `sync()` is explicit and may move bytes. - The debugger shows one process and many virtual threads. - A Debug Epoch stops all Wasm tasks and controlled native command tasks for the virtual process. - Task restart works from task entrypoint or last published VFS checkpoint. - Community hosted users cannot run arbitrary containers or native commands on your infrastructure. - Every public object has tenant ownership, quota accounting, and authorization checks. ### 1.4 What can be rough at launch These can be rough, provided the rough edges are honest and isolated: - Windows sandboxing can be a stubbed backend with a clear `windows-command-dev` capability, as long as hosted managed Windows compute is not offered to untrusted users. - Nix can be included as a default plugin but not required for the flagship Containerfile demo. - The operator panel can be scaffolded and visible as a preview, with the full immediate-mode UI in the second milestone. - Debugger variable inspection can prioritize task args, Disasmer handles, top-level locals around Disasmer API calls, stack frames, and command status before supporting every possible Rust compiler local perfectly. - NAT traversal can be coordinator-aided without relay. If direct node-to-node connectivity fails, scheduling and artifact flows should avoid requiring peer transfer rather than silently relaying bulk data through the coordinator. --- ## 2. Hard MVP decisions from the project constraints ### 2.1 Product scope The MVP includes both: - **open-source local-first runtime**, because credibility depends on not hiding the core runtime; and - **hosted coordinator**, because the GitHub Actions replacement story needs an easy entry point, public rendezvous, auth, node registry, debug sessions, and a web surface. The hosted coordinator is a control plane, not a general-purpose compute platform. ### 2.2 First target user The first target user is: > A developer or small team that currently uses GitHub Actions, Forgejo Actions, or similar CI, and wants something more programmable, local-first, debuggable, and self-hostable. The first user does not need managed compute from Disasmer. They need: - local and self-hosted nodes; - a hosted coordinator for convenience; - VS Code debugging; - artifact metadata, ephemeral artifact handles, and explicit output export; - a better mental model than YAML pipelines; - and a credible path to replacing CI. ### 2.3 Host platforms MVP node platforms: ```text Linux required Windows required macOS not required for launch, but kept in the model BSD-like not required for launch, but kept in the model ``` Linux should be the most complete backend. Windows should be complete enough to run a real attached worker and prove cross-platform task placement. macOS remains a design-supported later node type. ### 2.4 Environment support MVP user-facing environment support: ```text Containerfile / Dockerfile discovery: required Nix plugin: included by default, but not launch-critical for all flows Windows sandbox backend: interface + stub, not trusted managed compute ``` The MVP should discover: ```text envs//Containerfile envs//Dockerfile ``` It may also discover: ```text envs//flake.nix envs//flake.lock ``` But the flagship launch demo should not depend on Nix unless it is already stable in the implementation. ### 2.5 Execution support The MVP must run both: - Wasm task entrypoints, using Wasmtime on node runtimes; and - native commands, through `cmd!(...)`, executed only by node runtimes inside declared environments. The coordinator must not run native commands or containers. The hosted control loop may run zero-capability Wasm, but only with strict fuel, memory, log, state, and wall-clock limits. ### 2.6 Wasm model Use the simplest Wasmtime model that is secure and functional: ```text Core Wasm module + explicit host imports first. Component Model support later or behind an internal abstraction. ``` Reasoning: - Core Wasm embedding is enough for task entrypoints, host imports, memory access, and debug probes. - The MVP does not need to expose a stable cross-language component ABI yet. - The runtime should keep an internal `GuestAbi` trait so Component Model support can replace or augment this later. ### 2.7 Data model MVP task boundaries allow only: ```text small serializable values immutable content-addressed objects Disasmer runtime handles ``` Do not allow: ```text raw pointers borrowed references local file handles sockets process handles MutexGuards unbounded byte vectors native closures live stack migration ``` Large data moves through: ```text SourceSnapshot Blob Artifact VirtualFile VfsManifest ``` ### 2.8 Checkpoint model MVP checkpointing means: ```text filesystem/artifact checkpoints for task restart ``` It does not mean: ```text live Wasm stack checkpointing native process checkpointing socket checkpointing transparent migration of running processes hot-swapping code into existing stack frames ``` A restartable task must be recreated from: ```text task name new module hash original serialized args same environment handle same source snapshot or selected dirty overlay latest allowed VFS checkpoint ``` ### 2.9 Debugging model All-stop debugging is mandatory. The debugger must present: ```text one virtual process many virtual threads/tasks normal breakpoints in Disasmer Rust task code normal thread list stacks selected locals / task args / handle inspection stdout/stderr per task artifacts and VFS state in optional views restart selected task ``` The MVP should provide a Disasmer-owned DAP adapter. VS Code is the first supported client, but the adapter should speak ordinary DAP so other IDEs or tools can integrate later without Disasmer providing those integrations. LLDB/lldb-dap can be used internally later for richer DWARF support, but the virtual-process illusion should remain owned by Disasmer. ### 2.10 Transport Use Quinn for MVP QUIC transport, behind a transport abstraction: ```text node-to-node data/control links: QUIC over Quinn node-to-coordinator persistent session: QUIC preferred, HTTPS/WebSocket allowed for auth/bootstrap bulk transfer: node-to-node only, no coordinator bulk relay ``` Quinn is the MVP default because it is Rust-native and async-friendly, which keeps packaging and dogfooding simpler than an external C runtime dependency. Keep the `Transport` trait strict enough that quiche, TQUIC, or another implementation can replace it later if benchmarks, platform packaging, or protocol features justify a change. The coordinator can help with rendezvous and endpoint candidate exchange. It should not relay bulk traffic. MVP can avoid cross-node bulk transfer if direct connectivity fails by scheduling tasks where the data already is or requiring explicit export to a receiver node or user-configured storage integration. ### 2.11 Storage Use an abstract coordinator store with a strict split between persistent product state and ephemeral virtual-process state. Persistent state should use Postgres in the hosted deployment and should be limited to things that must survive coordinator restart: ```text users / tenants / memberships where applicable projects node identities and enrollment records node public keys / credentials / revocation state bundle registry and environment metadata worth reusing source-provider configuration metadata hosted/community policy configuration billing/plan flags later longer-lived audit events where needed ``` Ephemeral runtime state should live in coordinator memory for the MVP: ```text active virtual processes virtual threads and task placement Debug Epochs pending spawn queues scheduler decisions live node sessions and heartbeats recent logs/events VFS manifest cache for active processes artifact metadata for active processes quota counters for active processes, except hosted/community abuse ledgers that the private service wants to persist ``` If the coordinator restarts during the MVP, active virtual processes may be lost and should restart cleanly. That is acceptable. Do not overbuild durable process recovery before the runtime/debugging thesis is proven. The store abstraction still matters because later the coordinator may need durable process recovery, Raft, sharding, or a different store. The MVP should not spend engineering effort on distributed consensus before the product is proven. ### 2.12 Auth Support two auth modes from the start: ```text hosted/community service auth: Authentik through OAuth2/OIDC, implemented in a private hosted module CLI human auth: browser callback against the hosted service CLI/agent auth: public-key identity with explicit enrollment/authorization node attach: short-lived enrollment token -> long-lived node key/certificate service-to-service: scoped service tokens where needed open-source/private coordinator auth: simple local/team trust model with pluggable auth hooks, not the hosted OIDC stack by default ``` User OAuth tokens must not be passed to nodes. Nodes authenticate with node identity, and tasks receive only explicit, scoped, short-lived task tokens or secrets. The CLI should be comfortable for both humans and agents. Browser login is fine for a developer at a workstation; agents should be able to use a generated keypair, an enrollment challenge, and a scoped project/node credential without opening a browser. ### 2.13 Community hosted abuse stance The community hosted tier should prove the platform, not subsidize builds. The word “community” is intentional: it should be useful for serious users, while enterprise/team use cases should naturally need more than one active process, longer retention, stronger identity, or managed capacity. Community hosted mode supports: ```text sign in create project (may be implicitly created, max 1) create virtual process (may be implicitly created, max 1) open coordinator UI attach own node (max 4 nodes) spawn tasks onto own node (max 8 tasks in flight) debug from VS Code see logs/artifact metadata for active processes (logs can't be bigger than 16KiB) ``` Community hosted mode does not support: ```text hosted containers hosted native commands default hosted artifact storage public artifact hosting network access from hosted Wasm long-running compute relay bandwidth ``` ### 2.14 Operator panel The operator panel is second milestone, but the MVP should design toward it: - runtime-state structures for panel state and events; - protocol namespace for typed UI updates; - control-plane UI slot for a panel; - built-in widgets only; - no custom HTML/JS; - read-only behavior when the virtual process is stopped in a debugger. - The panel has very tight size/state constraints for community mode For public launch, a fixed coordinator dashboard is enough if it clearly hints at the immediate-mode panel API. --- ## 3. MVP architecture ### 3.1 High-level architecture ```text VS Code ├─ Disasmer VS Code extension └─ Disasmer DAP adapter │ ▼ Hosted or local coordinator ├─ Auth / tenants / projects ├─ node registry and rendezvous ├─ virtual process registry ├─ scheduler ├─ debug epoch coordinator ├─ VFS/artifact metadata for active processes ├─ hosted/community policy hooks ├─ operator panel state/events scaffold ├─ in-memory active runtime state └─ Postgres persistent project/identity store │ ├────────────── QUIC / control session ───────────────┐ ▼ ▼ Linux node runtime Windows node runtime ├─ Wasmtime task runner ├─ Wasmtime task runner ├─ command runner ├─ command runner ├─ Podman environment backend ├─ Windows command backend ├─ Nix plugin backend ├─ Windows sandbox stub backend ├─ VFS overlay store ├─ VFS overlay store ├─ ephemeral artifact/blob cache ├─ ephemeral artifact/blob cache ├─ debug agent ├─ debug agent ├─ node cache ├─ node cache └─ Quinn peer transport └─ Quinn peer transport ``` ### 3.2 Control plane vs data plane The coordinator owns: ```text identity membership node capabilities bundle metadata virtual process metadata task placement decisions debug epochs VFS manifest metadata for active processes artifact metadata for active processes quota accounting operator panel metadata rendezvous ``` All of these need constraints so that the community service can't be abused. The nodes own: ```text Wasm execution native command execution environment materialization source checkout access build caches VFS bytes artifact bytes large blob transfer debug stop/resume implementation ``` User-attached nodes are the users' responsibility and can run broad local capabilities. The hosted coordinator must still validate what capabilities are accepted for a hosted/community process, but the open-source/private coordinator should not block normal local-node capabilities by default. The coordinator must not become: ```text a network filesystem a blob proxy a remote compiler filesystem a hidden build runner a relay by default ``` ### 3.3 Recommended repository layout ```text disasmer/ crates/ disasmer-sdk/ # guest-facing Rust API disasmer-macros/ # #[disasmer::main], #[disasmer::task], env!() disasmer-abi/ # guest/host ABI structs and serialization disasmer-cli/ # user CLI disasmer-coordinator/ # hosted/local coordinator service disasmer-node/ # node runtime daemon disasmer-dap/ # custom Debug Adapter Protocol server disasmer-vscode-protocol/ # extension <-> CLI/DAP shared types disasmer-transport/ # Quinn-backed QUIC wrapper and protocol framing disasmer-store/ # store traits + Postgres persistent-store impl disasmer-runtime-state/ # in-memory active process/thread/debug state disasmer-vfs/ # VFS manifests, overlays, blob/chunk references disasmer-env/ # env discovery, normalization, caching disasmer-source/ # source-provider traits and manifests disasmer-git/ # default optional Git source provider module disasmer-debug-model/ # breakpoints, epochs, stack/locals schema disasmer-ui-model/ # immediate-mode UI schema scaffold disasmer-auth/ # node enrollment, pubkey identities, pluggable auth hooks disasmer-protocol/ # coordinator/node/DAP wire messages private/ hosted-oidc/ # Authentik/OIDC integration for the public service hosted-policy/ # community quotas, abuse controls, zero-cap hosted-loop rules hosted-admin/ # service admin tools, kill switches, internal dashboards vscode/ extension/ # TypeScript extension examples/ launch-build/ # flagship Linux + Windows build demo local-one-node/ # minimal spawn/join example operator-panel-preview/ # second-milestone preview docs/ MVP.md architecture.md protocol.md security.md debugging.md ``` The private folders are normal repository folders during development, but must be easy to exclude from the public source release. The public crates should expose policy/auth interfaces; the private modules implement hosted-service-specific OIDC, community quota enforcement, hosted zero-capability Wasm limits, abuse tooling, and admin affordances. ### 3.4 Process model MVP process types: ```text disasmer coordinator Hosted or local control-plane service. disasmer node Long-running node runtime attached to a coordinator. disasmer dap Debug Adapter Protocol process launched by VS Code. disasmer cli User/agent CLI for login, init, build, run, node attach, artifact export. ``` ### 3.5 Primary user commands ```bash # Authenticate a human against the hosted coordinator. disasmer login https://disasmer.example.com # Create or enroll a non-browser CLI identity for an agent. disasmer auth keygen --name forgejo-worker disasmer auth enroll-key --coordinator https://disasmer.example.com --project my-project --public-key ~/.config/disasmer/keys/forgejo-worker.pub # Initialize project metadata; optional because defaults should work. disasmer init # Show discovered environments, source providers, and snapshot policy. disasmer inspect # Build the virtual binary/bundle. disasmer build # Attach a node. OS, architecture, container backend, Git availability, and most # capabilities are auto-detected; --cap remains available for explicit overrides. disasmer node attach --coordinator https://disasmer.example.com --token dis_node_enroll_... # Override or add an auto-detected capability only when needed. disasmer node attach --coordinator https://disasmer.example.com --token dis_node_enroll_... --cap env.windows-command-dev=true # Run without VS Code. Hosted coordinator is the default when logged in; # project defaults to the current directory. Entry point is optional. disasmer run disasmer run build-linux disasmer run --project ../other-repo package-release disasmer run --local smoke-test # Export final artifacts by asking an attached/local receiver node to pull them. # There is no default coordinator artifact store. disasmer artifact export --process --to ./dist ``` `disasmer run` should accept an optional entry point. This is a good idea because one Disasmer project may contain several useful virtual-process entrypoints: `build`, `test`, `package`, `release`, `watch`, `operator`, or project-specific workflows. The default remains the `#[disasmer::main]` entrypoint when no entry is supplied. VS Code should hide most of this behind: ```text Disasmer: Login Disasmer: Attach Local Node Disasmer: Build Bundle Disasmer: Run Disasmer: Debug Disasmer: Restart Failed Task Disasmer: Export Artifacts ``` --- ## 4. The flagship launch example ### 4.1 Project layout ```text launch-build-demo/ Cargo.toml Cargo.lock src/ main.rs envs/ linux/ Containerfile windows/ Containerfile inputs/ build-config.json disasmer.toml # optional; only used for launch polish/overrides ``` ### 4.2 Example `disasmer.toml` The file should be optional, but the demo can include it to make platform constraints obvious. ```toml [project] name = "launch-build-demo" [envs.linux] path = "envs/linux/Containerfile" kind = "container" requires = { os = "linux", arch = "x86_64" } [envs.windows] path = "envs/windows/Containerfile" kind = "container" requires = { os = "windows", arch = "x86_64" } backend_hint = "windows-command-dev" [vfs] include = [ "src/**", "Cargo.toml", "Cargo.lock", "inputs/**" ] exclude = [ "target/**", ".git/**" ] [git] enabled = true dirty_overlay = true submodules = true ``` ### 4.3 Example Rust build program ```rust use disasmer::prelude::*; #[derive(Serialize, Deserialize, Clone, Debug, DisasmerArg)] enum Target { LinuxX64, WindowsX64, } impl Target { fn env(&self) -> Env { match self { Target::LinuxX64 => env!("linux"), Target::WindowsX64 => env!("windows"), } } fn name(&self) -> &'static str { match self { Target::LinuxX64 => "linux-x64", Target::WindowsX64 => "windows-x64", } } fn command(&self) -> CommandSpec { match self { Target::LinuxX64 => cmd!("cargo", "build", "--release"), Target::WindowsX64 => cmd!("cargo", "build", "--release"), } } } #[disasmer::main] async fn main() -> Result<()> { // The hosted coordinator control loop is intentionally capless. Source access // happens on a user-attached node through the default Git source-provider // module, not inside the coordinator process. let src = spawn::task(prepare_source) .placement(placement::any_node().with_cap("source.git")) .queue_until_node_available() .start() .await? .join() .await?; let completed = DAtomicU32::new(0).await?; let linux = spawn::task(compile) .arg(BuildArgs::new(Target::LinuxX64, src.clone(), completed.clone())) .env(env!("linux")) .start() .await?; let windows = spawn::task(compile) .arg(BuildArgs::new(Target::WindowsX64, src.clone(), completed.clone())) .env(env!("windows")) .start() .await?; let linux_artifact = linux.join().await?; let windows_artifact = windows.join().await?; let dist = package(vec![linux_artifact, windows_artifact]).await?; assert_eq!(completed.load(Ordering::SeqCst).await?, 2); // Export is explicit code. The coordinator does not provide a default // artifact store; this asks an attached receiver/local node to pull bytes. artifact::export_to_node(dist, receiver::local("dist")).await?; Ok(()) } #[disasmer::task] async fn prepare_source() -> Result { source::git::current_repo().snapshot().await } #[derive(Serialize, Deserialize, Clone, Debug, DisasmerArg)] struct BuildArgs { target: Target, src: SourceSnapshot, completed: DAtomicU32, } impl BuildArgs { fn new(target: Target, src: SourceSnapshot, completed: DAtomicU32) -> Self { Self { target, src, completed } } } #[disasmer::task] async fn compile(args: BuildArgs) -> Result { fs::mount("/src", args.src).await?; args.target.command() .cwd("/src") .env("DISASMER_TARGET", args.target.name()) .run() .await?; let output_path = match args.target { Target::LinuxX64 => "/src/target/release/app", Target::WindowsX64 => "/src/target/release/app.exe", }; let artifact_path = format!("/vfs/artifacts/{}/{}", args.target.name(), basename(output_path)); fs::copy(output_path, &artifact_path).await?; fs::flush().await?; args.completed.fetch_add(1, Ordering::SeqCst).await?; Artifact::from_path(artifact_path).await } #[disasmer::task] async fn package(outputs: Vec) -> Result { fs::create_dir_all("/vfs/artifacts/dist").await?; for output in outputs { fs::copy(output.path(), "/vfs/artifacts/dist/").await?; } fs::flush().await?; Artifact::from_path("/vfs/artifacts/dist").await } ``` ### 4.4 What the demo proves The demo proves: - The build graph is source code, not YAML. - Environment resources live in the repo. - Source code references `env!("linux")` and `env!("windows")`, not runner labels. - The coordinator schedules by logical environment and node capability. - Linux and Windows tasks can run in the same virtual process. - `cmd!(...)` runs on nodes, not on the hosted coordinator. - Outputs become artifact handles through the VFS. - `flush()` publishes artifacts for downstream tasks. - Final artifact export is explicit code and does not imply coordinator artifact storage. - VS Code sees one virtual process and multiple virtual threads. - A Debug Epoch stops the whole virtual process. - Restarting a failed task after editing source is part of the normal loop. --- ## 5. Rust SDK and task API ### 5.1 MVP API surface The MVP Rust SDK should include: ```rust #[disasmer::main] #[disasmer::task] env!("name") spawn::task(task_fn) TaskHandle Artifact Blob SourceSnapshot VirtualFile DAtomicU32 DAtomicU64 DChannel fs::mount() fs::copy() fs::flush() fs::sync() / artifact::export_to_node() cmd!(...) source::git::current_repo().snapshot() ui::* scaffold only ``` Do not include every distributed data structure in the first API. The launch demo needs small data, atomics/counters, task handles, source snapshots, artifacts, and VFS operations more than it needs a complete distributed memory library. ### 5.2 Task registration `#[disasmer::task]` should generate: - a stable task symbol; - a task ABI descriptor; - argument/result serialization metadata; - debug metadata for task entry, source spans, and selected local/argument values; - capability declarations inferred from APIs used where possible; - task restart compatibility hash; - a registration entry in the bundle manifest. Example generated manifest entry: ```json { "task_id": "compile@sha256:...", "symbol": "compile", "source": "src/main.rs", "arg_type": "BuildArgs@sha256:...", "result_type": "Artifact@sha256:...", "requires": ["vfs", "cmd", "artifact.publish"], "debug": { "spans": "debug/compile.spans.json", "locals_schema": "debug/compile.locals.json" } } ``` ### 5.3 Argument rules Task arguments and results must implement `DisasmerArg`. Allowed: ```text u32, u64, bool, String small structs/enums SourceSnapshot Artifact Blob Env DAtomicU32 / DAtomicU64 DChannel VirtualFile small Vec with configured size limit ``` Rejected: ```text &T / &mut T raw pointers Rc RefCell std::fs::File TcpStream MutexGuard native thread/process handles unbounded Vec closures with implicit captures ``` The compiler error should explain the distributed boundary. For example: ```text BuildArgs cannot be passed to a Disasmer virtual thread because it contains `&Path`. Use String, SourceSnapshot, VirtualFile, Blob, or Artifact instead. ``` ### 5.4 Size limits Set a default task argument size limit such as: ```text soft warning: 256 KiB hard default: 4 MiB project override: allowed but noisy ``` The hosted/community coordinator control loop should use much tighter limits, for example a 4 KiB warning and 16 KiB hard limit. User-attached nodes can keep the normal node defaults unless project policy overrides them. A user accidentally passing an 82 MB `Vec` should see: ```text Task argument is 82 MB and would be copied to the spawned virtual thread. Use Blob::from_file, SourceSnapshot, Artifact, or VirtualFile instead. ``` ### 5.5 Host imports The MVP guest ABI should be explicit and small. Core imports: ```text disasmer.spawn.start disasmer.spawn.join disasmer.task.yield_safepoint disasmer.task.current disasmer.fs.mount disasmer.fs.copy disasmer.fs.flush disasmer.fs.sync disasmer.artifact.from_path disasmer.artifact.open disasmer.blob.open disasmer.cmd.start disasmer.cmd.poll disasmer.cmd.kill disasmer.atomic.new_u32 disasmer.atomic.fetch_add_u32 disasmer.atomic.load_u32 disasmer.debug.probe disasmer.log.write disasmer.ui.emit_snapshot # scaffold only ``` ### 5.6 Native command API `cmd!(...)` should build a `CommandSpec` rather than immediately running a process. ```rust cmd!("cargo", "build", "--release") .cwd("/src") .env("RUSTFLAGS", "...") .timeout(Duration::from_secs(600)) .run() .await?; ``` The host checks: - current virtual thread has `cmd.run` capability; - selected environment supports native commands; - the task is running on a node, not the hosted coordinator; - process limits are configured; - filesystem mounts are inside the VFS/environment policy; - network policy is explicit; - logs are capped and streamed with backpressure. The hosted coordinator control plane never has this capability. An open-source/private coordinator may run with an embedded local node for trusted deployments; that node can have normal local command capabilities by default. Keep this as a compact policy/runtime module boundary rather than a separate node implementation, so dogfooding still uses the same node runtime code. ### 5.7 Compile-time environment references `env!("linux")` should validate against generated environment metadata. Implementation: 1. `disasmer build` scans `envs/**` and optional `disasmer.toml`. 2. It writes generated metadata to `target/disasmer/generated/envs.rs` and a JSON manifest. 3. The macro consults generated metadata during build. 4. VS Code reads the same metadata for autocomplete and diagnostics. Missing env error: ```text env!("windows") does not exist. Discovered environments: linux. Create envs/windows/Containerfile or add [envs.windows] to disasmer.toml. ``` --- ## 6. Bundle and build pipeline ### 6.1 Bundle concept A Disasmer bundle is a content-addressed virtual binary. It includes: ```text Wasm module(s) Disasmer manifest registered tasks debug metadata environment definitions selected environment context files source snapshot metadata selected VFS seed files capability declarations content hashes ``` It should not embed full container images. ### 6.2 Bundle layout ```text app.dis/ manifest/disasmer.json modules/main.wasm debug/main.dwarf debug/disasmer-debug.json envs/linux/Containerfile envs/windows/Containerfile envs/manifest.json source/source-snapshot.json source/dirty-overlay-manifest.json inputs/build-config.json vfs/seed-manifest.json capabilities.json ``` The on-disk `.dis` can initially be a directory during development and later become a zip/tar-like archive with content-addressed chunks. ### 6.3 Build command pipeline ```text disasmer build 1. discover project root 2. load disasmer.toml if present 3. discover envs/**/Containerfile and envs/**/Dockerfile 4. discover Nix flakes if plugin enabled 5. resolve source snapshot through enabled source-provider module, Git by default 6. compute dirty overlay manifest 7. generate env metadata for env!() 8. compile Rust guest to Wasm 9. run macro/task registration collection 10. collect debug metadata 11. build VFS seed manifest 12. compute bundle digest 13. write app.dis ``` ### 6.4 Digest strategy Bundle digest should include: ```text Wasm bytes task ABI descriptors environment recipe files environment context file hashes source snapshot provider identity and revision, such as a Git commit id dirty overlay file hashes selected input file hashes capability declarations debug metadata hash SDK ABI version ``` It should not include: ```text node-local cache paths absolute host checkout path local build timestamps unselected .git internals unselected target directory contents ``` ### 6.5 Compatibility hashes Task restart requires compatibility checks. Generate hashes for: ```text task public signature argument schema result schema capability declaration environment reference set VFS mount contract Disasmer ABI version ``` If only implementation changed and public schema is compatible, allow selected task restart. If schema changed, restart the whole virtual process. --- ## 7. Node runtime ### 7.1 Node responsibilities The node runtime is where real work happens. It must support: ```text registration with coordinator capability reporting heartbeat and health reporting bundle fetch or local bundle lookup Wasmtime task instantiation native command execution environment materialization/caching VFS overlay storage ephemeral artifact/blob cache source snapshot providers debug epoch handling logs/events streaming quota enforcement Quinn peer links ``` ### 7.2 Node state directories Linux default: ```text ~/.local/share/disasmer/node/ config.toml identity/ bundles/ env-cache/ vfs/ blobs/ artifacts/ logs/ tmp/ ``` Windows default: ```text %LOCALAPPDATA%\Disasmer\node\ config.toml identity\ bundles\ env-cache\ vfs\ blobs\ artifacts\ logs\ tmp\ ``` ### 7.3 Node capability model Node capabilities should be explicit and signed by node identity plus coordinator policy. Example: ```json { "node_id": "node_123", "tenant_id": "tenant_abc", "os": "linux", "arch": "x86_64", "capabilities": { "wasm": true, "cmd": true, "container.podman.rootless": true, "nix": true, "git.source_provider": true, "debug.suspend_process_tree": true }, "limits": { "max_tasks": 4, "max_log_bytes_per_task": 10485760, "max_arg_bytes": 4194304 } } ``` Windows dev backend example: ```json { "node_id": "node_win_123", "tenant_id": "tenant_abc", "os": "windows", "arch": "x86_64", "capabilities": { "wasm": true, "cmd": true, "windows.command_dev": true, "windows.sandbox_stub": true, "debug.job_object_suspend": true }, "security_note": "not a managed-untrusted sandbox backend" } ``` ### 7.4 Wasmtime task runner Each running virtual thread/task maps to: ```text Wasmtime Engine compiled module cache Store Linker with Disasmer imports task entrypoint dispatcher fuel/epoch/resource limits debug probe table VFS overlay handle handle table ``` `ThreadHostState` should contain: ```rust struct ThreadHostState { tenant_id: TenantId, project_id: ProjectId, virtual_process_id: VirtualProcessId, virtual_thread_id: VirtualThreadId, task_id: TaskId, bundle_digest: Digest, debug_epoch: Option, vfs_overlay: VfsOverlayHandle, handles: HandleTable, limits: ResourceLimits, coordinator: CoordinatorClient, node_store: NodeStore, command_runner: Arc, } ``` ### 7.5 Command runner The command runner must abstract platform differences. ```rust trait CommandRunner { async fn start(&self, spec: CommandSpec, ctx: CommandContext) -> Result; async fn poll(&self, handle: CommandHandle) -> Result; async fn suspend_for_debug(&self, handle: CommandHandle, epoch: DebugEpochId) -> Result<()>; async fn resume_from_debug(&self, handle: CommandHandle, epoch: DebugEpochId) -> Result<()>; async fn terminate(&self, handle: CommandHandle) -> Result<()>; } ``` Linux implementations: ```text PodmanRootlessCommandRunner LocalProcessGroupCommandRunner for development ``` Windows implementations: ```text WindowsCommandDevRunner WindowsSandboxRunnerStub ``` Do not offer `WindowsCommandDevRunner` as secure managed compute for untrusted public workloads. ### 7.6 Linux Podman backend Linux MVP should use rootless Podman. Responsibilities: ```text build image from Containerfile/Dockerfile cache image by environment digest mount /src, /work, /cache, /out or VFS paths run command with resource limits where practical capture stdout/stderr pause/unpause or suspend process tree for debug epoch publish outputs into VFS overlay ``` Default mounts: ```text /src source snapshot or bind mount /work task-local writable overlay /cache node-local dependency cache keyed by env/project /out artifact staging path ``` ### 7.7 Windows backend For the public-launch MVP, Windows should support an attached node that can run build commands in a controlled workspace. Minimum Windows backend: ```text create per-task temp workspace materialize source snapshot into workspace or bind local checkout when safe set environment variables from normalized env run command as non-admin service user where possible capture stdout/stderr track process tree with Job Object-like lifecycle management stage artifacts into VFS overlay support best-effort debug suspend/resume ``` Windows sandboxing should be behind an interface but not claimed as production isolation until implemented and tested. Potential future backend options: ```text Windows Sandbox CLI backend AppContainer / Less-Privileged AppContainer backend Hyper-V isolated runner backend Windows container backend, if project needs it ``` ### 7.8 Logs Logs are both a UX feature and an abuse vector. Node should: - stream recent log chunks to coordinator; - keep larger logs node-local unless `sync()` or retention policy requires upload; - redact configured secrets before sending; - enforce per-task and per-process log byte caps; - include task id, virtual thread id, sequence number, timestamp, and stream name. Log event: ```json { "tenant_id": "tenant_abc", "process_id": "vp_123", "thread_id": "vt_456", "task_id": "task_789", "stream": "stderr", "seq": 42, "bytes": "...", "truncated": false } ``` --- ## 8. Coordinator ### 8.1 Coordinator responsibilities The coordinator should provide: ```text auth/session handling; hosted OIDC through private module project registry node enrollment and identity node heartbeats node capability registry virtual process lifecycle task placement bundle registry debug session registry Debug Epoch coordination VFS manifest metadata for active processes artifact metadata for active processes hosted/community policy hooks and optional private quota ledger operator panel state/events scaffold rendezvous for node-to-node QUIC web dashboard CLI API VS Code/DAP API ``` It should not run arbitrary user commands. ### 8.2 Coordinator state split The coordinator needs two state layers in the MVP. Persistent Postgres state: ```text users / tenants / memberships if hosted module is enabled projects node enrollment records node public keys / credentials / revocations project-scoped source-provider configuration bundle registry entries worth reusing persistent environment metadata/cache hints hosted/community plan or policy records in the private module longer-lived audit records where needed ``` In-memory runtime state: ```text active virtual processes virtual threads pending task queues live node sessions and heartbeats scheduler placements Debug Epochs recent logs/events VFS manifest metadata for active processes artifact metadata and locations for active processes operator panel snapshots/events for active processes ``` For the MVP, if the coordinator restarts, active virtual processes restart or fail cleanly. Do not persist every process/thread/log/VFS row merely to survive coordinator restarts. Persistence is for product identity and project continuity, not durable runtime recovery. ### 8.3 Store interfaces and indicative persistent tables Define a persistent store trait for things that must survive restart: ```rust trait CoordinatorPersistentStore { async fn create_tenant(...); async fn create_project(...); async fn register_node_identity(...); async fn revoke_node_credential(...); async fn create_bundle(...); async fn save_environment_metadata(...); async fn save_source_provider_config(...); async fn authorize_persistent_object(...); } ``` Define a separate runtime-state trait that is in-memory for the MVP: ```rust trait CoordinatorRuntimeState { async fn create_virtual_process(...); async fn create_virtual_thread(...); async fn update_thread_state(...); async fn create_debug_epoch(...); async fn publish_vfs_manifest(...); async fn publish_artifact_metadata(...); async fn append_recent_log_event(...); async fn reserve_runtime_quota(...); } ``` Indicative persistent Postgres tables: ```sql users( id, authentik_subject nullable, email nullable, created_at, suspended_at ) tenants( id, slug, plan, created_at ) tenant_members( tenant_id, user_id, role ) projects( id, tenant_id, name, visibility, created_at ) nodes( id, tenant_id, project_id nullable, display_name, os, arch, trust_label, created_at, revoked_at ) node_capabilities_last_seen( node_id, capability, value, reported_at ) node_credentials( node_id, public_key, cert_fingerprint nullable, expires_at nullable, revoked_at ) bundles( id, tenant_id, project_id, digest, manifest_json, created_at ) environments( id, tenant_id, project_id, bundle_id nullable, name, kind, digest, requirements_json, recipe_manifest_json, created_at ) source_provider_configs( id, tenant_id, project_id, provider, config_json, created_at ) audit_events( id, tenant_id, project_id nullable, actor_id nullable, event_kind, object_id nullable, event_json, created_at ) ``` Runtime-only structures may look like the old tables conceptually, but they should be ordinary in-memory maps/queues first: `virtual_processes`, `virtual_threads`, `debug_epochs`, active `vfs_manifests`, artifact locations, recent logs, and operator panel snapshots. ### 8.4 Tenant safety invariant Every row that belongs to user content should include `tenant_id`. Every API should authorize by tenant and object relationship. Required tests: ```text user A cannot list user B nodes user A cannot attach to user B process user A cannot read user B logs user A cannot read user B artifact metadata for active processes user A cannot request debug memory from user B process node A cannot claim tenant B process node A cannot publish artifact for tenant B ``` ### 8.5 Node enrollment Flow: ```text 1. User opens coordinator UI or CLI. 2. User creates a node enrollment token for a project or tenant. 3. Token has short TTL and may include allowed capability constraints; node OS, architecture, and most capabilities are auto-detected at attach time. 4. Node CLI starts with token. 5. Node generates keypair. 6. Node calls enrollment endpoint. 7. Coordinator validates token and policy. 8. Coordinator creates node identity/certificate. 9. Node stores credential locally. 10. Node opens persistent session to coordinator. ``` Enrollment token example: ```json { "tenant_id": "tenant_abc", "project_id": "proj_123", "expires_at": "...", "allowed_capabilities": [ "wasm", "cmd", "container.podman.rootless" ], "auto_detect": ["os", "arch", "env", "source_providers"], "overrides_allowed": true } ``` ### 8.6 Scheduler The MVP scheduler should be simple, deterministic, and locality-aware. It should also be an easily replacible module in case the user wants different scheduling, or perhaps they have limitless resources (e.g. cloud backed) Inputs: ```text task environment requirements node capabilities node health source snapshot locations environment cache locations artifact locations node current load tenant/project policy quota availability debug affinity / restart affinity ``` Scoring: ```text score(node) = + environment compatible + source snapshot already local + environment digest cached + dependency cache likely warm + needed artifacts local + same node as previous failed attempt when restart is desired - current load - expected bytes to transfer - recent failures - policy warnings ``` Placement should prefer the local node if it can run the task and already has the repo. ### 8.7 State machine Virtual process states: ```text CREATED STARTING RUNNING FREEZE_REQUESTED FROZEN RESUMING COMPLETED FAILED CANCELLED ``` Virtual thread states: ```text PENDING PLACED STARTING RUNNING WAITING COMMAND_RUNNING FREEZE_REQUESTED FROZEN COMPLETED FAILED CANCELLED RESTARTING ``` Debug epoch states: ```text REQUESTED FREEZING FROZEN RESUMING RESUMED FAILED ``` --- ## 9. Transport and networking ### 9.1 Transport choices Use Quinn for QUIC in the MVP, behind a Rust abstraction. ```rust trait Transport { async fn connect_node(&self, endpoint: Endpoint, identity: NodeIdentity) -> Result; async fn accept_node(&self) -> Result; async fn open_stream(&self, kind: StreamKind) -> Result; async fn datagram(&self, msg: Datagram) -> Result<()>; } ``` The abstraction is important because Rust bindings and platform packaging may be rough. Keep QUIC-specific details in `disasmer-transport`. ### 9.2 Coordinator sessions The coordinator session carries: ```text node heartbeat capability updates task start/stop messages debug epoch messages VFS manifest publication artifact metadata for active processes publication log event stream rendezvous endpoint exchange quota updates ``` For development, it is acceptable to bootstrap auth over HTTPS and then use QUIC for persistent sessions. ### 9.3 Node-to-node sessions Node-to-node sessions carry: ```text blob chunk requests artifact reads source snapshot delta transfer VFS chunk transfer debug lazy memory/page reads where appropriate ``` They should not carry unauthenticated traffic. Every stream must include: ```text tenant id project id process id when applicable authorized object id fencing token / epoch where applicable request id quota accounting key ``` ### 9.4 Rendezvous without relay Coordinator-aided connection: ```text 1. Node A needs artifact from Node B. 2. A asks coordinator for artifact locations. 3. Coordinator authorizes A and returns candidate endpoints for B. 4. Coordinator notifies B to expect A. 5. A and B attempt direct QUIC connection. 6. If connection succeeds, data moves node-to-node. 7. If connection fails, scheduler avoids this transfer or asks user to sync/export through configured durable storage. ``` No silent bulk relay through the coordinator. ### 9.5 Failure handling If direct node-to-node connection fails: - do not fall back to coordinator bulk transfer by default; - mark the object location as unreachable from the requesting node; - choose a different placement if possible; - if impossible, fail with a clear action: ```text Artifact is on node windows-1, but linux-1 cannot establish direct QUIC. Run an artifact receiver node, configure an explicit external storage/export task, or run the dependent task on windows-1. ``` --- ## 10. Environment system ### 10.1 Discovery Scan default paths: ```text envs//Containerfile envs//Dockerfile envs//flake.nix envs//flake.lock ``` Normalize to: ```json { "name": "linux", "kind": "container", "digest": "sha256:...", "recipe": "envs/linux/Containerfile", "context": "envs/linux", "requires": { "os": "linux", "arch": "x86_64" }, "capabilities": { "cmd": true, "network": false } } ``` ### 10.2 Containerfile MVP Containerfile is the MVP path. Linux: ```text Containerfile -> Podman build -> local image id -> run task command ``` Windows: ```text Containerfile discovered and hashed Windows backend may treat it as a declared environment contract/stub actual command execution uses WindowsCommandDevRunner for attached user nodes ``` This is intentionally not marketed as secure Windows container isolation. ### 10.3 Nix plugin Nix should be present as a plugin because the architecture already expects pluggable environment resources, and Nix is useful for development reproducibility. MVP Nix support can be: ```text discovery of flake.nix and flake.lock hashing into environment digest optional local materialization on nodes with Nix installed cache location reporting not required for the flagship launch demo ``` ### 10.4 Environment materialization cache Cache key: ```text env digest + backend kind + platform + Disasmer env ABI version ``` Cache record: ```json { "env_digest": "sha256:...", "node_id": "node_123", "backend": "podman-rootless", "materialized_ref": "localhost/disasmer/env@sha256:...", "status": "ready", "last_used_at": "..." } ``` ### 10.5 Environment security policy Default task environment policy: ```text network disabled unless explicitly granted host filesystem inaccessible except declared VFS mounts secrets absent unless explicitly granted no privileged containers no Docker/Podman socket mount read-only source mount when possible write access only to /work and artifact staging logs capped process lifetime capped by policy ``` For user-attached nodes, Disasmer should still enforce its policy, but documentation must be honest that the node owner controls the machine and can bypass local restrictions. --- ## 11. VFS, artifacts, and latency ### 11.1 Core rule ```text Move metadata eagerly. Move bytes lazily. Run work where the bytes already are. ``` This should be treated as an engineering invariant, not a performance optimization later. ### 11.2 Mount model Default virtual filesystem: ```text /src or /vfs/project read-only source/input snapshot /work or /vfs/work task-local writable overlay /vfs/artifacts published artifact namespace /vfs/shared optional explicit shared namespace, probably not launch-critical ``` ### 11.3 Source snapshots and source-provider modules Source support should be pluggable. Git is the default included source-provider module, not a hard dependency of the VFS core. A source-provider module must be able to produce a `SourceSnapshot` manifest such as: ```text provider name and version source identity base revision or content root sub-resource state if applicable changed-file / dirty overlay manifest included untracked or generated inputs ignored-file policy ``` For Git specifically, the default module should model a source snapshot as: ```text repo identity commit hash submodule state dirty overlay manifest included untracked files ignored-file policy ``` Local node behavior: ```text if task runs on node with checkout: bind mount checkout read-only where safe materialize dirty overlay if needed else: fetch missing content through the enabled source provider or by content hash ``` Do not tar and upload the entire repo by default. Do not require users who use another source system to carry Git dependencies in their runtime. ### 11.4 Dirty overlay Dirty overlay manifest: ```json { "base_commit": "abc123", "files": [ { "path": "src/main.rs", "mode": "100644", "digest": "sha256:...", "size": 12345 } ], "deleted": ["old/file.rs"] } ``` Dirty bytes move only if the chosen node does not already have them. ### 11.5 VFS overlay commits Each task has a writable overlay. `flush()` creates a manifest delta: ```json { "process_id": "vp_123", "thread_id": "vt_456", "parent_epoch": 4, "epoch": 5, "created": [ { "path": "/vfs/artifacts/linux-x64/app", "digest": "sha256:...", "size": 9912321, "locations": ["node_linux_1"] } ], "modified": [], "deleted": [] } ``` The coordinator stores the manifest and locations. It does not receive the bytes unless a configured durability/export policy requires it. ### 11.6 `flush()` semantics `flush()` means: ```text close current task write epoch hash changed files/chunks publish VFS manifest delta publish artifact metadata for /vfs/artifacts changes make changes visible to downstream tasks according to VFS consistency rules return after metadata is accepted by coordinator ``` `flush()` should not: ```text upload large artifacts by default replicate to other nodes by default export to local dist by default route bytes through coordinator ``` ### 11.7 `sync()` and explicit artifact movement semantics `sync()` means: ```text apply an explicit export/transfer/durability policy chosen by the program or CLI verify referenced bytes still exist on at least one node or explicit store move bytes directly between nodes or to a user-configured external destination update artifact metadata for active processes with the result ``` `sync()` should not imply a default coordinator artifact store. By default, artifacts and other large data are nearly ephemeral: they live on nodes for as long as a task, downstream consumer, export, restart, or configured cache/GC policy needs them. They may linger long enough for practical task restart, but unsynced node-local artifacts are not a durability guarantee. Example policies: ```rust // Ask an attached/local receiver node to pull bytes into ./dist. artifact::export_to_node(dist, receiver::local("dist")).await?; // Send bytes to a user-provided storage integration implemented by project code. artifact::copy_to(outputs, storage::s3("release-bucket")).await?; // Keep artifacts available on the producing node long enough for likely restarts. artifact::retain(outputs, RetainPolicy::BestEffort { min_ttl: Duration::from_hours(24) }).await?; ``` MVP should implement at least: ```text metadata-only flush node-local ephemeral artifact/blob cache explicit export to an attached/local receiver node best-effort retention/GC knobs clear errors when bytes have been garbage-collected or are unreachable ``` Users who want artifacts on their own machine should either run a receiver node there or write/use an explicit storage/export integration. The coordinator should keep artifact metadata for active processes, not act as the default artifact data store. ### 11.8 Artifact lifecycle Artifact states: ```text PUBLISHED_METADATA AVAILABLE_ON_NODE RETAINED_BEST_EFFORT EXPORTED DURABLE_EXTERNAL EXPIRED DELETED ``` To allow producing artifacts without much effort from the user, the control pane could provide a download button through the immediate-mode UI, which will stream the (best-effort retained) artefact from the node that retains it. "Best effort" and usage tier limits apply as normal, but it should be an error to create a download link to an artifact that can't be downloaded within the limits in the first place. Download links should be secure and not just allow anyone that guesses the correct link to download any artifact. Artifact metadata: ```json { "artifact_id": "art_123", "path": "/vfs/artifacts/linux-x64/app", "digest": "sha256:...", "size": 9912321, "producer_thread_id": "vt_linux", "locations": [ { "kind": "node", "node_id": "node_linux_1" } ], "durability_state": "PUBLISHED_METADATA", "retention": "best_effort_node_gc" } ``` ### 11.9 Local-first acceptance test For a local Linux build from an existing Git checkout: - no full-repo tarball is created; - no source bytes are uploaded to coordinator; - compiler file reads are local; - build cache reads are local; - artifact bytes remain local after `flush()`; - coordinator receives only metadata, logs within caps, and debug/session events; - explicit export/transfer code is the first operation allowed to move final artifact bytes for durability or user-visible export. --- ## 12. Task execution lifecycle ### 12.1 Spawn flow ```text 1. Guest calls spawn::task(compile).arg(...).env(env!("linux")).start().await. 2. Wasm guest invokes disasmer.spawn.start host import. 3. Node host validates current task capability. 4. Host serializes TaskSpec with task id, env id, args, handles, VFS base epoch. 5. Coordinator creates virtual_thread row in PENDING state. 6. Scheduler chooses node. 7. Coordinator sends StartTask to chosen node. 8. Node fetches or locates bundle. 9. Node materializes environment if needed. 10. Node creates VFS overlay. 11. Node instantiates Wasmtime module or command wrapper. 12. Node calls named task entrypoint. 13. Thread becomes visible in debugger and coordinator UI. ``` ### 12.2 Join flow ```text 1. Parent awaits TaskHandle.join(). 2. Parent task parks or yields. 3. Child completes with serialized result or error. 4. Result may contain handles such as Artifact. 5. Coordinator records completion. 6. Parent is woken. 7. Parent receives decoded result. ``` ### 12.3 Command flow inside task ```text 1. Guest task calls cmd!(...).run().await. 2. Host import validates command capability. 3. Node command runner starts process in task environment. 4. Task enters COMMAND_RUNNING state. 5. Logs stream to node and coordinator under caps. 6. Debug Epoch requests suspend controlled process tree. 7. Command exit status is returned to guest. 8. Guest decides whether to publish artifacts. ``` ### 12.4 Failure behavior MVP rules: ```text Task trap -> task FAILED, process may enter debug break depending policy. Command nonzero exit -> task receives error, can handle or fail. Node lost with non-replicated running state -> virtual process FAILED. Node lost after flushed but unsynced artifact -> artifact may be unavailable. Node lost after synced artifact -> artifact survives according to sync policy. ``` Do not pretend non-replicated state is recoverable. ### 12.5 Restart flow ```text 1. User edits source. 2. VS Code triggers disasmer build. 3. New bundle is produced. 4. User selects Restart Failed Task or Restart Selected Virtual Thread. 5. DAP/coordinator checks task compatibility hash. 6. Runtime stops old task if needed. 7. Unflushed overlay changes are discarded unless explicitly preserved. 8. New task starts with original args, same env, chosen VFS checkpoint. 9. Breakpoints are rebound. 10. Other tasks remain alive only if compatibility checks pass. ``` If compatibility fails: ```text Task signature changed; restarting whole virtual process is required. ``` This capability is important to advertise in any demo(s). --- ## 13. Debugging ### 13.1 Design goal The user should experience Disasmer as one ordinary debug target: ```text Process: launch-build-demo Threads: 1 main 2 compile linux-x64 3 compile windows-x64 4 package ``` The debugger should not force the user to understand node IDs, QUIC links, VFS manifests, or environment cache internals unless they open the Disasmer inspector. ### 13.2 Use a custom DAP adapter Use `disasmer-dap` as the VS Code debug adapter. Why: - DAP is the right integration point for VS Code. - The target is not one native process; it is a virtual process spanning nodes. - Disasmer must control the thread list, stop reasons, restart commands, artifact views, and all-stop semantics. - LLDB can be a future internal helper, but it should not own the UX. ### 13.3 Debug components ```text VS Code extension launches disasmer-dap and contributes commands/views disasmer-dap speaks DAP to VS Code speaks Disasmer debug protocol to coordinator maps DAP process/thread/stack/variables to virtual model coordinator debug service owns debug sessions and Debug Epochs broadcasts freeze/resume aggregates thread state node debug agent sets breakpoints/probes freezes/resumes Wasmtime tasks suspends/resumes controlled command processes captures stacks/locals/handle state ``` ### 13.4 Debug Epoch protocol When a breakpoint is hit: ```text 1. Node detects breakpoint/trap/probe in virtual thread T. 2. Node reports BreakHit(process, thread, location) to coordinator. 3. Coordinator creates DebugEpoch E. 4. Coordinator marks process FREEZE_REQUESTED. 5. Coordinator sends Freeze(E) to all nodes hosting threads for the process. 6. Wasm tasks stop at current probe/safepoint or next safepoint. 7. Native command tasks are suspended using backend-specific process-tree control. 8. VFS overlay manifests are pinned. 9. Memory/handle snapshots are pinned copy-on-write where applicable. 10. Nodes report Frozen(E, thread states). 11. Coordinator marks epoch FROZEN. 12. DAP emits stopped event for all threads. ``` Resume: ```text 1. User presses Continue. 2. DAP sends Continue(E) to coordinator. 3. Coordinator sends Resume(E) to nodes. 4. Nodes unpin debug snapshots. 5. Wasm tasks continue. 6. Native command tasks resume. 7. Coordinator marks epoch RESUMED. ``` ### 13.5 Safepoints and breakpoints MVP should use layered breakpoint support: #### Layer 1: explicit debug probes `#[disasmer::task]` and `#[disasmer::main]` generate probes at: ```text task entry before/after spawn before/after join before/after cmd run before/after flush/sync await boundaries explicit disasmer::debug::probe!() calls ``` This provides reliable all-stop behavior early. #### Layer 2: source-line breakpoints The build pipeline maps source line breakpoints to probe points and, where possible, Wasm code offsets using debug metadata. Launch acceptance should include: ```text breakpoint in main() breakpoint in compile() breakpoint before/after cmd!().run() breakpoint before artifact publish/flush breakpoint in package() ``` #### Layer 3: richer DWARF/LLDB integration After the DAP skeleton works, improve locals/stacks by reading Rust/Wasm debug metadata. This is important, but it should not block the all-stop architecture. ### 13.6 Variables and locals Launchable debugger variable support should include: ```text task arguments return values Target enums and simple structs Artifact handles SourceSnapshot handles Blob handles DAtomic values DChannel status current VFS mounts current command spec/status recent stdout/stderr tail ``` For arbitrary Rust locals, use best-effort DWARF support where available. If a local cannot be inspected, the debugger should say so clearly and still show task args/handles. ### 13.7 Native command tasks in debugger A native command is debugger-visible as part of the virtual thread, but MVP does not need to debug inside `cargo` or the compiler itself. The command frame should show: ```text command argv working directory environment name node id in inspector only start time status exit code when done recent stdout/stderr resource limits ``` When the virtual process is frozen, controlled native command processes should be suspended. If the backend cannot safely suspend a command, the node must report failure and the debugger must show a clear error rather than claiming all-stop succeeded. ### 13.8 DAP features required for launch Required: ```text initialize launch / attach setBreakpoints configurationDone threads stackTrace scopes variables continue pause next / step-over at probe granularity disconnect stopped events continued events output events custom restart-task request ``` Not required for launch: ```text watchpoints reverse debugging non-stop debugging hot code replace debugging arbitrary child process internals ``` ### 13.9 Disasmer inspector Default debugger view remains normal. Add an optional inspector tree: ```text Virtual Process Threads compile linux-x64 -> node linux-1 -> env linux -> running/frozen compile windows-x64 -> node windows-1 -> env windows -> running/frozen VFS epoch 4 artifacts Debug Epoch id E frozen nodes Environments linux digest cached on linux-1 windows digest cached/stub on windows-1 Transport node links status ``` This helps explain the system without polluting the normal debug UX. --- ## 14. VS Code extension ### 14.1 Extension responsibilities The extension should: ```text watch envs/**/Containerfile watch envs/**/Dockerfile watch envs/**/flake.nix watch envs/**/flake.lock watch inputs/** watch src/** run disasmer build before debugging launch/attach DAP session show virtual threads in normal debug UI show per-task logs show artifact tree show node connection status provide env!("...") autocomplete provide disasmer.toml diagnostics provide restart selected/failed task commands ``` ### 14.2 Launch configuration Example `.vscode/launch.json`: ```json { "version": "0.2.0", "configurations": [ { "name": "Debug Disasmer Build", "type": "disasmer", "request": "launch", "coordinator": "https://disasmer.example.com", "project": "${workspaceFolder}", "preLaunchTask": "disasmer: build", "localFirst": true } ] } ``` ### 14.3 Editor diagnostics Diagnostics: ```text env!("name") missing env discovered but incompatible with available nodes Containerfile changed; bundle rebuild needed large task argument warning cmd! used in task without cmd capability fs::sync missing for final artifact export warning Windows backend is dev/stub isolation warning ``` ### 14.4 VS Code views MVP views: ```text Disasmer: Nodes Disasmer: Virtual Processes Disasmer: Artifacts Disasmer: Logs Disasmer: Inspector ``` Operator panel preview can appear as: ```text Disasmer: Operator Panel Preview ``` But the full immediate-mode panel is second milestone. --- ## 15. Hosted coordinator and product UX ### 15.1 Hosted coordinator MVP Hosted coordinator pages: ```text Sign in Projects Project detail Node attach instructions Node list Virtual process list Virtual process detail Logs Artifacts Debug session status Operator panel placeholder/preview Settings / quotas ``` ### 15.2 Hosted Authentik/OIDC integration Use Authentik as the identity provider through OIDC in the private hosted module. The open-source/private coordinator should expose auth hooks and simpler local/team auth, but should not require the hosted OIDC stack. Implementation requirements: ```text authorization code + PKCE for web login strict redirect URI allowlist minimal scopes short session lifetime refresh handling if needed logout/back-channel support later CLI token issuance after web login node enrollment tokens separate from user sessions ``` ### 15.3 Community tier Community tier defaults: ```text 1 active virtual process tiny hosted Wasm control loop limit no hosted native commands no hosted containers no hosted network from tasks small logs small metadata small artifact retention 4 user-attached nodes allowed operator panel access ``` The hosted control loop is useful for trying Disasmer and coordinating attached nodes, not for mining, scraping, compiling huge repos, or hosting files. ### 15.4 Abuse controls as architecture Every resource-consuming action must pass through a hosted/community quota policy module before work starts. Quota dimensions: ```text active virtual processes hosted Wasm fuel hosted Wasm memory hosted process wall-clock API calls node heartbeats spawn attempts log bytes metadata bytes artifact bytes artifact downloads debug memory reads operator panel updates rendezvous attempts ``` Pattern: ```rust quota.reserve(tenant, Resource::SpawnAttempt, 1).await?; // perform operation quota.commit(reservation).await?; ``` For streams, use incremental accounting and backpressure. ### 15.5 Public input threat model Treat as hostile: ```text users OAuth identities nodes programs build files Containerfiles Git repos logs artifacts operator panel text/events debug requests node capabilities cache claims ``` ### 15.6 Public launch security baseline Required before public launch: - Tenant ID on every object. - Authorization check on every API endpoint. - Negative cross-tenant tests. - No hosted containers on community tier. - No hosted native commands on community tier. - No arbitrary network access from hosted Wasm. - Rate limits on auth, APIs, rendezvous, logs, spawn, and UI events. - Log and artifact size caps. - Private artifacts by default. - No public file hosting by default. - Node identity signing/revocation. - Task capabilities explicit. - Debug attach requires project permission. - Debug memory reads quota-limited and audited. - Operator panel uses built-in widgets only. - No custom HTML or JS in panels. - No password fields in user-defined panels. - No OAuth flows inside operator panels. --- ## 16. Operator panel scaffold and second milestone ### 16.1 MVP scaffold Even if the full panel is second milestone, the MVP should include: ```text ui model crate panel snapshot table panel event table coordinator API namespace web placeholder area DAP/debug interaction policy security restrictions ``` The first visible version can be a fixed coordinator dashboard showing: ```text process state tasks progress logs artifacts Debug / Restart / Cancel buttons as control-plane actions ``` ### 16.2 Immediate-mode second milestone The second milestone adds: ```rust let panel = ui::panel("Repo builder").await?; loop { let action = panel.draw(|ui| { ui.heading("Repo builder"); ui.text("Build status"); ui.task_table("Tasks", tasks.snapshot()); ui.log_tail("Recent output", logs.tail()); if ui.button("Restart failed").clicked() { ui.emit(Action::RestartFailed); } }).await?; if let Some(action) = action { handle(action).await?; } } ``` ### 16.3 Panel security model MVP/second milestone rules: ```text built-in widgets only no custom HTML no custom JavaScript all text escaped external links restricted no OAuth/password widgets panel labeled as user-program-provided rate-limited updates state size capped events typed and schema-checked read-only while debugger has process frozen control-plane actions may remain available while frozen ``` --- ## 17. Security and capability model ### 17.1 Capability-first execution Every task has a capability set. Example: ```json { "task": "compile", "capabilities": { "vfs.read": ["/src"], "vfs.write": ["/work", "/vfs/artifacts/linux-x64"], "cmd.run": true, "network.outbound": false, "secrets": [] } } ``` ### 17.2 Coordinator authz checks Every coordinator API checks: ```text authenticated principal principal tenant membership object tenant id project membership/role operation permission quota availability object state compatibility ``` ### 17.3 Node trust model Default rule: ```text A user's work runs only on that user's nodes, managed paid nodes, or explicitly shared team nodes. ``` For MVP: - do not mix tenants on user-provided nodes; - do not trust user-provided nodes to produce truthful artifacts for other tenants; - label artifacts with node provenance; - expose trust label in inspector; - eventually support reproducible repeated builds for higher trust. ### 17.4 Bundle identity Bundles should be content-addressed and integrity-checked. Later signing can be added, but MVP should at least ensure: ```text coordinator records bundle digest node verifies bundle digest before running manifest records environment/resource digests task specs reference bundle digest artifact provenance references bundle/task/node ``` ### 17.5 Secrets MVP secrets support should be minimal. Recommended launch stance: ```text no secrets in hosted/community tasks secrets not shown in logs/UI/debug by default explicit grant required per task/environment short-lived scoped secret materialization secret access audited secret redaction in logs ``` For the first launch demo, avoid secrets entirely. ### 17.6 Container/runtime hardening Linux Podman backend should default to: ```text rootless no privileged containers no host Docker/Podman socket mount read-only source mount writable task workdir only limited network by default seccomp/AppArmor/SELinux where available resource limits where practical ``` Windows dev backend should be clearly scoped as user-attached-node execution, not public managed secure isolation. --- ## 18. Implementation milestones This sequence avoids building a beautiful UI on top of a runtime that cannot yet prove the thesis. ### Milestone 0: Repository and protocol spine Deliverables: ```text monorepo layout shared IDs and protocol types CoordinatorStore trait Postgres persistent-store migrations skeleton CLI skeleton node skeleton SDK skeleton VS Code extension skeleton DAP skeleton example project skeleton ``` Acceptance criteria: ```text cargo workspace builds CLI can print discovered project info coordinator starts locally with Postgres for persistent project/node state and in-memory runtime state node starts and registers fake heartbeat locally VS Code extension can run a dummy DAP session ``` Key considerations: - Do not overdesign protocol versioning, but add a protocol version field immediately. - Use stable IDs everywhere. - Add tenant_id to data models from the first migration. ### Milestone 1: Local single-node Wasm task execution Deliverables: ```text #[disasmer::main] #[disasmer::task] DisasmerArg derive named task registry Wasmtime runner spawn/join on same node small serialized args/results basic logs basic DAtomicU32 or equivalent demo handle ``` Acceptance criteria: ```text example local-one-node runs main task main spawns child Wasm task child returns serializable result parent joins result logs show per virtual thread ``` Considerations: - Keep the first task ABI tiny. - Ban closures for MVP. - Fail early on invalid args. ### Milestone 2: Environment discovery and bundle generation Deliverables: ```text envs/** discovery env!("name") compile/editor validation disasmer build creates app.dis directory bundle manifest task manifest environment manifest source snapshot manifest placeholder VFS seed manifest ``` Acceptance criteria: ```text adding envs/linux/Containerfile makes env!("linux") valid missing env gives useful error bundle digest changes when Containerfile changes bundle digest changes when task ABI changes ``` Considerations: - Keep `disasmer.toml` optional. - Make generated metadata visible and inspectable. ### Milestone 3: VFS and artifacts Deliverables: ```text per-task VFS overlay /vfs/project or /src mount /vfs/work /vfs/artifacts fs::copy fs::flush metadata publication Artifact::from_path fs::sync export-to-local-directory content-addressed blob/chunk store ``` Acceptance criteria: ```text task writes artifact flush publishes artifact metadata for active processes parent joins Artifact handle package task reads artifact handle sync exports final artifact to dist/ large artifact bytes do not go through coordinator in local mode ``` Considerations: - Build the metadata/byte split now. - Ensure artifact path namespace is stable. - Add tests proving `flush()` is metadata-only. ### Milestone 4: Linux Podman command backend Deliverables: ```text Podman rootless env materialization Containerfile build/cache by digest cmd!(...).run() stdout/stderr streaming process limits/log caps artifact staging from command output basic process suspend/resume for debug epoch ``` Acceptance criteria: ```text Linux task runs cargo build in envs/linux/Containerfile logs stream to CLI/coordinator artifact is published through VFS local source checkout is bind-mounted when task runs locally ``` Considerations: - Keep the first Containerfile expectations simple. - Do not support arbitrary privileged containers. - Do not mount the container engine socket. ### Milestone 5: Coordinator/node split with hosted-ready persistent store Deliverables: ```text coordinator service Postgres-backed persistent store node enrollment token node identity node heartbeat bundle registration task placement StartTask command over persistent session logs/events over stream local coordinator mode still works ``` Acceptance criteria: ```text coordinator runs separately from node node attaches with token disasmer run schedules task onto attached node node executes task and reports completion Postgres contains persistent project/node/bundle metadata; active process/thread/artifact metadata lives in memory ``` Considerations: - Use same code path for local and hosted as much as possible. - Do not create a special “demo scheduler” that will be thrown away. ### Milestone 6: Quinn transport and rendezvous Deliverables: ```text Quinn transport wrapper node-to-coordinator persistent session if practical node-to-node direct connection rendezvous endpoint exchange authenticated stream framing blob/artifact chunk transfer protocol no coordinator bulk relay ``` Acceptance criteria: ```text two nodes establish direct QUIC link node A can request authorized artifact chunk from node B failed direct connection produces clear error coordinator never receives bulk artifact bytes by default ``` Considerations: - Keep HTTPS bootstrap if it makes auth simpler. - Put transport behind traits for testability. - Add trace logs for failed connectivity. ### Milestone 7: Windows attached node backend Deliverables: ```text Windows node build/test WindowsCommandDevRunner per-task workspace process/log capture artifact staging best-effort process tree lifecycle Windows sandbox backend trait + stub Forgejo Windows worker setup docs ``` Acceptance criteria: ```text Windows node attaches to hosted/local coordinator scheduler places env!("windows") task on Windows node Windows task runs cargo build or a simple command artifact is published and joined by parent VS Code/coordinator shows Windows virtual thread ``` Considerations: - Be explicit that this is not production-grade untrusted Windows sandboxing. - Do not offer managed public Windows compute until isolation is real. ### Milestone 8: All-stop debugging and DAP Deliverables: ```text custom DAP adapter launch/attach thread list breakpoint setting source/probe mapping Debug Epoch coordinator node freeze/resume Wasm safepoints/probes command process suspend/resume integration stack/scopes/variables for task args and handles stdout/stderr output events restart selected/failed task command ``` Acceptance criteria: ```text VS Code F5 starts a virtual process breakpoint in main stops all virtual threads breakpoint in Linux compile task stops Windows task too breakpoint in Windows task stops Linux task too thread list is coherent user can inspect Target arg and Artifact handle continue resumes all tasks task restart works after source edit ``` Considerations: - This is the riskiest milestone. Start early. - Use probe-granularity stepping first. - Make all-stop correctness more important than perfect DWARF locals. ### Milestone 9: Local-first Git/source snapshots Deliverables: ```text git source provider module commit/submodule/dirty overlay manifest local checkout detection bind-mount local checkout for local tasks remote missing-content transfer by hash scheduler locality scoring latency regression tests ``` Acceptance criteria: ```text local Linux build from local checkout does not upload repo remote Windows build receives only required source content changing one file transfers only that file/overlay where possible scheduler prefers node with source checkout and env cache ``` Considerations: - Keep source provider modular. - Do not hardwire Git into the VFS core. ### Milestone 10: Hosted coordinator launch hardening Deliverables: ```text private hosted Authentik/OIDC login tenant/project UI node attach UI hosted/community quota policy module authz middleware and private hosted OIDC integration cross-tenant negative tests community tier limits rate limiting log/artifact caps admin kill switch basic plan/policy flags, even if only community exists ``` Acceptance criteria: ```text public user can sign in public user can create project public user can attach own node public user cannot run hosted native command/container public user can run task on own node cross-tenant tests pass quota tests pass ``` Considerations: - Do not launch without abuse controls. - Design quotas as first-class resource accounting. ### Milestone 11: Launch demo polish Deliverables: ```text launch-build example README quickstart short screencast script hosted landing page install script/packages VS Code extension packaging error message pass first-run diagnostics artifact export UX fixed coordinator dashboard operator panel preview scaffold ``` Acceptance criteria: ```text fresh developer can follow quickstart Linux node attach works Windows node attach works F5 demo works breakpoint demo works restart failed task demo works artifact export works README explains community tier honestly ``` Considerations: - Public launch users will try weird setups immediately. - Error messages matter as much as features. - Make the first demo resilient. --- ## 19. Testing strategy ### 19.1 Unit tests Required areas: ```text DisasmerArg derive allowed/rejected types environment discovery bundle digest stability manifest generation VFS path normalization flush manifest generation sync policy selection hosted/community quota policy module scheduler scoring authz checks protocol encoding/decoding ``` ### 19.2 Integration tests Local integration tests: ```text single-node spawn/join two Wasm tasks with DAtomic task writes artifact and parent consumes it cmd runner fake backend Debug Epoch with two Wasm tasks restart failed task with compatible bundle restart fails with incompatible schema ``` Linux integration tests: ```text Podman build env Podman run command local checkout bind mount artifact publish/export process suspend/resume if supported ``` Windows integration tests: ```text node attach command execution log capture artifact staging process lifecycle cleanup ``` Network integration tests: ```text node-to-node QUIC connect artifact chunk transfer connection failure clear error no coordinator bulk relay ``` ### 19.3 End-to-end tests E2E test scenarios: ```text F5 local single node hosted coordinator + attached Linux node hosted coordinator + Linux + Windows nodes breakpoint in Linux task all-stops process breakpoint in Windows task all-stops process restart failed Windows task local-first no-upload test community tier quota enforcement cross-tenant forbidden access ``` ### 19.4 Latency regression tests Add tests that fail if: ```text local source bytes are uploaded during local build artifact bytes go to coordinator during flush full repo tarball is created by default large artifact is transferred before consumer/export/sync needs it scheduler chooses remote node despite local compatible warm node without reason ``` ### 19.5 Security tests Required negative tests: ```text user cannot read another tenant's process user cannot attach debugger to another tenant's process node cannot claim another tenant's task node cannot publish artifact for unauthorized process community hosted task cannot request cmd.run community hosted task cannot request network operator panel cannot submit custom HTML/JS log cap truncates noisy task quota rejects spawn storm ``` --- ## 20. Documentation required for MVP ### 20.1 User docs ```text README quickstart What is Disasmer? Install CLI Install VS Code extension Login to hosted coordinator Attach a Linux node Attach a Windows node Create envs//Containerfile Write first build program Debug in VS Code Publish artifacts with flush() Export artifacts through an attached receiver node or explicit storage integration Community tier limits Security model for user-attached nodes ``` ### 20.2 Engineering docs ```text architecture.md protocol.md task-abi.md vfs.md debugging.md transport.md security.md operator-panel.md windows-backend.md ``` ### 20.3 Error messages as docs Invest heavily in errors: ```text missing env node unavailable environment incompatible large task arg host-only type in DisasmerArg cmd used on coordinator cmd capability missing artifact bytes unavailable QUIC direct connection failed sync required for export Debug Epoch freeze failed restart incompatible after source change community tier quota exceeded ``` --- ## 21. Key risks and mitigations ### 21.1 Risk: debugger is the hardest part Mitigation: - Build DAP skeleton early. - Implement Debug Epochs before fancy UI. - Use generated debug probes for reliable breakpoints first. - Add DWARF/local richness incrementally. - Define launch acceptance around real breakpoints in the demo code, thread list, task args, handles, stacks, all-stop, continue, and restart. Experiment result: `experiments/debugger-gate` has derisked the core VS Code debugging UX. It proves a custom DAP adapter can present a virtual process with `main` and task threads, bind ordinary VS Code gutter breakpoints to Wasmtime-backed Rust/Wasm source locations, hit breakpoints in multiple virtual tasks, support basic stepping/watch/variables, and keep the backend boundary shaped like a replaceable virtual-process runtime. Treat that experiment as a proof and reference, not as implementation foundation: it is intentionally hacky prototype code with hardcoded example-local variable mapping and a simplified local Wasmtime scheduler. ### 21.2 Risk: Windows isolation is not real yet Mitigation: - Keep Windows as user-attached node execution only. - Label backend as `windows-command-dev`. - Do not offer public managed Windows compute in MVP. - Keep sandbox backend interface ready. - Investigate Windows Sandbox/AppContainer after public launch or before managed nodes. ### 21.3 Risk: hosted community tier becomes compute abuse target Mitigation: - No community hosted containers. - No community hosted native commands. - Zero-cap hosted Wasm control loop only. - Fuel, wall-clock, memory, logs, and metadata limits. - Rate-limit every API and event stream. - Require attached user node for real work. ### 21.4 Risk: QUIC/NAT becomes launch blocker Mitigation: - Use coordinator control session for scheduling and metadata. - Require direct node-to-node only when bytes must move. - Prefer scheduling where bytes already are. - Fail clearly when direct transfer is impossible. - For launch demo, place Linux and Windows nodes in a connectivity setup you control. ### 21.5 Risk: local-first goal gets lost Mitigation: - Add tests that fail on unwanted uploads. - Instrument bytes moved per build. - Show “bytes moved” in logs/dashboard. - Make `flush()` metadata-only by default from day one. ### 21.6 Risk: product sounds too ambitious Mitigation: - Demo one narrow workflow extremely well. - Keep the public README focused on build systems. - Keep architecture docs honest about non-goals. - Do not claim managed secure compute until it exists. - Emphasize open-source local-first runtime. --- ## 22. Definition of done for the public-launch MVP The public-launch MVP is done when a clean machine can: ```text 1. Install Disasmer CLI. 2. Install the VS Code extension. 3. Sign in to hosted coordinator through Authentik/OIDC. 4. Create or select a project. 5. Attach a Linux node. 6. Attach a Windows node. 7. Clone the launch-build demo repo. 8. Open it in VS Code. 9. See env!("linux") and env!("windows") recognized. 10. Press F5. 11. See one virtual process start. 12. See Linux and Windows virtual threads in VS Code. 13. Hit a breakpoint in a spawned task. 14. Observe all-stop Debug Epoch behavior. 15. Inspect task args, command status, logs, and artifact handles. 16. Edit source and restart only the failed/selected task. 17. Finish the build. 18. Export artifacts through an attached/local receiver node or explicit storage integration. 19. Confirm local-first behavior in logs/metrics. 20. Confirm hosted coordinator did not run arbitrary native commands. ``` --- ## 23. Suggested implementation order inside each milestone When in doubt, build vertical slices rather than broad infrastructure. Recommended pattern: ```text 1. Fake backend, real API. 2. Real backend, local only. 3. Real backend, coordinator split. 4. Real backend, hosted auth/quotas. 5. Polish only after the vertical slice proves the thesis. ``` Example for debugging: ```text 1. DAP shows fake virtual process with fake threads. 2. DAP attaches to local one-node Wasm process. 3. Breakpoint probe freezes one Wasm task. 4. Debug Epoch freezes two Wasm tasks. 5. Debug Epoch freezes Wasm + command task. 6. Same flow across two nodes. 7. VS Code restart selected task. 8. Richer locals/stacks. ``` Example for VFS: ```text 1. In-memory manifest. 2. Local disk overlay. 3. Artifact handle. 4. Metadata-only flush. 5. Export sync. 6. Node-to-node chunk fetch. 7. Explicit receiver-node export. ``` --- ## 24. External technical references to keep nearby These are useful implementation references, not product dependencies: - Wasmtime Rust embedding API: https://docs.wasmtime.dev/api/wasmtime/ - Wasmtime Component Model API: https://docs.wasmtime.dev/api/wasmtime/component/index.html - Quinn repository and docs: https://github.com/quinn-rs/quinn - Debug Adapter Protocol: https://microsoft.github.io/debug-adapter-protocol/ - VS Code debugger extension guide: https://code.visualstudio.com/api/extension-guides/debugger-extension - Authentik OAuth2/OIDC provider docs: https://docs.goauthentik.io/add-secure-apps/providers/oauth2/ - Podman: https://podman.io/ - Nix flakes: https://nix.dev/concepts/flakes.html - Windows Sandbox CLI: https://learn.microsoft.com/en-us/windows/security/application-security/application-isolation/windows-sandbox/windows-sandbox-cli - Windows AppContainer isolation: https://learn.microsoft.com/en-us/windows/win32/secauthz/appcontainer-isolation - OWASP API Security Top 10 2023, unrestricted resource consumption: https://owasp.org/API-Security/editions/2023/en/0xa4-unrestricted-resource-consumption/ - OAuth 2.0 Security Best Current Practice, RFC 9700: https://datatracker.ietf.org/doc/rfc9700/ - NIST SP 800-190, Application Container Security Guide: https://csrc.nist.gov/pubs/sp/800/190/final --- ## 25. Final guidance Do not shrink the ambition of the demo, but aggressively constrain the implementation semantics. The MVP should be ambitious in what the user sees: ```text one Rust build program hosted coordinator Linux + Windows nodes source-level VS Code debugging all-stop Debug Epochs task restart local-first artifacts ``` It should be conservative in what the runtime promises internally: ```text no transparent raw remote pointers no live stack migration no coordinator bulk data plane no community hosted native compute no fake Windows sandbox claims no arbitrary child-process source debugging no automatic survival of non-replicated node failure ``` That combination is what can make the launch credible: the visible experience is striking, while the implementation model remains honest enough to actually build.