clusterflux-public/disasmer_spec.md

Disasmer Specification

Status: draft design spec
Name: Disasmer, short for distributed Wasm. The name intentionally sounds a bit like Wasmer and a bit like “disaster.”
Primary goal: make distributed execution feel like normal multithreaded programming and normal source-level debugging.

Disasmer is a distributed WebAssembly runtime that lets multiple real machines behave like one logical machine. A program runs as one Virtual Process with many Virtual Threads. Each virtual thread runs on exactly one real machine at a time, but different virtual threads, memory spaces, filesystems, and build tasks may live on different machines.

The core promise:

Developers debug Disasmer programs like ordinary multithreaded programs.
The runtime handles the distributed reality underneath.

Disasmer is also intended to support a build-system use case: one build is a virtual process, and build steps are virtual threads/tasks that may run in containers or Nix-defined environments on different operating systems and architectures.

---

1. Goals

Disasmer should provide:

distributed Wasm execution
normal VS Code debugging
virtual threads across machines
distributed/sharded memory spaces
cluster-wide suspend/resume
per-thread virtual filesystems
container/Nix-backed execution environments
single-source-file distributed builds
explicit artifact publication
low-latency reliable control-plane coordination

The debugger should see:

one process
many threads
normal stacks
normal locals
normal source files
normal breakpoints
normal memory inspection

The optional Disasmer inspector may expose distributed internals, but the default debugging experience should remain normal.

---

2. Non-goals for the MVP

The MVP should not attempt:

transparent raw remote pointers
live hot-swapping arbitrary Wasm/Rust stack frames
checkpointing live sockets
checkpointing arbitrary external OS state
consensus on every memory access
eagerly copying every byte of cluster memory on each breakpoint
transparent survival of lost non-replicated nodes
full Kubernetes/CI-platform replacement semantics

The MVP should favor simple, explicit, debuggable semantics over maximum transparency.

---

3. Core terminology

Cluster
  A set of real (or virtual) machines participating in one Disasmer runtime.

Coordinator
  The control-plane entry point for the cluster.
  Usually reachable through DNS, e.g. cluster.example.com.

Node
  A real machine participating in the cluster.

Node Runtime
  The Disasmer runtime process running on a node.
  It hosts virtual threads/tasks, memory shards, filesystem data, debug services,
  environment caches, and node-to-node communication.

Virtual Process
  The logical process seen by the program and debugger.
  Internally, it may span many nodes.

Virtual Thread
  A debugger-visible logical execution unit.
  For ordinary Disasmer programs, this is usually a Wasm execution context.
  For build-system use, it may be backed by a containerized process, sandboxed task,
  or platform-specific build command.

Virtual Task
  Synonym for Virtual Thread when the unit of work is task-like rather than
  thread-like. The debugger still sees it as a thread.

Memory Space
  A logical addressable memory region.
  It may be owned by any node and accessed remotely by other threads.

Filesystem Overlay
  A per-thread writable virtual filesystem layer.

Environment Resource
  A Containerfile, Dockerfile, Nix flake, or equivalent project resource that
  defines where and how a virtual thread/task should run.

Disasmer Bundle / Virtual Binary
  The packaged executable artifact containing Wasm modules, manifests, debug
  metadata, environment definitions, and selected input files.

Checkpoint
  A saved execution image containing enough state to resume a virtual thread.

Debug Epoch
  A globally coordinated stopped state used for debugging.

---

4. High-level architecture

VS Code / Debugger
  -> DAP / LLDB / custom Disasmer debug adapter
    -> Disasmer debug endpoint
      -> Coordinator
        -> Node Runtimes

Cluster connectivity:

new node starts
  -> connects to coordinator through DNS
  -> authenticates
  -> receives membership/configuration
  -> establishes peer links to other nodes

Node-to-node communication should prefer:

QUIC direct link
  -> NAT punching where possible
  -> relay fallback if direct connectivity fails

The coordinator is not a hypervisor. It is the cluster control plane. It manages membership, identity, placement, debug epochs, metadata, and failure decisions. The data plane should be mostly direct node-to-node traffic.

---

5. Project layout and resource discovery

Disasmer should make environment resources and input files first-class project resources.

Example project:

my-project/
  src/
    main.rs
  Cargo.toml
  Cargo.lock

  envs/
    linux/Containerfile
    windows/Containerfile
    macos/flake.nix
    macos/flake.lock

  inputs/
    config.json
    schema.sql
    test-data/
      example-a.txt
      example-b.txt

  disasmer.toml        # optional; used only for overrides/disambiguation

Default discovery rules:

envs/<name>/Containerfile
  -> defines environment <name> of kind container

envs/<name>/Dockerfile
  -> defines environment <name> of kind container

envs/<name>/flake.nix
  -> defines environment <name> of kind nix-flake

inputs/**
  -> included in the bundle's default VFS seed

src/**, Cargo.toml, Cargo.lock
  -> included for reproducible debugging/building when configured

The source code should be able to reference discovered environments by name:

env!("linux")
env!("windows")
env!("macos")

Adding this file:

envs/linux/Containerfile

should make this possible in source:

spawn::task(compile).env(env!("linux"))

No image name, host VM, registry, CI runner, or YAML platform configuration should be required in the source code.

---

6. Environment resources

An Environment Resource defines the execution environment for a virtual thread/task.

Supported MVP environment kinds:

container
  Built from Containerfile or Dockerfile.
  Produces or references an OCI-compatible image or equivalent runtime image.

nix-flake
  Resolved from flake.nix and flake.lock.
  Produces a reproducible command environment, dev shell, package closure, or
  runtime root depending on project configuration.

Containerfiles and Nix flakes are not identical concepts. Disasmer treats both as ways to produce a normalized Execution Environment:

Execution Environment
  name
  kind
  digest
  platform constraints
  runtime capabilities
  build/resolution recipe
  cached materialized form on capable nodes

Example normalized environments:

environment linux
  kind: container
  recipe: envs/linux/Containerfile
  context: envs/linux
  digest: sha256:...
  requires: os=linux, arch=x86_64

environment macos
  kind: nix-flake
  recipe: envs/macos/flake.nix
  lock: envs/macos/flake.lock
  digest: sha256:...
  requires: os=macos, arch=arm64

Environment materialization should be cached on nodes:

environment digest -> local image / Nix store path / sandbox root

The virtual binary should not embed full container images by default. It should embed recipes, manifests, hashes, lock files, and selected context files. Full vendoring should be an explicit mode:

disasmer build --vendor-envs

Possible vendored data:

OCI layers
Nix closure
prebuilt environment archive

---

7. Disasmer bundle / virtual binary

A Disasmer bundle is the packaged executable artifact.

It contains:

Wasm module(s)
Wasm component metadata
source maps / DWARF / debug metadata
Disasmer manifest
environment resource manifests
selected environment recipe files
selected environment context files
selected VFS seed files
source snapshot when enabled
content hashes
capability declarations

Conceptual bundle layout:

app.dis
  /modules/main.wasm
  /debug/main.dwarf
  /manifest/disasmer.json
  /envs/linux/Containerfile
  /envs/windows/Containerfile
  /envs/macos/flake.nix
  /envs/macos/flake.lock
  /inputs/config.json
  /inputs/schema.sql
  /inputs/test-data/**
  /source/src/main.rs
  /source/Cargo.toml
  /source/Cargo.lock

The bundle is content-addressed where possible. Every resource that affects execution should contribute to the bundle digest.

The coordinator schedules bundles, not loose files.

---

8. Execution model

A Disasmer application runs as one Virtual Process.

A virtual process contains:

virtual threads/tasks
memory spaces
filesystem overlays
runtime metadata
debug metadata
checkpoint metadata
environment bindings

Each virtual thread runs on exactly one node at a time:

thread T1 -> node A
thread T2 -> node B
thread T3 -> node C

A virtual thread may be implemented as:

Wasm execution context
containerized process
sandboxed native process
platform-specific build command

The debugger sees all of these as threads in one virtual process.

Thread placement is based on environment requirements:

spawn task with env!("linux")
  -> coordinator finds a node that can materialize/run environment linux
  -> node runtime starts the task

The source code selects logical environments, not specific machines:

env!("linux")      // logical environment

Not:

node("builder-17") // real machine, generally not wanted in portable source

---

9. Virtual filesystem model

Each virtual thread/task receives a virtual filesystem.

Default mount model:

/vfs/project
  Read-only source/input snapshot from the bundle.

/vfs/work
  Per-thread writable copy-on-write overlay.

/vfs/artifacts
  Published artifact namespace.

/vfs/shared
  Optional shared namespace with explicit synchronization rules.

A thread can create and modify files in its writable overlay.

File creation:

Thread T creates file F
  -> T owns the writable version of F
  -> bytes are physically stored on T's current node by default

Flush:

fs::flush()
  -> closes the current write epoch
  -> publishes filesystem changes
  -> makes changes visible according to VFS consistency rules

Sync:

fs::sync()
  -> makes selected published changes durable/replicated

Important distinction:

flush = visibility / synchronization point
sync  = durability / replication point

Suspend implicitly performs:

flush()
checkpoint_thread()
optionally sync()

Git-like analogy:

working tree  = thread-local filesystem overlay
commit        = flush point
object store  = chunk/blob storage
branch/head   = latest visible version

Disasmer does not need to literally use Git. A content-addressed chunk store is a better internal primitive.

---

10. Memory model

Disasmer memory is divided into Memory Spaces.

A memory space has:

stable virtual identity
address range or handle
current owner node
version/epoch metadata
optional cached pages
optional replicas

Allocation policy:

Memory allocated by a virtual thread is placed on that thread's current node by default.

Example:

thread T1 running on node A allocates memory M1
  -> M1 initially owned by node A

thread T2 running on node B allocates memory M2
  -> M2 initially owned by node B

Remote access is allowed, but mediated by Disasmer.

The MVP should not make ordinary raw pointers transparently remote. Instead, distributed state should use explicit Disasmer handles/types:

DBox<T>
DArc<T>
DMutex<T>
DVec<T>
DAtomicU64

Local state remains normal language state:

Vec<T>          // local to one Wasm context/thread
DVec<T>         // distributed memory object

std::sync::Mutex<T>       // local/node/thread concept
disasmer::sync::DMutex<T> // distributed synchronization primitive

The debugger sees a coherent virtual memory model. Internally, Disasmer maps:

virtual memory reference
  -> memory space
  -> owner node
  -> page/chunk
  -> epoch

---

11. Synchronization and consensus

The programming model should feel close to normal thread safety:

locks
atomics
happens-before
shared mutable memory
race conditions
linearizable operations

The distributed runtime still needs distributed-systems machinery underneath.

Consensus/control-plane coordination is used for:

cluster membership
node identity
debug epochs
memory ownership metadata
thread placement metadata
checkpoint ownership
filesystem version metadata
failure decisions

Consensus should not be on the hot path for every load/store.

Preferred split:

Control plane:
  consensus-backed metadata

Data plane:
  direct QUIC/RPC between node runtimes
  owner-based memory protocol
  leases
  fencing tokens
  copy-on-write pages/chunks

Every sensitive operation should carry an epoch or fencing token:

remote_load(addr, epoch)
remote_store(addr, epoch)
transfer_memory_owner(space, node, epoch)
migrate_thread(thread, node, epoch)
debug_freeze(epoch)
resume(epoch)

---

12. Suspend / resume / checkpoint

Disasmer supports suspending a virtual thread.

User-facing operation:

suspend().await?

Semantics:

1. Reach a safepoint.
2. Stop the virtual thread.
3. Flush the thread filesystem overlay.
4. Snapshot Wasm execution state.
5. Snapshot or reference stack/heap/memory pages.
6. Store checkpoint metadata.
7. Mark thread as PARKED.

Resume:

thread.resume().await?

Semantics:

1. Select a target node.
2. Fetch checkpoint metadata.
3. Restore Wasm execution state.
4. Restore/mount filesystem overlay.
5. Reconnect memory spaces.
6. Continue execution from a safepoint.

The coordinator should store checkpoint metadata, not necessarily all bytes.

Actual checkpoint data should live in:

node-local storage
replicated storage
content-addressed blob storage
object storage
peer-to-peer transferred chunks

This enables:

thread migration
hibernation
debug pause/resume
load balancing
manual relocation
fault recovery when state is replicated

---

13. Debugging model

Debuggability is a primary design requirement.

Recommended external interface:

VS Code
  -> DAP
    -> LLDB / lldb-dap or custom Disasmer adapter
      -> Disasmer debug endpoint
        -> Coordinator
          -> Node Runtimes

Disasmer should present the cluster as one virtual debug target.

When a breakpoint is hit:

1. A virtual thread hits a breakpoint or trap.
2. Coordinator creates a new Debug Epoch.
3. All node runtimes stop guest execution at safepoints.
4. Memory ownership/migration is frozen.
5. Thread states are collected.
6. Memory pages are pinned copy-on-write.
7. Filesystem overlay versions are pinned.
8. Debugger is told that all threads are stopped.

Debugger memory access should be served from a coherent snapshot.

Collect eagerly:

thread states
stacks
locals
globals
memory maps
filesystem manifests

Collect lazily:

heap pages
large memory regions
filesystem chunks
container logs beyond recent tail
large artifact data

The debugger illusion:

Debugger thinks:
  process P is stopped

Reality:
  cluster is stopped at Debug Epoch E

Optional Disasmer inspector:

thread -> node
thread -> environment
memory space -> owner node
file -> overlay version
artifact -> producer task
environment -> digest/cache state
remote calls
page migrations
lock wait graph
checkpoint state
consensus/control-plane state

The default debugger view should remain normal.

---

14. VS Code UX

Disasmer should provide a VS Code extension.

The extension should:

watch envs/**/Containerfile
watch envs/**/Dockerfile
watch envs/**/flake.nix
watch envs/**/flake.lock
watch inputs/**
watch src/**

run disasmer build before debugging
start or attach to a virtual process
bind breakpoints to Wasm/source debug info
show virtual threads in the normal debugger thread list
show stdout/stderr per virtual thread/task
show VFS/artifact state in an optional panel
show environment resources as first-class project items
provide autocomplete/diagnostics for env!("...") references

The intended UX:

1. Add envs/linux/Containerfile.
2. VS Code detects environment "linux".
3. Source autocomplete now accepts env!("linux").
4. Press F5.
5. Disasmer builds the virtual binary.
6. Disasmer starts the virtual process.
7. VS Code attaches to it as one normal debug target.
8. Breakpoints work across virtual threads/tasks.

The extension may generate editor metadata:

target/disasmer/generated/envs.rs
target/disasmer/generated/resources.json

But this generated metadata should not be hand-edited.

The project should not require YAML for ordinary usage.

Optional disasmer.toml may exist for overrides:

[envs.linux]
path = "envs/linux/Containerfile"
requires = { os = "linux", arch = "x86_64" }

[vfs]
include = ["inputs/**", "src/**", "Cargo.toml", "Cargo.lock"]
exclude = ["target/**", ".git/**"]

---

15. Editing source while debugging

Disasmer should support a fast edit/restart/debug loop.

MVP behavior:

source edit
  -> rebuild bundle
  -> restart selected virtual task/thread from a clean boundary
  -> reattach/rebind breakpoints

The MVP should not hot-swap arbitrary live code into existing stack frames.

Supported restart boundaries:

task entrypoint
last explicit task checkpoint
last published filesystem state
last clean build input snapshot

Useful VS Code commands:

Disasmer: Restart Selected Virtual Thread
Disasmer: Restart Failed Task
Disasmer: Restart From Last Flush
Disasmer: Restart Whole Virtual Process
Disasmer: Rebuild Bundle And Rebind Breakpoints

Semantics for restarting a selected task:

1. Save source changes.
2. Rebuild the Disasmer bundle.
3. Stop the selected virtual thread/task.
4. Discard unflushed task-local filesystem changes unless configured otherwise.
5. Start the task again using the new bundle version.
6. Mount the same declared environment.
7. Mount the same input/VFS seed or chosen checkpoint boundary.
8. Rebind breakpoints using new debug metadata.

The runtime may keep the rest of the virtual process alive only if compatibility checks pass:

same public task signatures
same serialized argument schema
compatible resource manifest
compatible memory/handle ABI

If compatibility checks fail, the extension should restart the whole virtual process.

Design rule:

Task restart is an MVP feature.
Live code hot-swap is not.

---

16. Rust as the primary source language

Rust should be the primary guest/source language for Disasmer.

Reasons:

explicit async boundaries map well to suspend/checkpoint
strong types map well to distributed handles
macros can register tasks and environments
source-level debugging is good
one source file can express real build logic

C/C++ should be supported later or in parallel as lower-level guest languages, but Rust should define the first-class ergonomic API.

Design rule:

Local things use normal Rust.
Distributed things use Disasmer types.

Examples:

Vec<T>                    // local
DVec<T>                   // distributed

std::sync::Mutex<T>       // local
disasmer::sync::DMutex<T> // distributed

std::fs                   // local/WASI-style filesystem, where supported
disasmer::fs              // virtual filesystem with flush/sync semantics

---

17. Rust API shape

The MVP Rust API should be explicit.

Example concepts:

#[disasmer::main]
async fn main() -> Result<()> { ... }

#[disasmer::task]
async fn compile(target: Target) -> Result<Artifact> { ... }

spawn::task(compile)
    .arg(Target::LinuxX64)
    .env(env!("linux"))
    .start()
    .await?;

The env!("linux") macro validates at build time that an environment resource named linux exists.

The #[disasmer::task] macro registers a task entrypoint so it can be started remotely by function identity plus serialized arguments.

Task arguments and results should be serializable:

arguments -> serialized into task start message
results   -> serialized into join result
handles   -> serialized as Disasmer object references

---

18. Rust example: multiplatform build in one source file

Project layout:

example-build/
  src/main.rs
  Cargo.toml
  Cargo.lock

  envs/
    linux/Containerfile
    windows/Containerfile
    macos/flake.nix
    macos/flake.lock

  inputs/
    build-config.json

Source file:

use disasmer::prelude::*;

#[derive(Serialize, Deserialize, Clone, Debug)]
enum Target {
    LinuxX64,
    WindowsX64,
    MacosArm64,
}

impl Target {
    fn name(&self) -> &'static str {
        match self {
            Target::LinuxX64 => "linux-x64",
            Target::WindowsX64 => "windows-x64",
            Target::MacosArm64 => "macos-arm64",
        }
    }

    fn triple(&self) -> &'static str {
        match self {
            Target::LinuxX64 => "x86_64-unknown-linux-gnu",
            Target::WindowsX64 => "x86_64-pc-windows-msvc",
            Target::MacosArm64 => "aarch64-apple-darwin",
        }
    }

    fn output_name(&self) -> &'static str {
        match self {
            Target::WindowsX64 => "app.exe",
            _ => "app",
        }
    }
}

#[disasmer::main]
async fn main() -> Result<()> {
    let completed = DAtomicU32::new(0).await?;

    let linux = spawn::task(compile)
        .arg(Target::LinuxX64)
        .arg(completed.clone())
        .env(env!("linux"))
        .start()
        .await?;

    let windows = spawn::task(compile)
        .arg(Target::WindowsX64)
        .arg(completed.clone())
        .env(env!("windows"))
        .start()
        .await?;

    let macos = spawn::task(compile)
        .arg(Target::MacosArm64)
        .arg(completed.clone())
        .env(env!("macos"))
        .start()
        .await?;

    let linux_artifact = linux.join().await?;
    let windows_artifact = windows.join().await?;
    let macos_artifact = macos.join().await?;

    package(vec![linux_artifact, windows_artifact, macos_artifact]).await?;

    assert_eq!(completed.load(Ordering::SeqCst).await?, 3);

    fs::sync().await?;
    Ok(())
}

#[disasmer::task]
async fn compile(target: Target, completed: DAtomicU32) -> Result<Artifact> {
    let target_name = target.name();
    let triple = target.triple();
    let output_name = target.output_name();

    cmd!("cargo", "build", "--release", "--target", triple)
        .cwd("/vfs/project")
        .run()
        .await?;

    let built_file = format!(
        "/vfs/project/target/{triple}/release/{output_name}"
    );

    let artifact_file = format!(
        "/vfs/artifacts/{target_name}/{output_name}"
    );

    fs::copy(&built_file, &artifact_file).await?;

    // Publish this task's filesystem changes so other tasks can see the artifact.
    fs::flush().await?;

    completed.fetch_add(1, Ordering::SeqCst).await?;

    Ok(Artifact::from_path(artifact_file).await?)
}

#[disasmer::task]
async fn package(outputs: Vec<Artifact>) -> Result<()> {
    fs::create_dir_all("/vfs/artifacts/dist").await?;

    for output in outputs {
        fs::copy(output.path(), "/vfs/artifacts/dist/").await?;
    }

    fs::flush().await?;
    Ok(())
}

---

19. What the Rust example does

The example defines a complete multiplatform build as one Rust program.

It does this:

1. Starts one Disasmer Virtual Process.
2. Creates one distributed atomic counter.
3. Spawns three virtual tasks:
   - Linux x64 build task
   - Windows x64 build task
   - macOS ARM64 build task
4. Places each task into an environment discovered from envs/<name>/.
5. Runs each build command in that environment.
6. Copies each produced binary into /vfs/artifacts.
7. Calls fs::flush() to publish each task's artifact.
8. Joins all tasks.
9. Packages all artifacts into /vfs/artifacts/dist.
10. Calls fs::sync() to make final outputs durable.

Runtime behavior:

env!("linux")
  -> resolved from envs/linux/Containerfile

env!("windows")
  -> resolved from envs/windows/Containerfile

env!("macos")
  -> resolved from envs/macos/flake.nix + flake.lock

spawn compile Linux
  -> coordinator finds a node capable of running the linux environment
  -> node runtime materializes/caches the environment
  -> task runs with its own VFS overlay

spawn compile Windows
  -> coordinator finds a node capable of running the windows environment
  -> task runs separately but in the same virtual process

spawn compile macOS
  -> coordinator finds a node capable of running the macos environment
  -> task runs separately but in the same virtual process

Debugger behavior:

VS Code sees one process:
  Thread 1: main
  Thread 2: compile LinuxX64
  Thread 3: compile WindowsX64
  Thread 4: compile MacosArm64
  Thread 5: package

The user can set breakpoints in:

main()
compile()
package()

When a breakpoint hits, Disasmer enters a Debug Epoch and stops the whole virtual process.

---

20. What the Rust example proves

The example proves the core Disasmer thesis:

A distributed, multiplatform build can be expressed as normal source code,
run across different machines/environments,
and debugged as one normal multithreaded program.

It proves these specific capabilities:

one source file can define the build graph
Containerfiles/Nix flakes can be project resources, not CI YAML
source code can reference environments by logical name
virtual tasks can run on different operating systems/architectures
artifacts can be published through flush()
final outputs can be made durable through sync()
shared distributed state can use explicit Disasmer types
the debugger can show all tasks as threads of one process

It also demonstrates the intended mental model:

Build = Virtual Process
Build Step = Virtual Thread / Virtual Task
Containerfile/Nix flake = Environment Resource
Build output = VFS artifact
flush() = publish/synchronization point
sync() = durability point

---

21. What this makes possible

Disasmer makes these workflows possible:

multiplatform builds without proprietary CI YAML
source-level debugging of build logic
breakpoints inside distributed build orchestration
restart one failed build task after editing source
inspect all build tasks in one debugger session
share artifacts through a versioned virtual filesystem
run Linux/Windows/macOS tasks from one source file
mix Wasm orchestration with containerized native commands
make distributed execution feel like threaded programming

Example VS Code loop:

1. Press F5.
2. Build starts as one virtual process.
3. Windows task fails.
4. Breakpoint/trap stops the virtual process.
5. User edits src/main.rs or project source.
6. User runs "Disasmer: Restart Failed Task".
7. Disasmer rebuilds the bundle.
8. Only the failed task restarts if compatible.
9. Breakpoints are rebound.
10. User continues debugging.

This is intentionally different from typical CI systems:

not a pile of YAML
not tied to one proprietary platform
not opaque remote job logs only
not separate debuggers per machine

---

22. C/C++ support

C/C++ should be possible but should not define the primary user experience.

C-style API shape:

dis_task_t linux = dis_spawn_env("linux", compile_task, &args, sizeof(args));
dis_join(linux);
dis_fs_flush();
dis_fs_sync();

Distributed memory in C should use handles:

dis_ref_t buffer = dis_mem_alloc(1024);
dis_mem_read(buffer, 0, tmp, sizeof(tmp));
dis_mem_write(buffer, 0, data, len);

The MVP should not make raw C pointers transparently remote.

Rust remains the preferred source language because it gives better type safety, async ergonomics, task registration, and IDE UX.

---

23. Failure semantics

MVP failure behavior should be simple.

If a node with non-replicated guest state is lost:

the virtual process is considered crashed

If a task is checkpointed and its checkpoint data is durable:

the task may be resumed elsewhere

If a task has only unflushed local filesystem changes and its node is lost:

those unflushed changes are lost

If artifacts have been flushed but not synced:

they may be visible but not durable

If artifacts have been synced:

they should survive according to the configured replication policy

Do not pretend missing non-replicated state can be debugged.

---

24. Security and identity

Disasmer should treat cluster identity and bundle identity as core primitives.

MVP requirements:

nodes authenticate to the coordinator
coordinator authorizes nodes for environments/capabilities
bundles are content-addressed
bundle manifests are signed or integrity-checked
environment resources are resolved by digest
tasks run with explicit capabilities
filesystem access is scoped to the virtual filesystem unless granted
network access is explicit per environment/task

The source should not casually select real machines by name. It should select logical environments and capabilities.

---

25. MVP feature set

The MVP should include:

Rust SDK
#[disasmer::main]
#[disasmer::task]
env!("name") environment references
auto-discovery of envs/<name>/Containerfile
auto-discovery of envs/<name>/Dockerfile
auto-discovery of envs/<name>/flake.nix
bundle manifest generation
selected input file bundling
per-task VFS overlay
fs::flush()
fs::sync()
spawn task into environment
join task
basic distributed handles
all-stop debugging
Debug Epochs
lazy memory/filesystem snapshotting
VS Code launch/debug extension
restart selected task from clean boundary after rebuild
QUIC node-to-node transport
coordinator-based node discovery

MVP restrictions:

all-stop debugging only
cooperative safepoints only
task restart, not live hot-swap
explicit distributed types only
no transparent raw remote pointers
no checkpointing live sockets
no automatic survival of non-replicated node loss
no eager full-cluster memory copy by default
no global consensus on every data operation

---

26. Later extensions

Possible later extensions:

non-stop debugging
live code replacement for compatible functions
state-schema-aware checkpoint migration
replicated memory spaces
recoverable distributed locks
watchpoints across distributed memory
record/replay debugging
artifact cache sharing across clusters
remote execution marketplace
Kubernetes backend
cloud worker backend
local laptop cluster mode
full offline vendored bundles
language SDKs for C/C++, Go, Zig, Python/Wasm

These are not MVP requirements.

---

27. Design summary

Disasmer is:

a distributed Wasm runtime
that presents a cluster as one virtual process
with virtual threads/tasks, sharded memory,
per-thread virtual filesystems, checkpoint/suspend/resume,
container/Nix-backed execution environments,
and normal debugger support.

For build systems, Disasmer means:

Build = Virtual Process
Build Step = Virtual Thread / Virtual Task
Environment = Containerfile / Dockerfile / Nix flake resource
Artifact = VFS-published file
Debugger = normal VS Code session

The core UX promise:

Add a Containerfile or flake.nix to the project.
Reference it from Rust with env!("name").
Press F5.
Debug the distributed build/program like one normal process.

The internal truth:

normal debugger illusion
  backed by
cluster-wide debug epochs
copy-on-write snapshots
node runtimes
QUIC peer links
distributed memory spaces
filesystem overlays
checkpoint manifests
environment caches
and a consensus-backed control plane