clusterflux-public/thread_args_wasmtime_notes.md

# Disasmer Thread Arguments with Wasmtime

## Goal

Disasmer should make spawning a virtual thread feel similar to Rust's `thread::spawn`, but with stricter rules because the thread may run on another machine, in another container, or after a restart.

The core rule:

```text
Anything passed to a virtual thread must be either:
  - a small serializable value
  - an immutable content-addressed object
  - a Disasmer runtime handle
```

Normal raw pointers, borrowed references, local file handles, sockets, and host-only resources are not valid virtual-thread arguments.

---

## Recommended MVP Model

Use Wasmtime as the embedded Wasm engine.

Do not try to migrate arbitrary Rust closures or live native stacks.

Instead, compile Disasmer programs into Wasm modules/components with named task entrypoints.

```rust
#[derive(DisasmerArg)]
struct BuildArgs {
    commit: git::Commit,
    profile: String,
    src: SourceSnapshot,
}

#[disasmer::task]
async fn linux_build(args: BuildArgs) -> Result<Artifact> {
    fs::mount("/src", args.src).await?;

    cmd!("cargo build --profile {}", args.profile)
        .cwd("/src")
        .run()
        .await?;

    let artifact = artifact::publish(
        format!("linux-app-{}", args.commit),
        "/src/target/release/app",
    ).await?;

    fs::flush().await?;
    Ok(artifact)
}

#[disasmer::main]
async fn main() -> Result<()> {
    let commit = git::Commit::current().await?;
    let src = git::Repo::open("/repo").snapshot(commit).await?;

    let args = BuildArgs {
        commit,
        profile: "release".into(),
        src,
    };

    let task = spawn(env!("linux"), linux_build(args)).await?;
    let artifact = task.join().await?;

    ui::log(format!("built {artifact:?}")).await?;
    Ok(())
}
```

---

## What This Means

The source looks like ordinary Rust, but spawning is not the same as `std::thread::spawn`.

```text
std::thread::spawn(move || ...)
  moves values into an OS thread in the same process

Disasmer spawn(...)
  creates a TaskSpec
  serializes small arguments
  copies handles for large/distributed objects
  asks the coordinator to place a virtual thread
  starts the task in a Wasmtime instance on the selected node
```

`src` is not a copied Git repo. It is a handle to an immutable source snapshot.

`env!("linux")` is not a container image. It is a handle/digest for an environment discovered from the project.

`artifact` is not a copied binary. It is a handle to a published output.

---

## Runtime Flow

When a Disasmer program calls `spawn(env!("linux"), linux_build(args))`:

```text
1. Guest code calls a Disasmer host import.
2. The host import receives:
     - task name / task id
     - environment id
     - serialized argument bytes
     - referenced runtime handles
3. The coordinator creates a Virtual Thread record.
4. Scheduler chooses a node based on environment and data locality.
5. The node runtime instantiates the Wasm module with Wasmtime.
6. The node runtime links Disasmer host functions.
7. The task entrypoint is called with decoded arguments.
8. The task runs until completion, suspend, trap, or cancellation.
```

The virtual thread is debugger-visible as a normal thread of the virtual process.

---

## Minimal Wasmtime Shape

Each node runtime embeds Wasmtime.

```text
Node Runtime
  Engine
  compiled module/component cache
  Store<ThreadHostState>
  Linker with Disasmer imports
  task entrypoint dispatcher
```

`ThreadHostState` contains the host-side context for the running virtual thread:

```text
thread id
virtual process id
debug epoch
filesystem overlay
handle table
resource limits
coordinator connection
```

Disasmer host imports are implemented through Wasmtime's linker/host-function mechanism.

Examples:

```text
disasmer.spawn(...)
disasmer.fs.flush(...)
disasmer.artifact.publish(...)
disasmer.channel.send(...)
disasmer.suspend(...)
```

If a host import needs to read serialized arguments from guest memory, it uses the caller context to access the guest's exported memory.

---

## Argument Categories

Default behavior:

```text
u32, bool, String, small structs
  serialized and copied into the TaskSpec

SourceSnapshot
  handle only

Artifact
  handle only

Env
  handle/digest only

DArc, DMutex, DChannel
  distributed runtime handle only

Vec<u8>
  copied only if small
  rejected or warned if large
```

Large data should be passed as explicit runtime objects:

```rust
let blob = Blob::from_file("large-input.dat").await?;
spawn(env!("linux"), process_blob(blob)).await?;
```

Not:

```rust
let bytes = fs::read("large-input.dat").await?;
spawn(env!("linux"), process_bytes(bytes)).await?;
```

---

## Preventing Confusing Situations

Disasmer should fail early and clearly.

### 1. Require `DisasmerArg`

Every task argument type must implement `DisasmerArg`.

Usually this is derived:

```rust
#[derive(DisasmerArg)]
struct Args {
    src: SourceSnapshot,
    profile: String,
}
```

If a field cannot cross a virtual-thread boundary, compilation should fail with a helpful message.

Bad:

```rust
#[derive(DisasmerArg)]
struct Args<'a> {
    path: &'a str,
}
```

Suggested error:

```text
Args cannot be passed to a Disasmer virtual thread because it contains a borrowed reference.
Use String, Arc-like Disasmer handles, Blob, SourceSnapshot, or Artifact instead.
```

### 2. Prefer Named Tasks

For the MVP, prefer:

```rust
#[disasmer::task]
async fn linux_build(args: BuildArgs) -> Result<Artifact> { ... }
```

over arbitrary async closures.

Closure syntax can come later, but should lower to:

```text
named generated task
captured argument struct
derived DisasmerArg
```

This makes spawning, restarting, debugging, and serialization much simpler.

### 3. Ban Host-Only Types

These should not be valid task arguments:

```text
&T / &mut T
raw pointers
Rc<T>
RefCell<T>
std::fs::File
TcpStream
MutexGuard
local process handles
thread handles
unbounded Vec<u8>
```

Use explicit Disasmer types instead:

```text
DArc<T>
DMutex<T>
DChannel<T>
Blob
Artifact
SourceSnapshot
VirtualFile
```

### 4. Make Latency Visible in the Type System

Operations that may cross machines should usually be async:

```rust
progress.lock().await?;
fs::flush().await?;
suspend().await?;
artifact::publish(...).await?;
```

This makes distributed boundaries visible without making the code noisy.

### 5. Add Size Limits and Warnings

If a user accidentally captures or passes a huge value, fail or warn:

```text
Task argument is 82 MB.
This would be copied to the spawned virtual thread.
Use Blob::from_file, SourceSnapshot, or Artifact instead.
```

### 6. Make Environment References Compile-Time Checked

`env!("linux")` should be generated from project resources such as:

```text
envs/linux/Containerfile
envs/linux/Dockerfile
envs/linux/flake.nix
```

If the environment does not exist, the user should get a compile-time/editor error.

### 7. Do Not Pretend `std::thread::spawn` Is Distributed

Disasmer should provide its own spawn API.

```rust
disasmer::spawn(...)
```

`std::thread::spawn` should either be unsupported in the Disasmer guest target or clearly documented as local-only/not part of the distributed runtime model.

---

## Restarting After Source Changes

For the MVP, do not hot-swap arbitrary live Wasm code.

Use task restart instead:

```text
edit source in VS Code
rebuild Wasm module
stop selected virtual thread
recreate task from:
  - task name
  - new module hash
  - environment handle
  - original arguments
  - latest allowed VFS snapshot/checkpoint
start again
```

This is simple, understandable, and debuggable.

Do not resume an old Wasm stack into a new compiled module unless Disasmer later grows explicit continuation/checkpoint support for that case.

---

## What This Proves

This model proves that Disasmer can express build-system tasks as normal-looking Rust code while keeping the runtime implementation realistic.

It enables:

```text
one source file for build logic
typed task arguments
container/Nix environment selection
source snapshots without copying repos
artifact handles without copying binaries
normal debugger thread views
safe task restart after edits
simple Wasmtime embedding
```

The key design choice is honesty:

```text
local Rust values are local
Disasmer handles are distributed
small arguments are copied
large data is passed by content-addressed handle
```

That avoids surprising network copies and prevents users from thinking arbitrary local process state can magically move across machines.

---

## References

- Wasmtime Rust embedding API: https://docs.wasmtime.dev/api/wasmtime/
- Wasmtime `Linker`: https://docs.wasmtime.dev/api/wasmtime/struct.Linker.html
- Wasmtime `Store`: https://docs.wasmtime.dev/api/wasmtime/struct.Store.html
- Rust `std::thread::spawn`: https://doc.rust-lang.org/std/thread/fn.spawn.html
- Serde derive: https://serde.rs/derive.html