Public dry run dryrun-20b59dc46b72
Source commit: 20b59dc46b72103c2f8a516c692b5cc3d54fab19 Public tree identity: sha256:aaa5ac7b58b2a0d23c6b11e5f76324bf3839ca1f62653a83deb736932adcfbd9
This commit is contained in:
commit
2bef715211
221 changed files with 79792 additions and 0 deletions
46
docs/architecture.md
Normal file
46
docs/architecture.md
Normal file
|
|
@ -0,0 +1,46 @@
|
|||
# Architecture
|
||||
|
||||
Disasmer presents a distributed workflow as one virtual process. The workflow
|
||||
program and its tasks are WebAssembly modules; the coordinator, attached nodes,
|
||||
CLI, and debugger are separate host processes.
|
||||
|
||||
## Components
|
||||
|
||||
- The SDK and macros turn Rust workflow functions into versioned Wasm exports.
|
||||
- The CLI builds and validates a bounded bundle, creates a virtual process, and
|
||||
launches its selected entrypoint as the coordinator main.
|
||||
- The coordinator owns identity, authorization, scheduling, process state, and
|
||||
debug epochs. It executes only the capless coordinator-main Wasm: that main
|
||||
may spawn/join tasks, wait, log bounded state, and participate in debugging,
|
||||
but cannot run commands, read source or host files, use sockets, or obtain
|
||||
secrets. The coordinator does not retain artifact bytes.
|
||||
- Attached nodes verify assignments and execute Wasm with Wasmtime. Host
|
||||
operations are available only through imports backed by granted capabilities.
|
||||
- The DAP adapter maps a virtual process and its tasks to ordinary debugger
|
||||
processes and threads without pretending that source-level stepping exists.
|
||||
|
||||
The standalone Core coordinator is the public self-hosted service. The Hosted
|
||||
coordinator wraps the same Core service with private identity-provider,
|
||||
community-policy, operator, and deployment code. Public/private is a source
|
||||
distribution boundary, not an authority boundary.
|
||||
|
||||
## State
|
||||
|
||||
Durable Postgres state contains tenants, users, projects, credentials,
|
||||
permissions, source-provider configuration, and service policy records. Active
|
||||
virtual processes, assignments, debug epochs, transient VFS views, and artifact
|
||||
locations are intentionally ephemeral for the MVP. Restarting the coordinator
|
||||
therefore preserves identity and project configuration but ends active work.
|
||||
|
||||
## Execution and placement
|
||||
|
||||
Wasm is the execution substrate, not a capability. The coordinator main uses a
|
||||
small fixed capless import set. A node task's `TaskSpec` declares the additional
|
||||
host operations and environment it needs, and a node is eligible only when its
|
||||
reported capabilities satisfy those requirements. For example, a Wasm task may
|
||||
request command execution in a declared Linux environment; a user-attached node
|
||||
with rootless Podman can provide that operation, while hosted community
|
||||
infrastructure denies arbitrary native commands and hosted containers.
|
||||
|
||||
The build-system workflow is the MVP wedge. General distributed application
|
||||
runtime features are deliberately outside the initial release.
|
||||
37
docs/artifacts.md
Normal file
37
docs/artifacts.md
Normal file
|
|
@ -0,0 +1,37 @@
|
|||
# Artifacts
|
||||
|
||||
Artifacts are named, content-addressed results retained by the node that
|
||||
produced or received their bytes. The coordinator stores authorization and
|
||||
location metadata, not an implicit copy of every output.
|
||||
|
||||
## Publish, flush, and sync
|
||||
|
||||
- Publishing from a Wasm task transfers bytes to its retaining node and records
|
||||
the digest and size.
|
||||
- `flush()` publishes metadata so downstream tasks can refer to the artifact. It
|
||||
does not imply coordinator storage or durable replication.
|
||||
- `sync()` is an explicit movement decision to another node or user-provided
|
||||
storage integration.
|
||||
|
||||
Large bytes should travel node-to-node or node-to-receiver. Task arguments carry
|
||||
artifact handles rather than embedding output data.
|
||||
|
||||
## Download and export
|
||||
|
||||
`disasmer artifact download <artifact>` asks the coordinator for an authorized,
|
||||
scoped stream and retrieves the real bytes from a retaining node. Authorization
|
||||
is bound to tenant, project, process, artifact, actor, retention state, expiry,
|
||||
and quota. Missing, expired, revoked, oversized, or unreachable artifacts fail
|
||||
explicitly before success is reported.
|
||||
|
||||
Final export is also explicit. A project can attach a receiver node or invoke a
|
||||
user-provided storage tool from an allowed environment. The MVP intentionally
|
||||
does not advertise a managed artifact-store product.
|
||||
|
||||
Node retention is best effort unless project code synchronizes the artifact to
|
||||
a durable destination. Coordinator restart does not manufacture or preserve
|
||||
artifact bytes. Each node enforces configured byte, object-count, and age
|
||||
bounds. Artifacts required by active tasks, live downloads, or a retained
|
||||
restart boundary are pinned; garbage collection evicts eligible older objects
|
||||
and reports unavailable bytes honestly. Coordinator metadata and download
|
||||
session histories are independently count- and time-bounded.
|
||||
30
docs/debugging.md
Normal file
30
docs/debugging.md
Normal file
|
|
@ -0,0 +1,30 @@
|
|||
# Debugging
|
||||
|
||||
Disasmer exposes the distributed workflow as one virtual process with virtual
|
||||
threads for tasks. The VS Code extension launches or attaches through the DAP
|
||||
adapter and reads the same live process registry used by the CLI.
|
||||
|
||||
## Supported controls
|
||||
|
||||
- Launch and attach to a real coordinator process.
|
||||
- Inspect processes, tasks, logs, artifacts, nodes, scopes, and variables.
|
||||
- Set function/source breakpoints supported by the compiled Wasm mapping.
|
||||
- Continue and pause through participant-acknowledged debug epochs.
|
||||
- Rebuild and restart a terminal task from its captured clean VFS entry boundary
|
||||
under a fresh task-instance ID when the edited task ABI and boundary remain
|
||||
compatible. Incompatible edits require a whole-process restart without
|
||||
altering the running process.
|
||||
- Cooperatively cancel or explicitly abort the whole virtual process.
|
||||
|
||||
Source-level stepping is not in the MVP. Step requests fail explicitly instead
|
||||
of returning a synthetic success. All-stop succeeds only after every active
|
||||
participant acknowledges the requested epoch; a missing participant is reported
|
||||
as a freeze failure.
|
||||
|
||||
When F5 finds an existing process in the same Coordinator Project, the extension
|
||||
offers attach, restart, cancel, or abort. If the process came from another local
|
||||
workspace, it does not silently attach with the wrong source mapping. Processes
|
||||
remain visible independent of whether they currently contain active tasks.
|
||||
|
||||
The DAP demo backend is an isolated adapter test fixture. Product launches use
|
||||
the coordinator/node runtime path and do not fall back to the demo model.
|
||||
54
docs/security.md
Normal file
54
docs/security.md
Normal file
|
|
@ -0,0 +1,54 @@
|
|||
# Security model
|
||||
|
||||
Disasmer runs untrusted project code as WebAssembly on attached nodes. Wasm is
|
||||
the isolation substrate; access to host operations is the capability boundary.
|
||||
Running a Wasm module alone does not grant commands, containers, source access,
|
||||
artifact access, networking, or coordinator authority.
|
||||
|
||||
## Authority lanes
|
||||
|
||||
- Client requests use an authenticated session scoped to one user, tenant, and
|
||||
project.
|
||||
- Nodes and non-interactive agents use Ed25519 request signatures, bounded
|
||||
timestamps, unique nonces, and explicitly scoped enrollment grants.
|
||||
- Identity requests create and poll browser-login transactions. The hosted
|
||||
callback exchanges the authorization code with Authentik and obtains identity
|
||||
from its authenticated user-info endpoint; the CLI cannot supply trusted
|
||||
identity claims.
|
||||
- Hosted Operator requests use a separate private credential and replay-safe
|
||||
proof. They are not Client protocol variants authorized by convention.
|
||||
|
||||
The coordinator uses its own clock for expiry, quota windows, and policy
|
||||
decisions. Quota meters are keyed by tenant, project, resource kind, and window.
|
||||
Client-provided actor or scope strings are not authority. Replay windows,
|
||||
login/enrollment transactions, task and debug histories, downloads, and
|
||||
artifact metadata all have explicit expiry or count bounds scoped to their
|
||||
authority or project/process object.
|
||||
|
||||
## Transport and secrets
|
||||
|
||||
Remote control traffic must use HTTPS and follows normal certificate
|
||||
validation. Plain JSON-lines TCP and HTTP are restricted to loopback and require
|
||||
an explicit local-development mode on the coordinator. The hosted deployment
|
||||
terminates TLS at its reverse proxy and forwards only to a loopback service.
|
||||
|
||||
Node keys are generated from operating-system randomness. Local credential
|
||||
directories and files use Unix modes `0700` and `0600`, reject symbolic-link
|
||||
paths, and are written atomically without overwriting an existing credential.
|
||||
Environment-provided keys remain the operator's storage responsibility.
|
||||
|
||||
## Trust boundary
|
||||
|
||||
Rootless Podman is a useful additional containment and environment mechanism on
|
||||
Linux nodes, but it does not turn a denied host operation into an allowed one.
|
||||
Native commands require an explicit environment, cwd, bounded environment map,
|
||||
wall-clock timeout, output mount, and network policy. Linux container commands
|
||||
run with resource and process-count limits, no-new-privileges, and dropped
|
||||
capabilities; Wasmtime stores independently enforce memory/table/instance and
|
||||
fuel bounds. A limit failure is a bounded task failure and does not terminate
|
||||
the node daemon.
|
||||
Node operators decide which projects they trust and which host capabilities a
|
||||
node advertises. The community hosted service does not provide arbitrary hosted
|
||||
native execution; users attach their own nodes for real work.
|
||||
|
||||
Report vulnerabilities according to [SECURITY.md](../SECURITY.md).
|
||||
49
docs/self-hosting.md
Normal file
49
docs/self-hosting.md
Normal file
|
|
@ -0,0 +1,49 @@
|
|||
# Self-hosting
|
||||
|
||||
The public workspace contains the standalone Core coordinator, node runtime,
|
||||
CLI, SDK, DAP adapter, and VS Code extension. Self-hosting does not require the
|
||||
private hosted website or Authentik integration.
|
||||
|
||||
Build the workspace, then start a coordinator with a protected bootstrap
|
||||
session:
|
||||
|
||||
```bash
|
||||
cargo build --workspace
|
||||
DISASMER_SELF_HOSTED_SESSION_SECRET="$SELF_HOSTED_SESSION_SECRET" \
|
||||
DISASMER_SELF_HOSTED_TENANT=my-team \
|
||||
DISASMER_SELF_HOSTED_PROJECT=my-project \
|
||||
DISASMER_SELF_HOSTED_USER=me \
|
||||
target/debug/disasmer-coordinator --listen 127.0.0.1:7999
|
||||
```
|
||||
|
||||
The coordinator's native JSON-lines transport is deliberately loopback-only and
|
||||
the server refuses a non-loopback listener. For administration from another
|
||||
machine, use an authenticated SSH tunnel and connect to its local endpoint:
|
||||
|
||||
```bash
|
||||
ssh -N -L 7999:127.0.0.1:7999 coordinator.example
|
||||
```
|
||||
|
||||
Do not expose plaintext port 7999 to a network. The hosted service is the
|
||||
reference HTTPS deployment shape: TLS reverse proxy to a loopback HTTP
|
||||
application endpoint.
|
||||
|
||||
Connect a CLI without putting the secret in its process arguments:
|
||||
|
||||
```bash
|
||||
printf '%s\n' "$SELF_HOSTED_SESSION_SECRET" | \
|
||||
disasmer auth connect-self-hosted \
|
||||
--coordinator 127.0.0.1:7999 \
|
||||
--tenant my-team --project-id my-project --user me \
|
||||
--session-secret-stdin
|
||||
```
|
||||
|
||||
Create a scoped node enrollment grant, then attach nodes that advertise only the
|
||||
host capabilities you intend to provide. Linux command/container environments
|
||||
require rootless Podman on those nodes. Wasmtime execution itself is always part
|
||||
of the runtime and is not an optional capability.
|
||||
|
||||
Configure `DATABASE_URL` for durable identity and project state. Active
|
||||
processes and tasks are ephemeral in the MVP and end when the coordinator
|
||||
restarts. Back up Postgres and any user-selected artifact destination; node-local
|
||||
artifact retention is best effort.
|
||||
45
docs/task-abi.md
Normal file
45
docs/task-abi.md
Normal file
|
|
@ -0,0 +1,45 @@
|
|||
# Task ABI
|
||||
|
||||
The MVP task ABI is a small, versioned contract between a compiled workflow
|
||||
module and the coordinator-main or node Wasmtime host. It is intentionally
|
||||
narrower than WASI and should grow only when the flagship workflow needs
|
||||
another host operation.
|
||||
|
||||
## Assignment
|
||||
|
||||
Each assignment carries a `TaskSpec` containing the virtual process and task
|
||||
identity, bundle module, declared Wasm export and ABI version, serialized
|
||||
arguments, required capabilities and artifacts, selected environment, and VFS
|
||||
epoch. The node rejects a missing or incompatible spec rather than guessing an
|
||||
entrypoint.
|
||||
|
||||
SDK task macros generate exports that decode versioned boundary values, invoke
|
||||
the Rust function, and encode the result. Values crossing a task boundary are
|
||||
limited to serializable data and Disasmer handles; host-only objects fail at
|
||||
compile time.
|
||||
|
||||
## Host imports
|
||||
|
||||
The current guest imports cover task spawn/join, command execution,
|
||||
cancellation observation, source snapshots, VFS file operations, artifact
|
||||
publication, and debug probes. Every import validates memory ranges and payload
|
||||
sizes. The coordinator main receives only spawn/join, waiting, bounded state,
|
||||
and debug controls; node hosts check the assignment's declared capabilities
|
||||
before performing source, command, network, or artifact operations.
|
||||
|
||||
`disasmer.command_run_v1` runs a command through the selected environment
|
||||
backend. On Linux, a task assigned to an environment requiring rootless Podman
|
||||
uses the normal Podman assignment path. The hosted community policy does not
|
||||
grant this on managed infrastructure; a user node may grant it for its own
|
||||
machine.
|
||||
|
||||
## Lifecycle
|
||||
|
||||
Cooperative cancellation is visible to guest code at a host boundary. Forced
|
||||
abort arms Wasmtime epoch interruption and also terminates active environment
|
||||
work. Debug freeze and resume synchronize at acknowledged host or Wasmtime epoch
|
||||
boundaries. These controls do not claim instruction-accurate source stepping.
|
||||
Each task assignment captures a restart-compatibility hash and clean VFS entry
|
||||
boundary. DAP task restart rebuilds the bundle and accepts edited code only when
|
||||
the task ABI, declared capabilities, environment, arguments, handles, and VFS
|
||||
boundary remain compatible.
|
||||
Loading…
Add table
Add a link
Reference in a new issue