Parallelism, Part I — Two flavors, and which to prioritize

These notes are about real parallelism — running compute on multiple CPU cores simultaneously — for SPy, given its WASI-based interpreter, its three compilation targets (native, WASI, emscripten), its self-hosting goal, and its memory model (GC + manual, no ownership in the type system).

They are not about concurrency. SPy is close to Go in spirit, and Go is famous for its concurrency model — goroutines, channels, async/await-style programming, lightweight cooperative scheduling. That is a different problem from what is addressed here. Concurrency is about structuring a program that has many in-flight, mostly-waiting tasks (I/O, event loops, coroutines) so they don’t block each other; it needs only one core and gives no speedup to a CPU-bound kernel. Parallelism is about using several cores at once to finish a compute faster. WASI 0.3’s stream<T>/future<T> and Go-style goroutines belong to the concurrency world; these notes belong to the parallelism world. The two are related and can compose, but conflating them is the first trap to avoid, and SPy’s concurrency story (if any) is a separate topic.

Within parallelism, the notes are in two parts, in two files. This file (Part I) compares the two possible models — shared-heap threads vs. isolated isolates — and explains why isolated isolates is the priority. Part II details the isolated-isolate design that Part I argues for.

This is a design-notes document, not an implementation plan: it lays out the reasoning and the tradeoffs, flags what’s still open, and gives a sequencing that follows from the difficulty gradient rather than from a schedule.

1. Context and constraints¶

SPy’s parallelism story must be compatible with:

• an interpreter based on WASI (the self-hosting plan is a WASI application calling libspy, sharing one linear memory);

• three compilation targets: native, WASI (Zig → wasm32-wasi), and emscripten (browser);

• a self-hosting milestone, which removes the CPython GIL as a ceiling;

• a memory model of GC + manual allocation (Boehm GC is temporary; basic refcounting and Whippet are planned), with a deliberate absence of Rust/Mojo-style ownership in the type system — a simplicity argument.

The relevant WebAssembly substrate is the core-wasm threads family (shared memory + atomics, or isolated instances) — not WASI 0.3 (stream<T>, future<T>, host event loop), which is cooperative concurrency on one core and out of scope here.

2. Flavor 1 — Shared-heap threads (intra-instance)¶

Multiple threads inside one wasm instance sharing one linear memory and one GC heap.

• Best for dense data-parallel numeric kernels over a pre-allocated array, where workers barely allocate and exchange is by raw pointer (zero-copy).

• Cost: every IncRef/DecRef and every collection must coordinate across threads → needs atomic refcounts, a thread-safe allocator, and a global stop-the-world (or concurrent) collector. This is the classic hard problem that motivated CPython’s GIL.

• Availability: easy on native (OS threads + multi-threaded Boehm); stable on emscripten (pthreads over SharedArrayBuffer); not available on WASI until shared-everything-threads ships.

3. Flavor 2 — Isolated isolates + shared data arena (inter-instance)¶

Multiple wasm instances, each with its own private linear memory and own GC, plus a separate shared atomic arena for bulk data exchange. This is the model Part II works out in detail.

• Best for task-parallel, allocation-heavy, interpreter-internal, and sandboxed workloads.

• Cost: data crossing the boundary is copied or lives in the explicit arena; no sharing of mutable GC’d object graphs.

• Key advantage: each collector runs independently — no global stop-the-world, no atomic refcounts, no cross-thread GC coordination (see the collector caveat in §5, point 2 — this holds for refcounting and for Whippet, but not yet for Boehm as currently used).

4. Cross-target availability¶

Flavor 2 is the only model uniformly available on all three targets today.

By contrast, Flavor 1 is missing on WASI until shared-everything-threads ships (draft, Wasmtime #9466 open). So if SPy wants one parallel model that works identically everywhere from day one, Flavor 2 is it.

5. Why Flavor 2 must be the priority¶

1. Uniform availability today. It is the only model that works on all three targets without waiting for any unshipped proposal. Flavor 1 blocks the WASI target.

2. It removes the hardest GC+threads problem — once the collector matches the design. Because heaps are isolated, there is no shared mutable GC graph, so refcounting per isolate can be plain, non-atomic IncRef/DecRef: it never needs to scan another isolate’s memory, so there is nothing to coordinate across threads. This makes the roadmap’s “basic refcounting without cycle detection” immediately viable for parallel SPy, and it is the collector this design actually depends on. Boehm, SPy’s current interim collector, does not give this for free: a single Boehm instance shared by several OS threads in one process is a process-wide, stop-the-world collector by design — running isolates as plain threads under Boehm does not yet give independent pauses, whatever the target. The independence property described in §3 is a property of refcounting (and, later, of Whippet with per-isolate collector choice), not of Boehm. This should not block the design, since refcounting is already the intended long-term collector, but it means an early Boehm-backed prototype would not yet demonstrate the isolation benefit.

3. No dependency on shared-everything-threads for the GC story. Flavor 2 shares only plain raw_ref data over the already-shipped shared-memory/atomics proposal. The future proposal exists to share GC reference values across threads — Flavor 2 deliberately never does that.

4. Matches SPy’s no-ownership philosophy. Because GC’d objects never cross the boundary, the Rust/Mojo ownership problem is absent. SPy keeps its simple “GC + manual” model.

5. Isolation is a bonus. Enforced heap isolation gives sandboxing and plugin safety for free, on top of pure performance.

The remaining dependencies on future work are narrow:

• component.new (CM runtime instantiation) for fully self-contained isolate spawning from within the interpreter — until then, WASI spawning is host-orchestrated.

• shared-everything-threads only if SPy later wants self-contained intra-instance threads (Flavor 1 on WASI) as a fast path.

6. Workload split (the two flavors are complementary)¶

The choice between flavors is workload-driven, not target-driven.

• Native: both flavors easy today — can offer Flavor 1 as a fast path and Flavor 2 for isolation.

• WASI: Flavor 2 now; Flavor 1 later via shared-everything-threads.

• emscripten: both now (pthreads for F1, custom workers + SharedArrayBuffer for F2).

7. Status of underlying proposals¶

Everything needed for Flavor 2’s core is available today on all three targets. The two genuine future-proposal dependencies are shared-everything-threads (self-contained WASI threads, Flavor 1 on WASI) and component.new (self-contained isolate spawning); everything else, including the GC story (modulo the Boehm caveat above), works today.

Part I’s conclusion — Flavor 2 as the priority model, Flavor 1 as a workload-driven complement — is the starting point for Part II: The isolated-isolate design, which works out what an isolate actually is in compiled SPy, the gc_ref/raw_ref memory boundary, the API surface, and the kernel constraints that make the model both simple and race-free.