Parallelism, Part II — The isolated-isolate design

Part I compared shared-heap threads (Flavor 1) against isolated isolates with a shared data arena (Flavor 2), and argued that Flavor 2 should be the priority: it is the only model available uniformly across native, WASI, and emscripten today, and isolating heaps removes the hardest GC+threads coordination problem (with the caveat, from Part I §5, that this independence is a property of refcounting and, later, Whippet — not yet of Boehm as currently used).

This file works out that design: what an isolate actually is in compiled SPy, the gc_ref/raw_ref memory boundary, the arena reclamation disciplines, the API surface, the kernel constraints, and an implementation sequencing.

1. What an “isolate” actually is in compiled mode¶

“Subinterpreter” is a CPython term and the wrong mental model for compiled SPy. A CPython subinterpreter is heavy because CPython is an interpreter: duplicating it means duplicating the module table, the __main__ namespace, the import cache, the builtins — the whole runtime state. In compiled SPy there is no interpreter to duplicate. The program is one binary (compiled C + libspy), and the kernel is a C function already linked into it. The worker doesn’t need a fresh copy of program state; it just needs to call a function that’s already there.

So on native, the isolate is not a fork, not a process, not a duplicated interpreter. It is much lighter — just “another small program with only what is needed.”

The worker is a thread + (optional) scratch memory¶

By the kernel constraints (no captures, nothing escapes, results go to shared_array or back as immutable return values — see §4), a worker running kernel(buf, 0, N//2) needs only:

1. The kernel’s code — already in the binary, shared with the main thread. Nothing to copy or load.

2. A stack — the thread’s own, free.

3. Access to the shared arena — via its shared_array args (pointers into the shared memory). Already there.

4. A place for temporaries — only if the kernel allocates (a local list, a temporary array). This is the only piece that isn’t free.

No module table, no import cache, no interpreter state — because there is no interpreter. No fork. The worker is a thread calling a function, with somewhere to put its scratch allocations.

The scratch heap needs no GC¶

Because of the kernel constraints (nothing escapes the kernel, results go to shared_array or come back as immutable return values deep-copied at join), every allocation the kernel makes dies when the kernel returns. So the private scratch heap doesn’t need a collector at all — it can be a bump allocator freed wholesale at join:

worker thread starts
  scratch = new_bump_arena()
  kernel(buf, i0, i1)        # temporaries allocate from scratch
  join barrier
  free_wholesale(scratch)   # no per-object free, no refcount decrements, no tracing
worker thread ends

No GC coordination, no cross-thread refcounts, no atomics on the scratch heap, no fork. For kernels that don’t allocate at all (pure numeric, writes straight to shared_array), it degenerates further — a bare thread with a stack, no scratch heap needed. About as light as parallelism gets, and much lighter than a CPython subinterpreter.

Uniform across the three targets, with one open question¶

The same description — shared code, private scratch memory, shared arena — holds everywhere, with different names for “private scratch memory”:

The lightness depends on the kernel constraints¶

The no-GC scratch-heap trick relies on the constraint that nothing escapes the kernel. The moment the design allows the kernel to return a mutable gc_ref, or capture a shareable that outlives the join, the scratch heap can’t be wholesale-freed anymore — something would need to escape into the parent’s heap, and you’re back to either a real private GC (Flavor 2 with a collector per worker) or a shared heap (Flavor 1). The kernel restrictions aren’t just a simplicity choice — they are what make the native compiled case this light, and lifting them would reintroduce the GC+threads problem.

2. The memory boundary: gc_ref vs raw_ref¶

The roadmap’s gc_ref/raw_ref distinction is exactly the type-level boundary Flavor 2 needs, and it is what makes the design type-safe without a borrow checker.

• gc_ref never crosses the isolate boundary. Passing one across is a type error (or a deep copy). This enforces, at compile time, the rule that PEP 734 enforces at runtime by discipline.

• raw_ref is the shareable currency. Big numeric arrays are allocated as raw_ref (unmanaged) bounded arrays (matching the roadmap’s planned pointer-to-bounded-array / slice type) and handed across by pointer + length.

• No reference value is shared across collectors → no cross-collector tracing, no cross-heap refcount coordination.

3. The arena reclamation question (no Rust-like ownership required)¶

Because the arena is outside the GC, a raw_ref buffer must be reclaimed manually. This is not an ownership-in-the-type-system problem — it is the ordinary raw_malloc problem SPy already accepts single-threaded, with a synchronization dimension added. Three conventional disciplines cover it, all implementable with primitives SPy already has (synchronization + manual free), no new type-system concept:

1. Move / handoff. The buffer is given to an isolate; the sender relinquishes access; the receiver frees (or returns it via a channel). Transfer, not sharing.

2. Borrow bounded by a barrier. Multiple isolates read (or read+write disjoint regions) for the duration of a parallel region; a join / barrier establishes “all done”; then the owner frees. Classic fork → join → free, like OpenMP and C malloc/free already coexist without a borrow checker.

3. Atomic refcount in the buffer header. For open-ended handoff, store a small atomic refcount in the arena buffer header; free at zero. This is the one place atomics reappear in the design — but it is manual refcounting of raw buffers only, opt-in, and unrelated to gc_ref refcounting (which stays non-atomic and single-threaded). It should stay a deliberate escape hatch rather than the default discipline, precisely because it’s the one place where the isolation model’s “no cross-thread coordination” property is given up.

The design rule: the high-level concurrent.isolates constructs (shared_array, queue) should own the buffer lifetimes for the user (auto-free at join, auto-handoff via queues) so most users never touch raw raw_free on the arena. Manual arena raw_malloc/raw_free remains an explicit escape hatch — symmetric to how single-threaded SPy already treats raw_malloc.

What SPy must not do: let a raw_ref arena buffer be “collected.” That would invent a cross-isolate GC and destroy the independence advantage of Flavor 2. The arena is deliberately manual; that is the price of independent heaps, and it is the right price.

4. API: follow PEP 734, with an architecture-honest name¶

SPy’s API should closely follow Python’s PEP 734 (interpreters module, landed in CPython 3.13/3.14) and concurrent.futures.InterpreterPoolExecutor, because the architecture is identical (isolated subinterpreters + queues for exchange + per-interpreter heap) and it is standard Python that SPy users will already know. No widely-adopted third-party library offers a better API; the main one is interpreters-pep-734, Eric Snow’s backport, and community patterns build on top of PEP 734 rather than replacing it.

One divergence on naming: “interpreter” is misleading for SPy in compiled mode, where the worker is a compiled libspy instance, not an interpreter.

Naming: isolate (not interpreter)¶

Use isolate as the architecture-neutral term for a parallel worker with its own private heap and GC. The term is borrowed from Dart isolates, which are the strongest well-known precedent for exactly this model (independent memory, message passing, no shared mutable heap). worker is a more conservative fallback if a less borrowed name is preferred. Either way, the rest of the API mirrors PEP 734 closely enough that familiarity carries.

Low-level API (concurrent.isolates)¶

Mirrors PEP 734’s interpreters module, with isolate in place of interpreter. Everything — create, create_queue, shared_array, IsolatePoolExecutor — lives in the single concurrent.isolates module.

A self-contained example: a kernel that fills a slice of a shared_array and returns an immutable (a frozen tuple of scalars). Immutables deep-copy across the boundary at join — the same mechanism as frozen args, run in the opposite direction — so a gc_ref never crosses.

from concurrent import isolates

def compute(i: i32) -> f64:
    return f64(i * i)

# red, top-level, no captures; returns an immutable (frozen tuple of scalars),
# which deep-copies back to the caller at join. A mutable gc_ref return would
# be rejected.
def kernel(buf: isolates.shared_array[f64], i0: i32, i1: i32) -> (f64, f64):
    s: f64 = 0.0
    m: f64 = buf[i0]
    for i in range(i0, i1):
        v = compute(i)
        buf[i] = v
        s = s + v
        if v < m:
            m = v
    return (s, m)

def main() -> None:
    N: i32 = 1000
    buffer = isolates.shared_array[f64](N)

    w = isolates.create()
    (s, m) = w.call(kernel, buffer, 0, N // 2)
    w.close()

w.call(kernel, args...) is the blocking launch form: it runs the kernel on the worker and returns its value when the kernel returns. It is the form to use when no interaction is needed while the worker runs.

For long-running workers you talk to through queues, the non-blocking form w.start(kernel, args...) / w.join() mirrors subprocess.Popen / Popen.wait(): start the worker in the background, exchange with it via queues while it runs, then wait for it to finish.

A self-contained queue example, exercising that non-blocking form. Queues are typed — queue[T] mirrors shared_array[T] — so recv() returns T statically; T must be shareable (a scalar or a frozen object), never a gc_ref. The worker drains an input queue and writes results to an output queue; a negative value is the sentinel that stops it. The send/recv alternation in main keeps one item in flight each way, so it never deadlocks and the two sides genuinely overlap.

from concurrent import isolates

# reads i32 work items from in_q, writes f64 results to out_q, stops on a
# negative sentinel. Typed queues let the compiler know every element type
# statically.
def worker(in_q: isolates.queue[i32], out_q: isolates.queue[f64]) -> None:
    while True:
        x = in_q.recv()
        if x < 0:
            out_q.send_nowait(-1.0)   # echo the sentinel, then stop
            return
        out_q.send(f64(x * x))

def main() -> None:
    in_q = isolates.create_queue[i32]()
    out_q = isolates.create_queue[f64]()

    w = isolates.create()
    w.start(worker, in_q, out_q)      # non-blocking: worker runs in the
                                      # background, like subprocess.Popen

    # feed work and drain results concurrently with the worker
    for i in range(100):
        in_q.send(i)
        y = out_q.recv()              # blocks until the worker writes one

    in_q.send(-1)                     # sentinel: tell the worker to stop
    w.join()                          # wait for the worker to return
    w.close()

send_nowait is the non-blocking send, used when you don’t want to block on a full queue — e.g. the worker pushing a final ack without stalling, or main probing before a blocking send.

Shareable rule — maps directly onto SPy’s gc_ref/raw_ref distinction, which is PEP 734’s “shareable” rule in SPy terms:

• ✅ immutable scalars, blue constant variables (compile-time constants), and shared_array (a raw_ref bounded array) — shared or copied;

• ✅ frozen (immutable) GC objects — deep-copied per worker into its scratch heap. Safe by construction: immutable, so a private copy can’t race, and it dies with the scratch heap at join (no cross-isolate reference, no lifetime question). The rule of thumb — frozen-and-small (coefficients, configs, lookup tables, index arrays) → copy; large bulk data → shared_array arena — is advisory only for now, not a checked constraint the way the other kernel-argument rules are (§5); a user can pass a large frozen object and get a silent, expensive per-worker copy. Worth a size lint at some point, but not required for a first version.

• ✅ immutable return values — a kernel may return an immutable (a frozen object or a tuple of scalars); it deep-copies back to the caller at join, by the same mechanism as frozen args above. A mutable gc_ref return is rejected (it would force the scratch heap to be collected rather than freed wholesale — §1).

• ❌ mutable gc_ref objects — rejected at call/start/send (copy or scalarize first).

shared_array[T] is the SPy analog of PEP 734’s memoryview/buffer sharing — zero-copy bulk data — but typed and arena-resident. It is the roadmap’s pointer-to-bounded-array (slice) type, living in a memory.shared arena. T must be a plain scalar (i32, f64, …), never gc_ref. queue[T] is the ringbuffer counterpart for cross-isolate exchange, with the same constraint on T.

High-level API (IsolatePoolExecutor)¶

Mirrors concurrent.futures.InterpreterPoolExecutor — the standard Python executor surface, no new keyword or syntax:

from concurrent.isolates import IsolatePoolExecutor, shared_array

def compute(i: i32) -> f64:
    return f64(i * i)

def kernel(buf: shared_array[f64], i0: i32, i1: i32) -> (f64, f64):
    s: f64 = 0.0
    m: f64 = buf[i0]
    for i in range(i0, i1):
        v = compute(i)
        buf[i] = v
        s = s + v
        if v < m:
            m = v
    return (s, m)

def main() -> None:
    N: i32 = 1000
    buffer = shared_array[f64](N)
    chunks = [(0, N // 2), (N // 2, N)]
    with IsolatePoolExecutor(max_workers=2) as ex:
        results = ex.map(kernel, [(buffer, i0, i1) for (i0, i1) in chunks])

    total = 0.0
    gmin = buffer[0]
    for (s, m) in results:
        total = total + s
        if m < gmin:
            gmin = m

ex.map(kernel, chunks) gives Mojo’s parallelize semantics — run kernel over each chunk in parallel, block until all complete, returning the list of kernel return values in chunk order — expressed as the concurrent.futures API Python developers already know. No do keyword, no custom block. The final combine loop is a plain serial reduction over the immutable partials.

Capture: forward-looking, not valid today¶

Today the kernel rule forbids any captured variable (§5). The intended long-term relaxation is to allow capturing an immutable (a blue constant) or a frozen object: both deep-copy per worker into the scratch heap and die at join, so no cross-isolate reference and no lifetime question is introduced. Capturing a mutable gc_ref would still be forbidden. The example below sketches that shape; it is not valid today.

# NOTE: forward-looking — not valid today. The kernel rule (§5) forbids any
# captured variable; this sketches the intended relaxation for immutable /
# frozen captures only.

from concurrent.isolates import IsolatePoolExecutor, shared_array

# A frozen lookup table, small enough that per-worker deep copy is cheap.
COEFFS = (0.5, -0.25, 0.125)   # frozen tuple of scalars

def compute(i: i32) -> f64:
    return f64(i * i)

def kernel(buf: shared_array[f64], i0: i32, i1: i32) -> f64:
    # COEFFS is captured; it is frozen, so each worker gets its own private
    # deep copy in its scratch heap and consults it freely.
    s: f64 = 0.0
    for i in range(i0, i1):
        v = compute(i)
        buf[i] = COEFFS[0] * v + COEFFS[1] * v + COEFFS[2] * v
        s = s + buf[i]
    return s

def main() -> None:
    N: i32 = 1000
    buffer = shared_array[f64](N)
    chunks = [(0, N // 2), (N // 2, N)]
    with IsolatePoolExecutor(max_workers=2) as ex:
        results = ex.map(kernel, [(buffer, i0, i1) for (i0, i1) in chunks])

    total = 0.0
    for s in results:
        total = total + s

Why callable passing works where PEP 734 uses source strings¶

PEP 734’s low level runs source strings (interp.exec(script_str)) because CPython cannot pass function objects across the subinterpreter boundary. SPy does not have that constraint: a top-level red function is, after redshift/linearize, a C function symbol in libspy that all isolates link against. So SPy offers the natural form w.call(kernel, args) instead of building a script string. The kernel constraints (below) are enforced by the linearizer.

5. Red/blue and the kernel rule¶

The kernel must be red¶

An isolate runs libspy at runtime. It needs a real, executable runtime function — a red function, after redshift, lowered to a libspy C symbol. A blue function is a compile-time construct; it does not exist as a callable at runtime. So:

• kernel is a red function (a normal runtime function).

• By the time ex.map(kernel, ...) runs (at runtime), kernel has already been redshifted/compiled into a libspy symbol. The executor passes that redshifted symbol to the isolate, which calls it as a normal libspy function.

Parallelism APIs are red-only (no spawning at compile time)¶

ex.map, isolates.create, create_queue are red-only APIs: they may only be called from red functions, at runtime. Launching parallel workers is a runtime action (spawning isolates, waiting on barriers); doing it from a blue function would mean spawning isolates during compilation, which is a category confusion — there is no program executing yet. Forbidding parallelism calls from blue functions is the clean, enforceable split: blue = compile-time metaprogramming; red = runtime; parallelism is a runtime (red) concern. This also means the orchestrator (the code calling ex.map) is red, not blue — everything happens at runtime.

Kernel constraints (simple, statically checkable)¶

The kernel is kept deliberately simple, and these restrictions are not temporary scaffolding to be lifted later — most of them are what make the compiled case light and race-free by construction (§1):

• kernel is a red, top-level function.

• kernel has no captured variables. Everything it needs is passed as explicit arguments. This removes the whole closure-capture analysis question.

• The only exception: blue constant variables (compile-time constants) may be used inside the kernel, since they are immutable and known at compile time — they are effectively inlined.

• All arguments to kernel are shareable: scalars (including blue constants), shared_array views, or frozen GC objects (deep-copied per worker — see §4). This is a simple type check, not a structural analysis.

• Read-only access to a whole shared_array by many workers is safe and is the idiomatic way to pass a large input that several workers must read (e.g. the input matrix of a stencil, or both operands of a blocked matmul). Reads of an immutable array never race; each worker still writes only its own disjoint output slice. So a kernel may take the whole input shared_array as a read-only arg plus a shared_array slice for its output — no atomics needed, no halo exchange machinery.

• No @parallel_work decorator or similar — the linearizer checks the constraints directly. A kernel is just a normal red function.

def kernel(buf: shared_array[f64], i0: i32, i1: i32):   # red, top-level, no captures
    for i in range(i0, i1):
        buf[i] = f64(i * i)

Sequential semantics in interp mode (today)¶

The high-level constructs are designed so the same source runs sequentially in interp mode and in parallel after compilation:

• In interp/doppler mode, ex.map(kernel, chunks) runs kernel over each chunk sequentially in the parent — a correct-if-slow result, with no parallelism. The kernel is still red; it just isn’t dispatched to isolates.

• After redshift/linearize, ex.map lowers to chunked isolates + arena + barrier.

This is the SPy “preserve the Pythonic feeling” principle applied to parallelism: write a loop body, it runs sequentially in development and in parallel in production, with no async/await coloring and no explicit thread handles in the common case.

This sequential behavior is a today-state, not a permanent design decision. The long-term goal is real parallelism in interp mode too, but that needs the interpreter itself to be a self-hosted, GIL-free program before an isolate model can apply to it (see §6, stage 3) — it is a genuinely long-term item, gated on self-hosting, and is called out separately below rather than folded into the “deferred capabilities” list, since it isn’t a capability gap in the kernel model, it’s a gap in how far that model currently reaches.

Exception propagation across the isolate boundary¶

Not addressed by the model above, and worth stating explicitly rather than leaving silent: what happens when a kernel raises inside a worker?

The intended answer is to follow PEP 734: an exception raised inside an isolate is not re-raised as-is in the parent (the original exception object doesn’t cross the boundary any more than a gc_ref does) — it is caught at the isolate boundary, and the join re-raises a new exception in the parent that carries the original’s string representation (type name + message, roughly PEP 734’s ExecutionFailed / interpreters.NotShareableError pattern).

This is, however, a genuinely long-term item, for a reason specific to SPy’s current state rather than to parallelism: every exception in SPy today leads to a panic, even sequentially — there is no exception-handling / unwinding story yet at all. Isolate-boundary exception propagation cannot be meaningfully designed in detail until SPy has a real exception model to propagate from. Until then, the practical behavior is that a panicking kernel panics the whole process, isolate or not, which is at least consistent with today’s single-threaded semantics even if not yet the target behavior above.

What is genuinely deferred (and what is not)¶

A few capabilities are not part of this design and would need separate work if ever wanted — several of them conflict with the properties above, so adding them is not straightforward:

• Reductions: write partials into a shared_array + a serial combine in the parent, or a tiny helper. Does not conflict with the model; just not wrapped in a helper yet.

• Stencils / convolutions: these read a neighborhood (halo) around each output element, so reads overlap across chunks — but they already fit via the read-only-whole-array convention above: output slices stay disjoint, and the overlapping halo reads are plain loads from a fully-populated, read-only input. No atomics, no halo exchange. Covered, not deferred.

• Blocked dense matmul (c = a @ b): also fits — every output block reads (almost) all of a and b, but reads are read-only and outputs are disjoint. The only loss vs a shared-heap (Flavor 1) implementation is some cache efficiency, a performance gradient not a capability gap.

• Closure capture of shareable variables: conflicts with the “no captures” rule that keeps the boundary trivially safe. If ever added, the intended relaxation is capture of immutable / frozen shareables only (see the forward-looking capture example in §4); mutable capture would still be forbidden. It requires the captured shareables to outlive the join, which reintroduces lifetime questions.

• Returning a mutable gc_ref from the kernel: conflicts with “nothing escapes the kernel.” (Returning an immutable is supported — see §4.) A mutable return would force the scratch heap to be collected rather than freed wholesale (§1), reintroducing the GC+threads problem.

• Nested parallelism — orchestration calling parallel compute (supported). A long-lived orchestrator thread that calls a numerical function which itself does ex.map is a common and important pattern, and it composes correctly: nested ex.map is just nested function calls, each call’s workers get their own scratch heaps freed at that call’s join, and the caller’s state is untouched. In compiled mode pthread_create (and the target spawn shims) work from any thread, so correctness needs no new machinery. The one piece of real work is a shared worker pool: a fresh-spawn-per-call is correct but pays thread-creation cost each time and can oversubscribe when an outer worker stays busy while inner workers run. A fixed pool of num_cores workers (the natural shape of IsolatePoolExecutor), drawn from by nested and sibling calls, with work-stealing so a blocked outer worker helps out while it waits, removes both problems — standard thread-pool engineering, not a research problem.

• Nested parallelism — a kernel internally calling ex.map (left out). A kernel is meant to be a leaf compute function (no captures, nothing escapes, scratch freed at join). A kernel that spawns sub-kernels is no longer a leaf, its scratch-heap lifetime must span the nested join, and it opens recursive oversubscription / scratch-heap-tree questions for little benefit — the useful pattern above is orchestration calling parallel compute, not a kernel recursively forking. Kernels stay leaves.

• Tightly-coupled linear algebra (blocked LU/QR/Cholesky factorizations, sparse direct solvers, some iterative solvers): these have genuine sequential dependencies and shared mutable state during the computation. Blocked versions parallelize a panel step and a trailing-matrix update, but the update mutates shared state with complex synchronization — a shared-heap (Flavor 1) problem at heart. Flavor 2 can only parallelize the embarrassingly-parallel trailing update and run the sequential parts on one worker. This is the one workload class that genuinely wants Flavor 1’s shared mutable heap, and it is the concrete case for eventually having Flavor 1 available too.

The point of listing these is honesty: the design is intentionally restrictive, and the restrictions are load-bearing. They are not a “v1” to be replaced by a “v2”; most of them are what make the model simple, light, and safe.

6. Sufficiency for the numerical stack (NumSPy / SciSPy)¶

This is the natural stress test, and the fit is good:

• NumSPy (a NumPy reimplementation): Flavor 2 is sufficient. Element-wise ops, reductions, stencils (via the read-only convention), and blocked matmul all fit. The only loss vs a Flavor 1 implementation is some cache efficiency on dense matmul — a performance gradient, not a capability gap.

• SciSPy (a SciPy reimplementation): sufficient for most, with one weak spot. Signal processing, interpolation, most of statistics, and the simpler scipy.linalg ops decompose into the covered patterns. The genuine weak spot is the tightly-coupled factorizations and sparse direct solvers above — the LAPACK-style core of scipy.linalg — which want shared mutable state. Those specific routines are where Flavor 1 (as a later complement) would earn its place.

So the restricted model is more sufficient than it first looks for the numerical stack — the read-only-array convention and the frozen-copy args quietly cover a lot — with dense linear algebra’s tightest kernels as the one place you’d eventually want the shared-heap option.

7. From ex.map to target-specific code: linearizer, then libspy¶

Two different stages do two different jobs here, and it matters to keep them separate. SPy’s real pipeline is redshift → linearize → cwrite → compile: linearize turns the redshifted AST into a flat, imperative, target-agnostic form (the general structured-AST-to-basic-blocks lowering any AST-to-C pipeline needs — nothing parallelism-specific about the stage itself); cwrite turns that into one C source; compile links that C source against libspy, a small C runtime that is built separately per target (native, WASI, emscripten) and only then linked in. The target only exists from compile onward — at linearize time, there is no target yet.

So for ex.map(kernel, chunks) with a red, capture-free kernel whose args are shareable (§5), the two stages contribute two different things:

What the linearizer does — target-agnostic, one expansion for all targets. It expands the high-level construct into a flat, generic sequence: still just AST/C, no target-specific primitive anywhere. Conceptually:

1. Sets up the shared arena for the shared_array arguments (if not already created) — a raw_ref region accessible by all workers.

2. Calls a generic libspy primitive to spawn one worker per chunk — e.g. spy_isolate_spawn(kernel_fn, args, ...). This call is the same on every target; which C code it actually runs is a libspy-build-time question, not a linearizer question (see below). Each worker is the “thread + private scratch” unit from §1: it runs the redshifted kernel symbol against its chunk’s args, allocating temporaries from its own bump-arena.

3. Runs the kernel body — plain compiled SPy code: scalar loads, shared_array element reads/writes, arithmetic. Because the chunks are disjoint (§5, §3), the element accesses are plain non-atomic loads and stores — no atomics needed on the hot path. The kernel never touches another worker’s scratch heap (it can’t; those are private), and its only cross-worker-visible writes go to its own disjoint slice of the arena.

4. Calls a generic libspy join/barrier primitive that waits for all workers, then publishes the arena writes to the parent (establishes happens-before) and frees each worker’s scratch heap wholesale (§1). No per-object free, no refcount decrements, no tracing.

This expansion is written once, and it’s genuinely target-agnostic: wasm atomics, C11 stdatomic, and JS Atomics share portable memory-ordering semantics, so even the fence placement in the generated C doesn’t need to vary per target.

What libspy does — the actual spawn/wake shims, resolved at build time, not by the linearizer. spy_isolate_spawn and the barrier’s wake/wait calls are ordinary C function signatures that every target’s libspy implements, but the bodies differ, and which body ends up in the binary is decided by which libspy.a/.wasm gets built and linked for --target native / --target wasi / --target emscripten:

• spawn:

  • native: pthread_create (the worker is a thread + thread-local bump arena);

  • WASI: the host instantiates a fresh wasm instance with its own linear memory (later, component.new for self-contained spawning — Part I §7);

  • emscripten: new Worker with its own wasm instance.

• wake/wait:

  • native: sem_post/sem_wait (or pthread_cond);

  • WASI: memory.atomic.notify/wait;

  • emscripten: Atomics.notify/wait.

So the “structural advantage” here is really two separate facts stacked on top of each other: the linearizer only ever has to know about one, generic libspy parallel API (no target branching in the AST lowering itself), and libspy’s existing per-target build setup (already required for everything else it does) is what supplies the three different implementations of that API. A worker on any target is the same “shared code + private scratch + shared arena” shape from §1; only the libspy build backing spy_isolate_spawn/wake differs — and, on WASI/emscripten, resolving the open question from §1 (multi-memory vs. host-mediated arena access) is itself a libspy-implementation decision, not a linearizer one.

Relation to interp mode: in interp/doppler mode, ex.map is not lowered by the linearizer at all — it runs sequentially in the parent as a plain loop (§5), calling kernel directly rather than expanding to libspy calls. The linearizer expansion above only happens on the path to compiled output, which is consistent with parallelism being, for now, a runtime (red) concern realized at compile time.

8. Implementation sequencing¶

The difficulty of implementing the model is not uniform across the targets: the compiled-native case is genuinely the simplest, and the difficulty ordering coincides with a real dependency ordering. This is a sequence — the order in which each case becomes tractable and worth building — not a feature roadmap with dates.

1. compiled-native (first; does not depend on self-hosting)¶

In compiled-native mode every ingredient is already there and is the lightest version of itself: the kernel is a C function already linked into the binary (no marshalling, no symbol resolution); a worker is pthread_create + a thread-local bump arena (no interpreter state to duplicate, no process, no wasm instance — the §1 model in its lightest form); the shared arena is mmap(MAP_SHARED) or a heap buffer with atomics (the most mature synchronization substrate of the three); the API lowers to plain C calls, nothing async, nothing crossing a language boundary.

Crucially, this case does not depend on self-hosting at all: the compiler pipeline (redshift → linearize → cwrite → C) already produces a native binary that links libspy today. So a preliminary version with the full API and real multi-core parallelism, running on native, can be built and demoed before any self-hosting work lands — real CPU parallelism, the PEP 734-style API, the scratch-heap-freed-at-join model, on hardware that exists now.

The one implementation note for this stage: even though only native is implemented, give spy_isolate_spawn and the wake/wait calls (§7) a stable signature in libspy from the start, with only the native .c body filled in. This costs nothing now and means stages 2 and 3 add a WASI/emscripten libspy build implementing the same signatures, rather than reshaping a hardcoded native-only API later — the linearizer’s expansion (§7) never has to change at all.

2. compiled-WASI + emscripten (next)¶

Each of these adds exactly one libspy build implementing the spawn/wake primitives from §7, plus resolving the arena question from §1: WASI adds host-orchestrated wasm-instance spawning, a memory.shared arena (once the §1 open question on private-vs-shared memory is resolved), and memory.atomic.notify/wait; emscripten adds new Worker spawning and a SharedArrayBuffer/Atomics arena. The linearizer’s expansion and the kernel model are unchanged — this stage is purely about writing the WASI and emscripten bodies of the same libspy functions the native build already implements. component.new (self-contained spawning) is not required at this stage — host-orchestrated spawning is fine (Part I §7).

3. interp-mode parallelism (last; long-term, needs self-hosting)¶

Today, ex.map in interp mode is deliberately sequential (§5) — it is not expanded by the linearizer into libspy calls (§7), so there is no spawn/join to speak of. The long-term goal is different and harder: give interp mode real parallelism too, not just the compiled path. That requires the interpreter itself to stop being a problem for parallelism, and today it is one on two counts — it’s CPython-hosted (GIL-bound), and a not-yet-redshifted kernel is dynamically dispatched, not a linked C symbol, so “run the kernel on a worker” isn’t “call a linked C function,” it’s “run interpreter code in a separate isolate.”

Self-hosting removes both obstacles at once: once the interpreter is itself a compiled WASI/native program calling libspy, it is the compiled case, and interp-mode parallelism collapses back into the already-solved compiled-native / compiled-WASI implementation from stages 1–2. Building this before self-hosting would mean solving the hard version (making a CPython-hosted interpreter spawn real parallel workers) for a result that self-hosting would then obsolete; building it after means mostly reusing the compiled implementation. That is a strong argument for this ordering, not just a convenience one — the difficulty ordering and the dependency ordering coincide, and this stage is realistically long-term, tracking the self-hosting milestone rather than anything on a nearer horizon.

Open questions¶

Collected here for visibility — none of these block the overall priority (Flavor 2 first) or the sequencing (native → WASI/emscripten → interp mode, §8), but each needs a decision before the relevant stage is implemented:

• Collector for isolate independence (Part I §5, point 2). The “independent GC, no global pause” property holds for refcounting (and later Whippet), not for Boehm as currently used with plain OS threads in one process. Decide whether an early native prototype uses refcounting specifically, or whether it’s acceptable to prototype with Boehm and defer the independence claim.

• Arena memory model on WASI/emscripten (§1). Multi-memory (private memory 0 + imported shared arena memory) vs. host-function-mediated arena access are different implementation strategies with different hot-path costs. Needed before stage 2 (§8).

• Enforcement of the “frozen-and-small” copy rule (§4). Currently advisory; consider a size lint if silent large copies turn out to be a real footgun in practice.

• Exception propagation (§5). The intended shape (PEP 734-style reraise with a string payload) is decided; the work is blocked on SPy getting a real exception/unwinding model at all, which is a pre-existing gap independent of parallelism.