SPy and POST Python: a comparison

This note grew out of evaluating whether SPy and POST Python (postpyc), a draft spec + reference compiler for a statically-compilable Python subset from OpenTeams, should work together. It fed into two SPy issues — spylang/spy#615 (a stable header + export manifest for SPy) and spylang/spy#616 (about additional backends for SPy) — and a proposal to POST Python itself.

What the two projects share¶

Both compile a typed Python subset ahead of time to native code while keeping source runnable as ordinary Python. Both are array-oriented, and both use a real numerical workload as a proving ground — PostSciPy for POST Python, the array API standard (NumSPy) for SPy.

SPy’s ambitions go well beyond arrays, though. Its most important proving ground is arguably self-hosting: implementing SPy’s own interpreter and compiler in SPy itself (nicknamed SPySPy). POST Python has no equivalent goal.

The root issue: primitives vs. built-ins¶

SPy gives you a small set of primitives — struct, generics, raw memory, @blue-phase metaprogramming — and builds arrays on top, as a library (NumSPy). Nothing about Array is special-cased in the compiler.

POST Python does the opposite: Array, Shape, dtypes, and @vectorize/@guvectorize are recognized by identity in the compiler frontend. There’s no primitive layer from which a user, or a second compiler, could build an array type independently — you get the array type as specified, or nothing. This is why SPy’s numerical story started with NumPy-equivalent work (NumSPy) while POST Python’s jumps straight to PostSciPy: arrays are a day-one compiler primitive, so there’s no lower milestone to hit first.

It also connects to something SPy is explicit about that POST Python isn’t: SPy is a Python variant, not a subset — it adds real new semantics (import-time freezing, @blue-phase metaprogramming, static dispatch on static types) that don’t exist in CPython, in exchange for predictable, zero-cost performance without a JIT (see Antonio’s Inside SPy, part 1). POST Python’s constraint — stay parseable as valid, unmodified Python — rules that out by design. What’s left is a fixed set of privileged abstractions with no general mechanism for anyone to build a new one with the same properties.

The spec is harder to implement than it looks¶

POST Python’s real bet is a standard with several independent compilers each conforming from scratch — its own roadmap notes says “a standard with N=1 implementations is a README; N=2 is a movement.” But conformance is heavy. §7.3 (Ownership) is a good worked example: it states a single-owner invariant, offers two incompatible mechanisms for enforcing it (static move semantics vs. runtime refcounting) without saying which is required, and neither is implemented in the reference compiler — no retain/release code, no diagnostic for a violation, unlike §7.4 (concurrency) which is at least explicit about being a debug-only check. Building this properly for real is a multi-year effort even for a single team (Rust’s borrow checker, Swift’s ARC). Asking Cython, mypyc, Numba, Codon, Pythran, and taichi to each build an equivalent, just for the baseline profile, is a lot to ask — especially for Cython/mypyc, which already get memory safety for free from CPython’s own refcounting.

Reframing: SPy as a frontend, with backends — if they’re useful¶

SPy’s redshift phase fully erases metaprogramming; the output (“red” code) is simple, static, and fully concretely typed. That’s a much smaller, closed surface to target than a full spec, which makes it realistic to reuse existing mature compilers as backends instead of asking every compiler to reimplement SPy’s semantics from scratch.

The caveat matters: this isn’t “more backends means more of a standard” — that’s the same “N=2 is a movement” logic, just with a different roster, and it’s not a goal worth chasing for its own sake. A backend is worth building only if it solves something concrete:

• Embedding/FFI — e.g. a Cython backend, replacing SPy’s current bare cffi call with proper exceptions and buffer/NumPy interop.

• Full codegen for array/GPU workloads — this is already happening, independent of any spec: Anaconda’s own Numba team is building an MLIR-based AOT compiler (numba/numbacc) that uses SPy directly as its frontend, with Antonio directly involved. It’s real convergence that happened without a spec document — a stronger argument than abstract compatibility claims about e.g. Codon’s memory model.

Separately, a documented, versioned export contract (a stable C header + JSON manifest, spylang/spy#615) is worth building regardless of any of the above — it’s useful with just the existing C backend, and POST Python’s own manifest schema (docs/toolchain.md) is genuinely reusable prior art for it.

Conclusion¶

PostSciPy — rebuilding scipy.* on a compilable, typed foundation — is a valuable goal on its own merits. But it would be much better served built on SPy: a language with real primitives and metaprogramming to build Array/DataFrame/@vectorize-equivalents as libraries, rather than a spec that has to bake each one in by hand and then ask every future compiler to reimplement it.