About SPy package distribution

What a SPy wheel might contain¶

A hypothetical mypackage SPy wheel would likely need three kinds of artefact:

.spy source files. The compiler needs these for cross-module type checking and redshifting — for the same reason C libraries ship header files. Without source, the SPy compiler cannot specialise calls into the package.

A precompiled .wasm library. This is the portable artefact. A .wasm file compiled from SPy→C→clang is genuinely OS- and architecture-agnostic: the same file runs on Linux, macOS, Windows, and in the browser. This is already how libspy.wasm itself works.

A native Python extension (.so / .pyd). For the Python binding layer, this is unavoidably platform-specific. Separate wheels would be needed for each OS/architecture combination, exactly as any C-extension package today. The wheel filename would encode this in the standard way: mypackage-1.0-cp312-cp312-manylinux_2_17_x86_64.whl.

This creates an interesting asymmetry: the SPy-to-SPy consumption path could in principle use a single portable wheel (.spy source + .wasm), while the Python binding forces the familiar platform-specific wheel proliferation.

Using a SPy package from SPy: two models¶

Interpreter path (WASM-to-WASM). In interpreted mode the SPy interpreter already loads .wasm modules via wasmtime. A package’s precompiled .wasm could be loaded as a separate wasmtime instance, with data exchanged through shared memory. No recompilation needed; full portability preserved.

Compiled path (AOT build). When producing a native binary with spy build, two sub-options exist:

• Recompile from .spy source: the compiler has full information, can inline and specialise across module boundaries, and produces the best possible output. Requires the full compiler toolchain at build time.

• Link the precompiled .wasm: clang can link WASM object files into a native binary via wasm-ld. Faster builds, but cross-module optimisation is limited.

The most likely outcome is that SPy will support both — use precompiled .wasm for fast and portable deployment, allow source recompilation for maximum performance. This mirrors the C world’s header + .a distribution model, and it matches the spirit of SPy’s redshifting philosophy.

Open design questions¶

Several important questions have no answer yet:

Cross-module redshifting. If mypackage.add(x, y) can be specialised for i32 arguments, does that specialisation happen at package-compile time (producing multiple .wasm variants) or at application-compile time (requiring source)? This is a fundamental trade-off between distribution convenience and runtime performance.

Python binding generation. There is currently no mechanism in SPy for auto-generating a Python extension from SPy code. This would require something like a CFFI or pybind11 equivalent for SPy — a significant piece of work that has not yet been designed.

PyPI WASM wheel support. At the Python packaging level, there is no agreed wasm32-unknown-wasi platform tag for PyPI wheels. Discussions have happened but no consensus has been reached.

The libspy.wasm versioning problem. A package’s precompiled .wasm is linked against a particular version of libspy. As libspy evolves, ABI compatibility between a package wheel and the installed SPy runtime will need to be managed — a solved problem in the C world (soname versioning) but not yet addressed for SPy.