Shared Memory Is a Contract, Not a Buffer
Why AtollJS's `SharedArrayBuffer` layer starts with a schema
Problem.
SharedArrayBuffergives you raw bytes and nothing else — byte offsets duplicated across thread boundaries drift silently, and the first mismatched constant is a corruption bug that throws nothing.Fix. Declare the layout once, as a schema:
defineSharedMemorycompiles a deterministic fixed-width layout that both threads import, so the two sides can't disagree.
If you've ever allocated a SharedArrayBuffer and then written
Float64Array offsets into a comment so both files agree — you already
know the failure mode this post is about.
postMessage at least serializes for you — slow, but correct. Raw
shared memory is fast and gives you nothing to keep the two sides
honest. The platform hands you a slab of bytes; the layout, the
versioning, the cross-thread agreement are yours to invent. Most code
invents it as constants in two files that drift the first time someone
adds a field.
AtollJS makes the layout a type. This is defineSharedMemory:
export const incidentsMemory = defineSharedMemory({
lists: {
incidents: field.list({
schema: reef.object({
id: reef.u32(), severity: reef.int(0, 3), status: reef.int(0, 2),
site: reef.string(10), /* … */
}),
count: 1_000_000,
}),
},
signals: { seedProgress: field.number() },
});One declaration, imported by the main thread and every worker. The spec compiles to a deterministic byte layout — same offsets everywhere, because there's only one source of them. Fork it and you're back to comments; import it and drift is impossible.
Fixed width is the feature
A SharedArrayBuffer can't grow, so every field is fixed-width — which
sounds like a limitation until you see what it composes into:
| Field | Storage |
|---|---|
field.number() |
8-byte f64 |
field.string({ schema }) |
length header + inline UTF-8, budget from reef.string(n) |
field.array({ schema }) |
count + bounded inline elements |
field.list({ schema, count }) |
fixed-layout record array |
reef.object members are fixed-width, so a million-record field.list is
a flat array of structs — readAt(i) and writeAt(i) are pointer
arithmetic, not parsing. The benchmark island scans and sorts a million
incidents through this layout. There's no decode step because there's
nothing to decode.
And every field carries a version counter — the mechanism the reactivity post builds on.
What the contract buys you
- Zero-copy reads.
read()on a typed-array field hands back a live view. State never crossespostMessage. - One source of truth. Both threads speak the same layout because they import the same object.
- Introspection.
memory.fields()reports{path, byteOffset, byteLength}for every slot — powering debugging and the Redis adapter's field-by-field mirroring covered in the server-side posts.
The takeaway: the buffer is plumbing; the contract is the API. Pools, reactivity, islands, persistence — everything else in this framework stands on that single sentence.
Source: the shared-memory guide and the reef schema reference; the persistence half lands in the Node.js docs.