Introducing AtollJS
Islands-based multithreading: pools, shared memory, and worker-rendered UI under one contract
Problem. Web workers are the only real threads the browser gives you, and the platform hands you the pieces,
Worker,postMessage,SharedArrayBuffer, but no structure. Calls are untyped messages, state is either cloned or hand-offset bytes, and the moment a second file needs to agree on a layout, drift becomes a production bug.Fix. AtollJS makes the boundary a contract: one typed spec both threads import, a worker pool behind a typed proxy client, reactivity over shared fields, and, at the far end, whole framework trees rendered inside workers.
JavaScript has had workers for fifteen years, and the way most apps use them hasn't changed: pick the expensive function, post it some data, await the result. That works: until the work isn't a function call. It's a million-row scan. It's a UI tree re-rendering every frame. It's state that both threads need live, not cloned-and-stale.
AtollJS is built around four goals, each one a layer:
1. Share state, don't serialize it
postMessage clones everything: fine for results, wrong for state a
worker updates continuously. The raw alternative, SharedArrayBuffer,
is fast and gives you nothing to keep both sides honest.
So the layout is a type. defineSharedMemory takes a spec of
fixed-width fields and compiles it to a deterministic byte layout:
one declaration, imported by main and worker alike:
// incidents.memory.ts: the contract both threads import
import { defineSharedMemory, field, reef } from '@atolljs/core';
export const incidentsMemory = defineSharedMemory({
lists: {
incidents: field.list({
schema: reef.object({
id: reef.u32(),
severity: reef.int(0, 3), // domain bound → narrowest width (u8)
site: reef.string(10), // 10 inline UTF-8 bytes, schema-enforced
open: reef.boolean(), // flag byte
}),
count: 1_000_000,
}),
},
signals: { seedProgress: field.number() },
});The schema is the layout: reef mints validators the compiler reads a
byte width from, so the two sides can't disagree: there's only one
source of offsets. And it costs nothing you didn't ask for: the schema
engine is vendored into reef, so no zod dependency ships with the SDK.
Deep dive: Shared Memory Is a Contract, Not a Buffer.
2. Make worker calls feel like method calls
A worker is an RPC surface: it should look like one. defineWorker
declares the method list inside the worker; connectWorker gives the
main thread a typed proxy over a lazily-spawned pool:
// incidents.worker.ts: worker side owns the runtime
import { defineWorker } from '@atolljs/core';
export const incidentsWorker = defineWorker({
sharedMemory: incidentsMemory,
methods: {
async seedIncidents(n: number) {
/* writes straight into shared memory: results don't cross postMessage */
},
queryIncidents(q: QueryArgs) { /* scan in place, return the page */ },
},
});
// main.ts: the client is a Proxy typed by typeof worker
import { connectWorker } from '@atolljs/core';
import type { IncidentsWorker } from './incidents.worker'; // type-only
const incidents = connectWorker<IncidentsWorker>({
sharedMemory: incidentsMemory,
worker: () => new Worker(
new URL('./incidents.worker.ts', import.meta.url), { type: 'module' },
),
poolSize: 'auto',
});
await incidents.queryIncidents({ offset: 0, limit: 50 });import type matters: worker code never enters the app bundle. Under
the hood the pool handles what bare workers don't: queueing,
taskTimeout, cancellation, crash respawn. A pool call always settles.
Deep dive: Worker Pools That Fail Gracefully.
3. Let results flow back as reactivity
The payoff of shared memory is live reads: main doesn't await a return value, it watches the field the worker is writing:
import { observe, defineTask } from '@atolljs/core';
const progress = observe(incidentsMemory, 'signals.seedProgress');
// snapshot-stable, refcounted: activates on first subscriber
const queryTask = defineTask((q: QueryArgs) => incidents.queryIncidents(q));
// { data, pending, settled, elapsedMs, error }: latest-wins by defaultobserve and defineTask are the primitives; framework bindings adapt
them: useSharedValue/useTask in React, equivalents in Vue, Solid,
Svelte, Angular, Next.js. Your component subscribes; the worker writes;
the diff flows through the version counter, not a message.
Deep dive: Reactivity Without Messages.
4. Render UI itself off-thread
Push the idea to its limit and the framework tree can live in the
worker too. @atolljs/islands mounts a registered app inside a worker;
it renders against a proxy document and streams serialized DOM ops back
to a driver that replays them. A React island, end to end:
// counter.app.tsx: a plain React component; it runs INSIDE the worker.
// islandApp() stamps it with its registry key (minification-proof).
import { useState } from 'react';
import { islandApp, type EventPayload } from '@atolljs/islands/worker';
import { emit } from '@atolljs/react-island/worker';
// Worker-side handlers get the wire payload, { type, value, key },
// not a SyntheticEvent; handler() adapts it to JSX's event prop types.
const handler = <E,>(fn: (e: EventPayload) => void): ((e: E) => void) =>
fn as unknown as (e: E) => void;
export const CounterApp = islandApp('counter', function CounterApp({
label = 'count',
}: {
label?: string;
}) {
const [count, setCount] = useState(0); // state never leaves the worker
return (
<button
onClick={handler(() => {
const n = count + 1;
setCount(n);
emit('incremented', { count: n }); // island → shell channel
})}
>
{label}: {count}
</button>
);
});
// counter.worker.tsx: the whole worker entry
import { defineReactPolyWorker } from '@atolljs/react-island/worker';
export const counterWorker = defineReactPolyWorker({
apps: { counter: CounterApp }, // one worker, many islands
});
// counter.island.ts: a shell-safe contract module: registry key + worker
// factory, no worker code imported. The dynamic import is the bundler's
// split point: it (and the worker entry) only loads when the island mounts.
export const app = 'counter';
export const worker = () =>
new Worker(new URL('./counter.worker.tsx', import.meta.url), { type: 'module' });
// app.tsx: the shell is ordinary React; the facade is the boundary.
// lazyIsland mints a proxy component off the contract: attrs ARE the props.
import { Suspense } from 'react';
import { lazyIsland } from '@atolljs/react-island';
const CounterIsland = lazyIsland(() => import('./counter.island'));
export function App() {
return (
<Suspense fallback={<p>mounting worker…</p>}>
<CounterIsland
label="clicks" // serialized across postMessage
onEvent={(name) => name === 'incremented' && console.log('worker-side click')}
/>
</Suspense>
);
}
// main.tsx: an ordinary bootstrap; React on both sides, DOM on one
import { createRoot } from 'react-dom/client';
createRoot(document.getElementById('root')!).render(<App />);Render, diff, and state all run off the main thread: what's left behind
is a thin shell replaying ops. The facade wraps the same mountIsland
driver call, the contract's dynamic import is the split point, so a
heavy island's chunk only loads when it mounts, and the driver takes a
bare element on framework-free shells.
Per-framework packages (react-island, vue-island, …) keep each
worker's renderer to the framework it actually uses.
Deep dive: Islands That Render in Workers.
On the server: NestJS
Everything above is the browser story; the pool doesn't change when the
runtime does. On node:worker_threads, @atolljs/nestjs puts the pool
behind dependency injection: AtollModule.registerPool makes it an
injectable provider, and @AtollService makes a service class the
interop surface: inject it anywhere and every method dispatches:
// digest.service.ts: the decorator picks the thread
import { Injectable } from '@nestjs/common';
import { AtollService } from '@atolljs/nestjs/decorators';
import { defineSharedMemory, field } from '@atolljs/core';
export const digestMemory = defineSharedMemory({ jobsDone: field.number() });
@Injectable()
@AtollService({ pool: 'digest' }) // every method dispatches to the pool
export class DigestService {
async hash(input: string) {
// executes inside the worker's own Nest context:
// injected dependencies resolve there too
}
}
// digest.module.ts: the API-thread side: the feature module owns the
// pool and the shared memory it was built on
import { Module } from '@nestjs/common';
import { Worker } from 'node:worker_threads';
import { AtollModule } from '@atolljs/nestjs';
import { digestMemory, DigestService } from './digest.service';
@Module({
imports: [
AtollModule.registerPool({
name: 'digest',
// webpack emits the worker entry as its own chunk
worker: () => new Worker(new URL('./digest.worker.ts', import.meta.url)),
sharedMemory: digestMemory,
poolSize: 2,
}),
],
providers: [DigestService],
})
export class DigestAtollModule {}
// digest.worker.ts: the whole worker entry
import { runAtollWorker } from '@atolljs/nestjs/worker';
void runAtollWorker(DigestAtollModule);
// digest.controller.ts: injecting the service is the whole consumer story
import { Body, Controller, Post } from '@nestjs/common';
@Controller('api/digest')
export class DigestController {
constructor(private readonly digest: DigestService) {}
@Post('hash')
hash(@Body() body: { input: string }) {
return this.digest.hash(body.input); // dispatches to the pool
}
}
// app.module.ts: forRoot once; feature modules own their pools
@Module({
imports: [AtollModule.forRoot(), DigestAtollModule],
controllers: [DigestController],
})
export class AppModule {}
// main.ts: an ordinary bootstrap; nothing about it is worker-aware
import { NestFactory } from '@nestjs/core';
const app = await NestFactory.create(AppModule);
app.enableShutdownHooks(); // pools terminate on module destroy
await app.listen(3100);Each worker boots its own Nest application context, so the method body
runs with real DI on both sides: the class file is the contract, the
same discipline as every browser layer. Node needs no COOP/COEP for
SharedArrayBuffer, so shared memory comes along free; and a route
subtree can even be housed inside workers outright, proxied at the
framework boundary.
Deep dive: Dependency Injection Across the Boundary.
What ties it together
Every layer leans on the same discipline: declare once, in types both
threads share. The memory contract is a schema; the worker surface is
typeof your worker module; islands scope every mount to an app@N
instance. Nothing is duplicated across the boundary, so nothing drifts.
And the footprint follows the same rule. Dependencies stay external and
opt-in: no shared-memory fields means no schema code in your bundle at
all; islands need no SharedArrayBuffer and no special headers, so the
graceful-degradation story is real. The whole stack is what you use of
it, byte for byte.
The overview walks the three-layer architecture end to
end. For a first build, atoll new scaffolds a working island app in one
command.