From an in-memory cache & pub/sub layer

1// The pattern this page is about — not any single product, but the
2// architecture nearly every app converges on once page load feels
3// slow: a hosted in-memory key-value store sitting between your app
4// server and your database, used for two separate jobs at once.
5 
6// Job 1 — read-through cache, so repeat requests skip the database:
7const cached = await cache.get('services:list')
8if (cached) return JSON.parse(cached)
9 
10const services = await db.services.findMany()
11await cache.set('services:list', JSON.stringify(services), 'EX', 300)
12return services
13 
14// Job 2 — pub/sub fan-out, so connected clients hear about changes
15// live instead of polling:
16await cache.publish('services:updated', JSON.stringify(newService))
17 
18// Both jobs work. Both also mean every page load and every live
19// update now depends on a THIRD piece of infrastructure being up,
20// reachable, and fast — in addition to your app server and your
21// database.

1// What this architecture actually costs, that doesn't show up until
2// you're paying for it at scale:
3 
4// 1. A network hop your user pays for on every cache HIT, not just
5//    misses. The cache is almost never in the same rack as your
6//    edge function — it's one more round trip between "request
7//    arrives" and "response leaves," every single time.
8 
9// 2. Connection overhead in serverless/edge environments. A
10//    traditional in-memory store expects long-lived TCP
11//    connections. Serverless functions don't have long-lived
12//    anything — you're either paying for a connection-pooling proxy
13//    layer, or eating a fresh connection cost on every cold start.
14 
15// 3. Cache stampede. TTL expires, 200 concurrent requests all miss
16//    at once, all hit the database simultaneously, and now your
17//    "cache" just multiplied your database load for one bad second.
18 
19// 4. Pub/sub fan-out cost that scales with connected clients, priced
20//    by the provider, whether or not those clients are actively
21//    looking at their screen.
22 
23// 5. None of this ever runs offline. A cache miss with no network
24//    is just... nothing. No stale-but-usable data, no queued
25//    retry — the request fails exactly like it would with no cache
26//    layer at all.

1// BEFORE — server-side read-through cache, re-fetched by every
2// client on every page load, invalidated by convention (hope
3// everyone remembers to bust the key on write):
4 
5// server
6app.get('/api/services', async (req, res) => {
7  const cached = await cache.get('services:list')
8  if (cached) return res.json(JSON.parse(cached))
9 
10  const services = await db.services.findMany()
11  await cache.set('services:list', JSON.stringify(services), 'EX', 300)
12  res.json(services)
13})
14 
15// client
16const { data } = useQuery(['services'], () =>
17  fetch('/api/services').then(r => r.json())
18)
19 
20// ────────────────────────────────────────────────────────────────
21 
22// AFTER — the cache moves to the browser that's actually going to
23// read the data. One bootstrap call per session populates every
24// registered store; every render after that reads from IDB with no
25// network hop, no matter how many times the page is revisited.
26 
27flux.register({
28  store: servicesStore,
29  channel: 'services',
30  idbKey: 'services:list',
31  ttl: 'medium',
32  ingestionType: 'COLLECTION_ALL',
33})
34 
35flux.bootstrap({
36  endpoint: '/api/bootstrap/public',
37  map: { services: servicesStore },
38})
39 
40// server — same query, no cache client, no key naming scheme to
41// keep consistent across a codebase:
42export async function GET() {
43  const services = await db.services.findMany()
44  return Response.json({ ok: true, data: { services }, assets: [] })
45}

1// BEFORE — server publishes a change, a pub/sub layer fans it out,
2// every connected client re-fetches or manually patches local state:
3 
4// server, on write
5await db.services.update(id, changes)
6await cache.publish('services:updated', JSON.stringify({ id, changes }))
7 
8// client, one subscriber per tab, every tab a fresh connection
9const sub = cache.duplicate()
10await sub.subscribe('services:updated')
11sub.on('message', (_, msg) => {
12  const { id, changes } = JSON.parse(msg)
13  queryClient.setQueryData(['services'], (old) =>
14    old.map(s => s.id === id ? { ...s, ...changes } : s)
15  )
16})
17 
18// ────────────────────────────────────────────────────────────────
19 
20// AFTER — one realtime adapter, one connection PER BROWSER SESSION
21// regardless of how many tabs are open (Section 4.4's leader
22// election collapses N tabs to 1 socket), normalized events dispatch
23// straight into the registered store:
24 
25const flux = createFlux({
26  adapter: createSupabaseAdapter(supabaseClient), // or socket.io / SSE / raw WebSocket
27})
28 
29flux.register({
30  store: servicesStore,
31  channel: 'services',
32  event: 'UPDATE',
33  idbKey: 'services:list',
34  ttl: 'medium',
35})
36 
37// server, on write — same query, no pub/sub client call at all if
38// your realtime adapter reads directly off the database's own
39// change stream (Postgres logical replication, MongoDB change
40// streams, etc.) rather than requiring an explicit publish step:
41await db.services.update(id, changes)

1// Cache stampede specifically — worth calling out on its own,
2// because it's the failure mode that only shows up under load,
3// usually during a launch or a traffic spike, which is the worst
4// possible time to discover it.
5 
6// BEFORE — no protection unless you build it yourself: a
7// distributed lock around the cache-miss path, a stale-while-
8// revalidate wrapper, or a separate warming job. All solvable, all
9// extra infrastructure you now own and maintain.
10 
11// AFTER — structurally not a thing that can happen. Bootstrap
12// leader election (Section 4.4) means exactly ONE tab per browser
13// session ever performs the network fetch; every other tab hydrates
14// from IDB on the same page load. At 10,000 concurrent users, your
15// database sees the shape of 10,000 individual sessions bootstrapping
16// once each — never 10,000 requests racing a single expired key at
17// the same instant, because there is no single shared key to race.

1// The one thing worth flagging explicitly if your existing cache
2// layer scopes keys per-user (a common pattern: user:{id}:dashboard,
3// invalidated on logout) — this has a direct equivalent, not just a
4// rough approximation:
5 
6flux.register({
7  store: dashboardStore,
8  channel: 'dashboard',
9  idbKey: 'dashboard',
10  ttl: 'short',
11  scope: 'user', // namespaced per userId in IndexedDB, structurally
12                 // isolated from every other tenant on the browser
13})
14 
15// on logout
16await flux.clearUserSession()
17// aborts in-flight user-scoped fetches, sweeps this user's IDB
18// cache, clears their queued mutations, tracked jobs, and sync
19// anchors, and gossips the teardown to every other open tab —
20// see core-concepts/multi-tenant-scoping for the full model,
21// including the boot-time sweep that catches sessions that never
22// call this at all (a crashed tab, a closed laptop lid).

1// What a browser-native cache layer does NOT replace — worth being
2// explicit about, since "get rid of your cache layer entirely" is
3// not the actual claim here:
4 
5// Distributed locks — coordinating exclusive access ACROSS servers.
6// Flux's Web Locks API usage (Section 4.4) coordinates tabs within
7// ONE browser, not server processes across a fleet.
8 
9// Server-side job queues and background workers — a queue that
10// needs to survive independently of any browser being open.
11 
12// Server-side session storage — auth state that must be validated
13// on the server regardless of what any client claims.
14 
15// Rate limiting and DDoS protection — this has to live in front of
16// your API, not inside a state-sync library running in the client.
17 
18// If your in-memory store is doing any of the above alongside
19// UI-facing caching and pub/sub, only the UI-facing half is what
20// this migration addresses. Keep the store for the rest.

1// Signals this migration is a bad fit, at least for the endpoint in
2// question — worth checking before ripping anything out:
3 
4// - The data is genuinely private per-request and never re-read by
5//   the same client twice (a one-time computed value, a signed URL).
6//   Nothing about repeat-read caching helps something that's read
7//   once.
8 
9// - The "cache" is actually being used as a lock or a queue, not a
10//   cache — see the "what stays" list above.
11 
12// - The data must never persist on the client at all for compliance
13//   reasons, even encrypted at rest in IndexedDB. (AES-GCM cache
14//   encryption is on the roadmap — Section 15 — but isn't a V1
15//   answer to this today.)
16 
17// - You need a shared cache ACROSS different users viewing the same
18//   data at the same time on different devices, and staleness
19//   tolerance is near zero. Browser-native caching is per-device by
20//   design — that's the whole mechanism. A pubsub-fed realtime
21//   channel closes most of the staleness gap, but the cache itself
22//   is never shared across devices the way a hosted store is shared
23//   across servers.