Ingestion Strategies

PAGINATED

Deferred, not yet shipped. PAGINATED will cover large datasets with virtual view windows — a significant, separate feature built on flux.attachViewWindow() rather than an extension of LRU.

paginated ingestion — build log

$ resolving ingestion target: paginated

$ mapping virtual view windows → attachViewWindow()

$ scoping cursor-based fetch boundary

$ checking interaction with IDB byte-budget model

$ queued as a separate feature, significant scope

$ status: deferred, not yet started

$ resolving ingestion target: paginated

$ mapping virtual view windows → attachViewWindow()

$ scoping cursor-based fetch boundary

$ checking interaction with IDB byte-budget model

$ queued as a separate feature, significant scope

$ status: deferred, not yet started

The other three ingestion types — SNAPSHOT, COLLECTION_ALL, and LRU — all assume the dataset either fits in memory whole or can be capped by a byte budget with individually keyed records. PAGINATED is for the case neither of those covers: datasets large enough that even byte-budget eviction isn't the right model, and the UI itself needs a virtualized view window into a cursor-based range rather than the full set ever landing in IDB at once. That's why it's scoped as a separate feature rather than a variant of LRU — the caching model, the fetch boundary, and the store-dispatch shape all differ enough to need their own design pass.

Available today

While PAGINATED is in queue, every dataset shape it's meant to eventually cover can still be served today with one of the three shipped ingestion types — most commonly LRU for individually keyed records with a byte budget:

SNAPSHOT →COLLECTION_ALL →LRU →

Track progress on flux.attachViewWindow() on the changelog, or watch github.com/tsworldtech/flux for packages/core/src/ingestion/paginated.ts to appear.