Persistent Reader Pool Design

Problem Statement

Once rows crossed isolates cheaply, the read path started failing in a different way. Large reads were no longer the clearest problem. Small reads were.

The flat-list ResultSet fixed the large-result transfer problem, but it exposed a different bottleneck on small reads. Every select() still spawned a fresh isolate, ran the query, transferred the result with Isolate.exit(), and terminated.

For large result sets, isolate setup was a small fraction of total time. For point queries, it dominated. A one-row lookup should not pay the full cost of creating a fresh isolate.

Background

The first persistent-pool attempt, Experiment 011, was rejected because it arrived too early. It still moved List<Map<String, Object?>> through SendPort.send(), so copying maps with repeated column keys erased the benefit of reusing workers.

Experiment 012 showed the raw trade-off more carefully. A SendPort round trip with no payload was around 6 us, while Isolate.spawn plus Isolate.exit was around 47-50 us. The pool had a real messaging advantage, but the result representation and protocol overhead had to be small enough for that advantage to survive.

This is the practical difference between a worker pool and a one-shot task helper. Dart isolates communicate by message passing, as described in the Dart isolate guide. SendPort.send lets a long-lived worker reply and then accept more work, but ordinary mutable messages may be copied. Isolate.exit can transfer ownership of the final message more efficiently, but only because the sending isolate exits. A pool therefore has to decide when keeping the worker alive is worth copying, and when the result is large enough to sacrifice the worker.

The flat-list ResultSet changed the economics. A small result was no longer a graph of repeated maps and keys. It was one compact values list plus one shared schema.

Hypothesis

A persistent reader pool should be faster if it uses two transfer paths:

Small results return through SendPort.send() so the worker stays alive.
Large results return through Isolate.exit() so the data transfers without copying, and the sacrificed worker is replaced.

This hybrid should remove spawn overhead for small reads without regressing large-result transfer.

What We Tried

Experiment 019 implemented the hybrid pool. Four reader workers stayed alive. Each query chose a path based on result size:

Below the threshold: send the ResultSet over SendPort.
Above the threshold: call Isolate.exit(replyPort, resultSet), intentionally killing the worker.

The pool then evolved through follow-up experiments:

Experiment 030 assigned each Dart worker a fixed native reader index, removing the C reader-pool mutex from the hot path.
Experiment 039 replaced a cell-count sacrifice threshold with a byte-size estimate.
Experiment 040 simplified the worker protocol to one command port, one event port, and one authoritative pending completer.

Results

The initial hybrid pool made small reads much faster:

Rows	One-off `Isolate.exit`	Hybrid pool	Path	Delta
1	0.11 ms	0.02 ms	SendPort	83% faster
10	0.13 ms	0.04 ms	SendPort	68% faster
100	0.12 ms	0.04 ms	SendPort	64% faster
500	0.24 ms	0.17 ms	sacrifice	28% faster
5,000	1.83 ms	1.76 ms	sacrifice	tied

Point-query throughput moved from roughly 14K qps to about 45K qps in that experiment.

Dedicated reader assignment then removed redundant native coordination:

Metric	Before	After
Point query throughput	37-50K qps	42-51K qps
CRUD ops/s	18-23K	18-23K
Read under write	0.30-0.41 ms	0.30-0.48 ms

The later event-port cleanup showed that simplifying the protocol also moved the target metric:

Metric	Before	After	Result
Point query throughput	101,010 qps	116,659 qps	+15%
`select()` maps, 10K rows	5.60 ms	4.84 ms	-14%

Outcome

The reader pool was accepted only after the row representation changed. The same architectural idea was a rejection with per-row maps and a win with flat results.

The lesson is that timing matters in architecture. A layer can be wrong when introduced and become right after the surrounding representation changes. Persistence helped by removing isolate setup from small reads, and later wins came from deleting coordination that persistence made redundant.

At this point resqlite had a plausible read engine. The next challenge was returning to one of the original product goals: making reads reactive without turning every write into an application-level invalidation problem.