002: Embedded-first - the engine is a library before it is a cluster¶
Status: adopted; the embedded path is a gated, benchmarked product surface, not a test harness.
Context¶
Stateful stream processing has historically forced a choice. Lightweight libraries start instantly and embed anywhere, but skip the hard correctness work: no event time, no checkpoints, no keyed state that survives a restart. Cluster platforms do that work, but demand daemons, orchestration, and operational ownership before the first result appears. The gap between "I have a stream and a laptop" and "I have a correct pipeline" is where most streaming projects stall.
The two halves are not actually in tension: watermarks, barriers, and keyed state are properties of the dataflow graph, not of a deployment topology.
Decision¶
Build one engine that is a library first:
- The whole engine lives in
libclink.clink run pipeline.sqlexecutes a pipeline in a single process with no daemons; the same binary embeds behind a pure-C ABI, andpyclinkexposes it to Python with Arrow-native results. - The distributed runtime is a consumer of the same engine, not its home: a Coordinator/Worker control plane deploys the same operator DAGs, and the same SQL file submits unchanged with parallelism, failover, and rescale.
- The embedded path carries full semantics - event time, keyed state, checkpoints, the same state backends - so graduating to a cluster changes where a pipeline runs, never what it computes.
- Cold start is treated as a feature with a regression gate: a Release-only test budgets time-to-first-row for the embedded path (~155 ms measured), so the library promise cannot silently rot.
Consequences¶
- The adoption path has no cliff: prototype in-process, embed in a service, or submit to a cluster, all with one artefact and one semantics.
- Engine internals must never assume a resident cluster: anything the distributed runtime needs (config, registries, state directories) arrives through explicit context rather than ambient infrastructure. This discipline is what keeps jobs deployable as plain compiled plugins.
- The trade-off accepted: two first-class execution paths must both stay correct and tested. The test suite runs the same operator and state contracts through the local executor and the cluster runtime, which is a permanent, deliberate cost.
See Embedded execution for how the path works today.