Capability catalogue

The complete shipped feature surface, one area per section, each row linking to the page that documents it in depth. A capability appears here only when it is backed by code and tests in the repository; config-gated or partial features carry their caveat in the row. The README status tables remain the fine-grained per-subsystem inventory; this page is the navigable map.

Execution model

Capability	Notes	Reference
Typed operator DAGs	Map, filter, reduce, keyed process, branching (fork), union with barrier alignment	Operator model
Fluent API	Pipeline / DataStream<T> / KeyedDataStream<T> builder chain	Operator model
Operator parallelism	Per-subtask keyed-state isolation, hash-partitioned shuffle	Jobs and scheduling
Backpressure	Bounded channels end to end, in-band flow control	Task lifecycle
Columnar execution	Operators opt in via process_columnar(); Arrow sidecar batches flow through shuffles without row materialisation	Columnar execution
Async state execution	Off-thread state reads with deadline-based resume and read coalescing for deferring backends	Async state execution

Time, windows, CEP

Capability	Notes	Reference
Event time and watermarks	Assigner strategies: monotonic, bounded out-of-orderness	Time and windowing
Windows	Tumbling, sliding, session; custom triggers (tumbling/sliding), allowed_lateness, late output tags. No evictors	Time and windowing
Interval join	Keyed stream-stream join, all 8 join types, watermark-driven eviction, late-arrival policy	Time and windowing
Complex event processing	NFA-based Pattern DSL: linear patterns, greedy quantifiers, strict/relaxed contiguity; reachable from SQL MATCH_RECOGNIZE	Time and windowing

SQL

Built behind CLINK_BUILD_SQL=ON (default off). One SQL file runs embedded or submits to a cluster, unchanged.

Capability	Notes	Reference
DDL and catalog	CREATE TABLE ... WITH (connector=...), session or directory-persisted catalog, SHOW/DROP, EXPLAIN with row estimates, ANALYZE TABLE statistics	SQL frontend
Queries	Projection and filtering with three-valued null semantics, expressions, aggregates, GROUP BY (including windowed), HAVING, ORDER BY / LIMIT, subqueries, DISTINCT, top-N	SQL frontend
Joins	Stream-stream interval joins, multi-way INNER joins with cost-based reordering (applied only when estimated cheaper), lookup joins	SQL frontend
Types	BIGINT/DOUBLE/VARCHAR/BOOLEAN/timestamps, DECIMAL (exact, 128-bit), ARRAY/MAP/ROW, MULTISET	SQL frontend
Pattern matching	MATCH_RECOGNIZE v1 on the CEP engine	SQL frontend
Extensibility	Scalar UDFs (LANGUAGE SQL and native), UDAFs, the Table API producing the same JobGraphSpec	SQL frontend
SQL-native ML	CREATE MODEL / ML_PREDICT (HTTP, ONNX opt-in, native closures), VECTOR_SEARCH, full-refresh materialized tables	SQL frontend

State

Capability	Notes	Reference
Keyed and broadcast state	keyed_state<K,V> / broadcast_state<V> on the operator RuntimeContext	State and backends
Backends	In-memory, file-backed, changelog (WAL + materialisation), RocksDB (always built), ForSt (opt-in) including object-store-resident variants	State and backends
Open snapshot format	Snapshots are documented Arrow IPC; checkpoints open directly in pyarrow, DuckDB, Polars; Parquet and Iceberg export	State snapshot format
Queryable state	Live keyed state served over HTTP: JSON point lookups, whole-slot scans, Arrow IPC streams; one job can SELECT from another job's live state	State and backends
Savepoints and state processor	Offline savepoint read and transform API	Fault tolerance and rescale
Schema evolution	Migrate-at-restore with a migration registry and a pre-deploy compatibility gate	Fault tolerance and rescale

Delivery guarantees

Capability	Notes	Reference
Checkpointing	Chandy-Lamport barrier alignment; unaligned checkpoints at multi-input operators; async snapshot workers; fsync-gated acks	Checkpointing
Exactly-once sinks	Generic committer (prepare at barrier, commit on global durability, recover-and-re-commit): file, Kafka, Parquet, raw S3 multipart, Postgres PREPARE TRANSACTION	Sink committer framework
Effectively-once upserts	Changelog upsert and delete by PRIMARY KEY: Postgres, MySQL, Cassandra, Redis	Sink committer framework
Source replay	Source-offset recovery generalised across connectors	Checkpointing

Scale and operations

Capability	Notes	Reference
Cluster runtime	Coordinator/Worker control plane over a length-prefixed TCP protocol; jobs deploy as compiled plugins or SQL	Distributed runtime
Failover	Lost-worker detection, drain, redeploy from the latest completed checkpoint (fail-fast by default, config-gated restarts)	Fault tolerance and rescale
Rescale	Key-group repartitioning on deploy, scale up and down, per-operator rescale over HTTP; load-driven autoscaler trigger	Fault tolerance and rescale
High availability	Multi-coordinator leader election via etcd (opt-in); filesystem-backed job persistence	Distributed runtime
Security	TLS and mTLS on the cluster transport	Network stack
Kubernetes	Helm chart and a ClinkCluster/ClinkJob operator with savepoint-on-upgrade	Distributed runtime
HTTP API and console	JSON API, Prometheus metrics, SSE events, embedded dashboard; the full operations console is a separate project	Distributed runtime
Efficiency	Measured 3.87x less CPU per event and 15.5x less memory than a JVM stream processor on identical hardware with byte-identical output; raw per-run data published	Efficiency and environmental impact
Allocator choice	jemalloc as the process allocator, opt-in at build time (CLINK_WITH_JEMALLOC=ON, Linux); measured at +5% throughput on a windowed query, neutral elsewhere, no memory change. The allocator in use is reported by clink_node --version and at node startup	Build options

Observability and debugging

Capability	Notes	Reference
Metrics	Counter and gauge registry, Prometheus exposition, per-process system gauges	Distributed runtime
Structured logging	clink::log facade with an in-memory ring served over HTTP and zstd-rotated files	Distributed runtime
Data lineage	Per-job source/sink dataset graph with column-level lineage for SQL; built-in OpenLineage exporter	Data lineage
Deterministic replay	Flight recorder captures per-epoch operator input; clink replay re-executes byte-identically offline and can freeze an incident into a regression test	Replay determinism

Embedding and APIs

Capability	Notes	Reference
Embedded engine	clink run pipeline.sql: one process, no daemons; first result in ~155 ms (gated by a release test)	Embedded execution
C ABI	libclink embeds the engine behind a pure-C ABI with Arrow C stream results	Embedded execution
Python	pyclink returns results as pyarrow tables	Embedded execution
Arrow wire format	Every operator-to-operator data frame is an Arrow IPC stream; columnar schemas for built-in types, binary fallback for user types	Network stack
Testing framework	Public clink::test harnesses: state inspection, snapshot/restore, failure injection, MiniCluster	Testing framework

Connectors

Twenty-plus sources and sinks, each documented with dependencies, factory names, options, and SQL usage in the connector catalogue: Kafka, Postgres (snapshot, CDC, sink), MySQL, ClickHouse, Cassandra, MongoDB, Redis, S3 and S3 Parquet, GCS Parquet, Azure Parquet, WebHDFS Parquet, Iceberg, Avro, HTTP, MQTT, NATS, Pulsar, RabbitMQ, file and built-ins.