Skip to content

Compatibility

What a 1.x release promises to code and scripts written against an earlier 1.x release, surface by surface, and how each promise is enforced. The decision behind this page is design record 011; the byte-level counterpart (protocols, snapshot and checkpoint formats, plugin ABI) is Protocol and format compatibility.

The promise

Every public surface carries one of three tiers.

Tier Promise Where it is declared
Stable Source written against it keeps compiling and keeps its meaning on every 1.x release. Additions arrive at any time. A removal or a change of signature or semantics happens only at 2.0, after the member has carried a deprecation for at least one minor release. scripts/public-api-surface.txt ([stable]), scripts/libclink-abi-symbols.txt, tests/api_conformance/, tests/sql_conformance/
Evolving Documented and supported, but may change in a minor release. Every such change is named in the CHANGELOG; a rename keeps the old spelling working for at least one minor. scripts/public-api-surface.txt ([evolving])
Internal Installed because a plugin's include closure or a tool needs it. No promise. scripts/public-api-surface.txt ([internal])

Version numbers follow from the tiers. A major release is the only place a Stable surface may break. A minor release may add to any tier, change Evolving surfaces with notice, and rotate the plugin ABI fingerprint (a compiled job is rebuilt per release; the load gate refuses a stale one by name). A patch changes no surface.

The promise is source-level. A compiled job or plugin links the engine statically and is rebuilt against each release; the source it was built from keeps compiling. That boundary is design record 010 and this page does not move it.

C++

Tiers are assigned per header and recorded in scripts/public-api-surface.txt, regenerated by scripts/gen-public-api-surface.py and checked in CI. A header the rules do not name is Internal, so nothing is promoted by accident.

The Stable tier is the authoring surface the consumer examples and the internals pages teach:

  • the fluent pipeline (clink/api/), job registration (clink/job/) and plugin registration (clink/plugin/plugin.hpp, install_defaults.hpp);
  • the operator bases and the standard operator library (clink/operators/);
  • what an operator is handed: RuntimeContext, TimerService, OutputTag, the dead-letter seam;
  • the local runtime: Dag, LocalExecutor, JobConfig;
  • keyed and broadcast state, the typed state views, the StateBackend interface and factory, schema versioning and migrations;
  • Codec<T> and the declared-types machinery (CLINK_FIELDS, the derived codec, ArrowBatcher<T>);
  • watermarks, strategies and event time;
  • connector authoring: CommittingSink, PollingSource, the file and Parquet connectors, capability and delivery-guarantee records;
  • the CEP pattern API, the public testing framework (clink/test/), and EmbeddedEngine;
  • per-connector builders (clink/api/<vendor>_builders.hpp) and install() entry points.

Evolving covers surfaces that are real but not yet settled: the Table API, the Flight SQL server, JobSubmitter and JobGraphSpec, Coordinator and Worker as embeddable classes, the SQL Catalog and script runner, the queryable-state clients, the lineage listener, the state-processor readers, the metrics handles, the HTTP server and client, the columnar operator fast path, and the typed connector classes behind the builders.

Two conventions apply inside a Stable header: namespace detail and trailing-underscore identifiers are Internal, and a Stable header's include closure may reach lower-tier headers. Those reached types are reachable but not promised; the manifest lists them under [stable-reaches-evolving] and [stable-reaches-internal], and any change to that reach is a reviewed manifest diff.

Which members are promised is enumerated by tests/api_conformance/: compile-only translation units that use every promised class, function, macro and virtual hook the way a consumer would, with operator hooks overridden with override. The directory only grows; a change the build needs there is a Stable-tier break. Every Stable header is also compiled on its own, so a header that only compiles after some other header cannot pass as a contract. Both build as part of the normal test build.

C

include/clink/embed/clink.h is Stable in its entirety and versioned separately from the library. CLINK_EMBED_ABI_VERSION moves only for an incompatible change, which within 1.x is never; compare clink_abi_version() against it once, at load. The header states the growth rules it lives by:

  • Functions are only ever added. One scheduled for removal at 2.0 carries CLINK_DEPRECATED for at least one minor release first.
  • clink_engine_options grows by appending fields. Its leading struct_size (filled in by CLINK_ENGINE_OPTIONS_INIT) tells the library how much of the struct the caller compiled, so a field added later is invisible to an older binary and an older binary gets defaults from a newer library. A zero size is refused by name.
  • clink_version() names the library release for logging.
  • The exported symbol set is scripts/libclink-abi-symbols.txt, append-only, held equal to the header without a build and to the built library's dynamic symbol table as a test.

pyclink tracks the header and is Stable at the Python level: Engine, its documented keyword arguments and methods, and ClinkError.

SQL

The statements, clauses, types, functions and WITH option keys documented in the SQL reference are Stable. A script that runs on 1.0 runs unchanged, with the same results, on every 1.x:

  • The grammar only grows. clink's extensions must not capture text that previously parsed as ordinary SQL.
  • A WITH option key is never removed or given a new meaning; a renamed key keeps its old spelling as an alias through 1.x. Unknown keys keep failing at compile time.
  • A built-in function's result type and semantics never change, and a user-defined function shadows a built-in of the same name, so adding a built-in can never change what an existing script computes.
  • A persisted catalog directory (--catalog-dir) written by 1.0 loads on every 1.x: additive JSON, readers ignore unknown keys, pinned by frozen fixtures.

Kept Evolving until their shapes have had a release to settle: the AI table functions (CREATE MODEL, ML_PREDICT, VECTOR_SEARCH) and the WebAssembly aggregate form. Not contracts at all: the wording of diagnostics, the text of EXPLAIN, and the output format of SHOW TABLES.

The mechanical half is tests/sql_conformance/: a corpus of scripts, each with its inputs and the output the engine produced when the case was frozen, run through the embedded engine by one data-driven test. Cases are added, never edited to make a run pass; a case that must change to stay green is a semantic change and is reviewed as a Stable-tier break.

Everything else

Surface Tier Notes
CMake package: clink:: target and component names, clink_add_job_module() Stable
clink CLI subcommand names and documented flags Stable Human-readable output is Evolving
REST routes under /api/v1 Stable Additive JSON; a break ships as /api/v2 alongside
Capabilities manifest JSON Stable within its schema_version
Metric names Evolving A rename keeps the old name for one minor
Configuration keys and environment variables Evolving
Helm chart values Evolving
Wire protocols, snapshot and checkpoint formats, capture files, plugin ABI Governed by Protocol and format compatibility

Reading a change

Four tracked artefacts turn a contract change into a diff a reviewer sees: the header tier manifest, the C symbol manifest, the conformance translation units and the SQL corpus. A CHANGELOG entry names every Evolving change and every deprecation. If a change to any of the four is needed to make a build pass, it is a break of the Stable tier, and 1.x does not take it.