Skip to content

objects-on-kdb

git-object-format specs/git-object-format/objects-on-kdb.kmd

Specification body

Git objects on kdb — promotion gate

Why this exists

stack-RFC-006 §3 lays out a separate object-storage plane (kdb-obj, object-native; ≠ kdb core transactional plane) for git objects. Refs landed in 2026-05 (FLOW-147 spike + FLOW-148/#156-#165 productionize). Objects are deferred until the kdb-obj substrate decision lands in infra/data/kdb.

This spec is the placeholder + promotion-gate doc that keeps the work visible while the substrate is decided.

R1 — Code shape today

A stub package lives at products/dev/flow/engine/modules/kdb_obj/. Its Client interface mirrors the refs adapter shape:

type Client interface {
    Stat(ctx, repoID, objectID)        → (size, present, err)
    Load(ctx, repoID, objectID)        → ([]byte, err)
    LoadStream(ctx, repoID, objectID)  → (io.ReadCloser, err)
    Store(ctx, repoID, objectID, body) → err
    StoreStream(ctx, repoID, objectID, expectedSize, body) → err
    Delete(ctx, repoID, objectID)      → err
    Walk(ctx, repoID, fn)              → err
    BulkStat(ctx, repoID, objectIDs)   → (map[string]int64, err)
    Close()                            → err
}

New() returns a stubClient whose every method (except Close) returns kdb_obj.ErrNotImplemented. The feature gate [kdb_obj] ENABLED in app.ini defaults to false. When the substrate lands, the stub is swapped for a real client in New(); the rest of Flow keeps depending on Client.

R2 — Substrate decision (the blocker)

Owned by infra/data/kdb (per the data-plane convergence direction in stack-RFC-006). The candidates per RFC-006 §3 are:

  • TiKV with an object-native column-family layout
  • Foundationdb
  • A purpose-built object store sharing kdb core's WAL

Until infra/data/kdb ratifies one and the substrate is provisioned, no Flow-side work proceeds beyond the stub. This is the only substrate-side blocker; once it closes, FLOW-186 is unblocked.

R3 — Promotion gates (Flow side)

Once the substrate decision lands and kdb-obj is provisioned in dev, the following gates close in order:

GateClosed when
G1 — Substrate livekdb-obj endpoint is reachable from the flow LXC with auth working.
G2 — Real client landsStub stubClient replaced by a kdbObjClient implementing all Client methods against the substrate. ErrNotImplemented disappears from kdb_obj/.
G3 — Shadow-write soaksWrite path mirrors every git object the on-disk store accepts; comparator job confirms byte-equivalence over a soak window. (Mirrors the refs FLOW-148 phase 1 pattern.)
G4 — Shadow-read agreesRead path also consults kdb-obj; mismatches surface via metric (sibling to koder_flow_kdb_shadow_mismatch_total).
G5 — Backfilladmin git sync-objects walks existing repos and bulk-writes their object stores to kdb-obj. (Sibling to FLOW-148's BulkBackfillRefs.)
G6 — CAS phaseOptimistic CAS rejects double-writes that would silently lose data. (Sibling to FLOW-157 KDB.REF_CAS_ENABLED.)
G7 — Primary flip[kdb_obj] PRIMARY = true — reads served from kdb-obj; on-disk store becomes secondary.

The phasing matches what refs went through (FLOW-148 → FLOW-157 → eventual primary flip), so reviewers can audit each step against the refs precedent.

R4 — Out of scope

  • Migration of existing repos. A backfill mode covers that under G5.
  • Sha256 default flip. Different gate; see sha256-promotion.kmd. Object format and object plane are orthogonal.
  • kdb core (transactional plane). That's already running per [kdb] ENABLED=true; objects live in a separate plane per RFC-006 §3.

R5 — koder.toml tracking

[gates.planned]
kdb_obj_primary = "doc:meta/docs/stack/specs/git-object-format/objects-on-kdb.kmd"

(Not yet added to products/dev/flow/engine/koder.toml — gated on the substrate decision; will be added when G1 closes.)

Status

Track-on-hold. Stub + config + this doc shipped 2026-05-28 (FLOW-186 close). No further Flow-side work until infra/data/kdb ratifies the substrate.

GateStateLast checkedEvidence
Substrate decisionopen2026-05-28infra/data/kdb substrate ticket pending
G1 — Substrate liveopen2026-05-28n/a
G2 — Real clientopen2026-05-28stub-only
G3 — Shadow-writeopen2026-05-28n/a
G4 — Shadow-readopen2026-05-28n/a
G5 — Backfillopen2026-05-28n/a
G6 — CAS phaseopen2026-05-28n/a
G7 — Primary flipopen2026-05-28n/a