KIP-101 — leader-epoch-based log truncation
Status: partial — see the KIP index.
What's missing
- The leader-epoch cache and lookup.
DiskStorageEngine::offset_for_leader_epoch(crates/kaas-storage/src/disk.rs) returns the(-1, -1)sentinel — the segment epochs are recorded on disk but no epoch→offset cache is materialized. - Wire key 23 (
OffsetForLeaderEpoch) is not registered — clients see it as unsupported via ApiVersions (it appears in the matrix's gap table).
What the KIP changes in Apache Kafka
KIP-101 replaced high-watermark-based follower truncation with
leader-epoch-based truncation: each replica tracks which offset ranges were
written under which leader epoch, and a rejoining follower asks the leader
"what's the last offset for epoch E?" (OffsetForLeaderEpoch) to truncate
divergent history precisely instead of over- or under-truncating.
What kaas has
Half of the KIP's substance — the epoch bookkeeping — exists structurally:
every segment filename carries the leader epoch it was written under
({epoch:08x}-{base_offset:020d}.log), so the divergent-history problem the
KIP solves is prevented by construction rather than repaired after the
fact. A deposed leader's late writes land in files named with a dead epoch
and are never part of the new leader's log
(storage engine).
Because kaas has no replication, there are no followers
to truncate — the KIP's driving scenario doesn't arise inside the broker.
What does still want the lookup is the client-side surface: KIP-320-aware
consumers send OffsetForLeaderEpoch to detect log truncation after
unclean leadership changes. Registering key 23 backed by a real epoch cache
is the tracked follow-up.
How the partial state is verified
The segment-epoch construction is exercised by the storage engine's unit
tests and the takeover paths in bins/kaas/tests/cluster_smoke.rs; the
stubbed lookup is explicit in crates/kaas-storage/src/disk.rs (grep for
offset_for_leader_epoch).