Repository navigation
fix(inkless:switch): preserve classic prefix on partition increase during switch [KC-387] - #737
Conversation
There was a problem hiding this comment.
Pull request overview
Prevents classic-prefix loss when partitions are added during a classic-to-diskless switch.
Changes:
- Creates control-plane rows only for newly added partitions.
- Makes the committed seal authoritative over empty placeholders.
- Adds backend and end-to-end regression coverage; regenerates jOOQ schema metadata.
Reviewed changes
Copilot reviewed 7 out of 125 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
core/src/main/scala/kafka/server/ControllerApis.scala |
Passes the prior partition count to control-plane creation. |
core/src/test/java/kafka/server/InklessConsolidatedDisklessTopicsTest.java |
Adds the end-to-end regression test. |
storage/inkless/src/main/java/io/aiven/inkless/control_plane/CreateTopicAndPartitionsRequest.java |
Adds the first-partition range bound. |
storage/inkless/src/main/java/io/aiven/inkless/control_plane/InMemoryControlPlane.java |
Adopts seals over empty placeholders. |
storage/inkless/src/main/java/io/aiven/inkless/control_plane/postgres/TopicsAndPartitionsCreateJob.java |
Inserts only the requested partition range. |
storage/inkless/src/main/resources/db/migration/V23__Init_diskless_log_authoritative_seal.sql |
Adds guarded authoritative-seal upsert behavior. |
storage/inkless/src/test/java/io/aiven/inkless/control_plane/AbstractControlPlaneTest.java |
Tests seal adoption on both backends. |
storage/inkless/src/main/jooq/org/jooq/generated/UDTs.java |
Regenerates schema-23 metadata. |
storage/inkless/src/main/jooq/org/jooq/generated/DefaultSchema.java |
Regenerates schema-23 metadata. |
storage/inkless/src/main/jooq/org/jooq/generated/Domains.java |
Regenerates schema-23 metadata. |
storage/inkless/src/main/jooq/org/jooq/generated/Indexes.java |
Regenerates schema-23 metadata. |
storage/inkless/src/main/jooq/org/jooq/generated/Keys.java |
Regenerates schema-23 metadata. |
storage/inkless/src/main/jooq/org/jooq/generated/Routines.java |
Regenerates schema-23 metadata. |
storage/inkless/src/main/jooq/org/jooq/generated/Tables.java |
Regenerates schema-23 metadata. |
storage/inkless/src/main/jooq/org/jooq/generated/enums/* |
Regenerates all changed enum bindings for schema 23. |
storage/inkless/src/main/jooq/org/jooq/generated/routines/* |
Regenerates all changed routine bindings for schema 23. |
storage/inkless/src/main/jooq/org/jooq/generated/tables/* |
Regenerates all changed table bindings for schema 23. |
storage/inkless/src/main/jooq/org/jooq/generated/tables/records/* |
Regenerates all changed table-record bindings for schema 23. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/* |
Regenerates all changed UDT bindings for schema 23. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/paths/* |
Regenerates all changed UDT path bindings for schema 23. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/records/* |
Regenerates all changed UDT record bindings for schema 23. |
Suppressed comments (1)
core/src/main/scala/kafka/server/ControllerApis.scala:1006
- Add a focused controller test that captures the
CreateTopicAndPartitionsRequestand verifiesfirstPartitionis the pre-increase count. The new end-to-end test can still pass if this code sends0, because the authoritative-seal upsert independently repairs that placeholder, and the existing control-plane test uses the three-argument constructor explicitly starting at 0. Thus the stated “only newly added partitions” behavior is currently unprotected.
val firstPartition = priorPartitionCounts.getOrElse(req.name(), 0)
Some(new CreateTopicAndPartitionsRequest(topicIdOrError.result(), req.name(), firstPartition, req.count()))
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
bed7928 to
f4d43f3
Compare
…ring switch [KC-387]
A CreatePartitions committed during a classic-to-diskless switch created the
switching partition's control-plane logs row at high_watermark=0. That made
diskless LATEST report 0, so the consolidation fetcher truncated the classic
prefix and lost the pre-switch records.
The create path now snapshots each topic's ID and partition count from one
metadata image before the mutation. A matching post-mutation ID creates only
[priorCount, count). A missing or changed ID creates [0, count), with overlap
resolved by the control-plane conflict handling. Validation-only requests skip
this work.
init_diskless_log_v1 now locks existing target rows in topic-partition order,
replaces an empty placeholder with a positive seal, and replaces producer state
when that update applies. The in-memory control plane mirrors this behavior.
SQL evolution from V11 to V23:
```diff
--- V11__Init_diskless_log.sql
+++ V23__Init_diskless_log_authoritative_seal.sql
@@
DECLARE
l_request RECORD;
l_producer_state RECORD;
- l_inserted BOOLEAN;
BEGIN
+ -- Every caller takes existing row locks in the same order; responses still follow arg_requests order.
+ PERFORM 1
+ FROM logs
+ JOIN unnest(arg_requests) AS request
+ ON logs.topic_id = request.topic_id
+ AND logs.partition = request.partition
+ ORDER BY logs.topic_id, logs.partition
+ FOR UPDATE OF logs;
+
FOR l_request IN
SELECT *
FROM unnest(arg_requests)
@@
INSERT INTO logs (topic_id, partition, topic_name, log_start_offset, high_watermark, byte_size, diskless_start_offset)
VALUES (l_request.topic_id, l_request.partition, l_request.topic_name,
l_request.log_start_offset, l_request.diskless_start_offset, 0, l_request.diskless_start_offset)
- ON CONFLICT (topic_id, partition) DO NOTHING;
-
- l_inserted := FOUND;
+ ON CONFLICT (topic_id, partition) DO UPDATE
+ SET log_start_offset = EXCLUDED.log_start_offset,
+ high_watermark = EXCLUDED.high_watermark,
+ diskless_start_offset = EXCLUDED.diskless_start_offset
+ WHERE logs.high_watermark = 0
+ AND logs.diskless_start_offset = 0
+ AND logs.byte_size = 0
+ -- A zero-offset re-init would not advance the placeholder, so leaving it alone keeps
+ -- re-initing an existing empty partition a no-op.
+ AND EXCLUDED.high_watermark > 0;
- IF NOT l_inserted THEN
+ IF NOT FOUND THEN
RETURN NEXT (l_request.topic_id, l_request.partition, 'already_initialized')::init_diskless_log_response_v1;
CONTINUE;
END IF;
+ -- The update path can reach a row an earlier init already wrote producer state for, and
+ -- producer_state is keyed on a BIGSERIAL row_id, so those rows would accumulate rather than be
+ -- replaced. The incoming snapshot is the authority for the partition.
+ DELETE FROM producer_state
+ WHERE topic_id = l_request.topic_id
+ AND partition = l_request.partition;
+
FOR l_producer_state IN
SELECT *
FROM unnest(arg_producer_states)
```
Tests cover both control-plane backends, exact controller ranges, topic
recreation, topics appearing during a request, validation-only behavior, and
the end-to-end partition-increase switch path.
Co-authored-by: Cursor <cursoragent@cursor.com>
f4d43f3 to
a8fd625
Compare
jeqo
left a comment
There was a problem hiding this comment.
Looks good to me overall. Few minor findings
| @@ -0,0 +1,52 @@ | |||
| /* | |||
| * Inkless | |||
| * Copyright (C) 2024 - 2025 Aiven OY | |||
There was a problem hiding this comment.
nit:
| * Copyright (C) 2024 - 2025 Aiven OY | |
| * Copyright (C) 2024 - 2026 Aiven OY |
| Some(new CreateTopicAndPartitionsRequest(topicIdOrError.result(), req.name(), req.count())) | ||
| val topicId = topicIdOrError.result() | ||
| // The cached range is only usable when it belongs to the topic the controller just mutated. | ||
| // Otherwise create the full range and rely on init_diskless_log_v1 to resolve overlap (KC-387). | ||
| val firstPartition = priorTopicStates.get(topicName) match { | ||
| case Some(state) if state.topicId == topicId => math.min(state.numPartitions, req.count()) | ||
| case _ => 0 | ||
| } | ||
| Some(new CreateTopicAndPartitionsRequest(topicId, topicName, firstPartition, req.count())) |
There was a problem hiding this comment.
Note: Claude found an interesting side effect of this: As Controller commits the metadata mutation before Control Plane call: If Control Plane fails to write, nothing reconciles this gap, and these diskless partition will not have rows and fail with UNKNOWN_TOPIC_OR_PARTITION. Also, repair_diskless_log_v1 will not be able to help here.
Before this PR, if a larger increase on the same topic would be attempted, it would be healed by inserting [0, count). With this PR, it would insert [N, count) missing the failed partitions in between.
A same-count retry never reached the control plane either way, then nothing was really depending on this so I don't think we need to fix this here.
Let's have a follow up to reconcile this. Maybe let's also mention this on the commit?
There was a problem hiding this comment.
This is a good point, will create a follow-up fix. I was thinking in some kind of reconciliation loop: take the controller metadata as the ground truth, then query the control plane for log row and reconcile where needed (with retry and backoff).
|
|
||
| @AfterEach | ||
| def tearDown(): Unit = { | ||
| metadataCache.setImage(MetadataImage.EMPTY) |
There was a problem hiding this comment.
nit: Another non-obvious Claude catch: this per-instance val with no @TestInstance(PER_CLASS), so JUnit constructs a fresh KRaftMetadataCache per test, and a fresh one already holds MetadataImage.EMPTY -- so this setImage is inert (same on :1096).
Tested locally and seems to be right.
There was a problem hiding this comment.
Removed the redundant setImage instead of adding PER_CLASS as removing seems to be a more narrow change with less possible side effects.
…ring switch [KC-387] (#737) A CreatePartitions committed during a classic-to-diskless switch created the switching partition's control-plane logs row at high_watermark=0. That made diskless LATEST report 0, so the consolidation fetcher truncated the classic prefix and lost the pre-switch records. The create path now snapshots each topic's ID and partition count from one metadata image before the mutation. A matching post-mutation ID creates only [priorCount, count). A missing or changed ID creates [0, count), with overlap resolved by the control-plane conflict handling. Validation-only requests skip this work. init_diskless_log_v1 now locks existing target rows in topic-partition order, replaces an empty placeholder with a positive seal, and replaces producer state when that update applies. The in-memory control plane mirrors this behavior. --------- Co-authored-by: Cursor <cursoragent@cursor.com>
…ring switch [KC-387] (#737) A CreatePartitions committed during a classic-to-diskless switch created the switching partition's control-plane logs row at high_watermark=0. That made diskless LATEST report 0, so the consolidation fetcher truncated the classic prefix and lost the pre-switch records. The create path now snapshots each topic's ID and partition count from one metadata image before the mutation. A matching post-mutation ID creates only [priorCount, count). A missing or changed ID creates [0, count), with overlap resolved by the control-plane conflict handling. Validation-only requests skip this work. init_diskless_log_v1 now locks existing target rows in topic-partition order, replaces an empty placeholder with a positive seal, and replaces producer state when that update applies. The in-memory control plane mirrors this behavior. --------- Co-authored-by: Cursor <cursoragent@cursor.com>
A CreatePartitions committed during a classic-to-diskless switch created the switching partition's control-plane logs row at high_watermark=0. That made diskless LATEST report 0, so the consolidation fetcher truncated the classic
prefix and lost the pre-switch records.
The create path now snapshots each topic's ID and partition count from one metadata image before the mutation. A matching post-mutation ID creates only [priorCount, count). A missing or changed ID creates [0, count), with overlap
resolved by the control-plane conflict handling. Validation-only requests skip this work.
init_diskless_log_v1 now locks existing target rows in topic-partition order, replaces an empty placeholder with a positive seal, and replaces producer state when that update applies. The in-memory control plane mirrors this behavior.
SQL evolution from V11 to V23:
Tests cover both control-plane backends, exact controller ranges, topic recreation, topics appearing during a request, validation-only behavior, and the end-to-end partition-increase switch path.
Co-authored-by: Cursor cursoragent@cursor.com