Skip to content

feat(inkless:controller): gate ISR, reassignment, and leader election on switch phase - #833

Merged
jeqo merged 7 commits into
mainfrom
giuseppelillo/kc-586-switch-phase-rules
Oct 23, 2026
Merged

jeqo merged 7 commits into
mainfrom
giuseppelillo/kc-586-switch-phase-rules

Conversation

@giuseppelillo

@giuseppelillo giuseppelillo commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

In the two-phase classic-to-diskless switch, phase 1 lasts while a partition has a seal (classicToDisklessStartOffset != -1) and classicPrefixInRemoteStorage is false. During phase 1, the classic prefix below the seal exists only on the replicas' local disks, so the controller must enforce classic ISR, reassignment, and leader election rules. Once classicPrefixInRemoteStorage is true (phase 2), the prefix lives in remote storage and no replica relies on its local copy, so the controller applies born-diskless rules.

This PR adds isSwitchedWithLocalClassicPrefix(PartitionRegistration) on ReplicationControlManager and gates the controller's switch-sensitive paths on it. Broker-side ISR shrink for lag in phase 1 moves to a follow-up PR, as it requires consolidation stop in phase 1 which is not implemented yet.

Changes

  • Add ReplicationControlManager.isSwitchedWithLocalClassicPrefix: returns true when classicToDisklessStartOffset != NO_CLASSIC_TO_DISKLESS_START_OFFSET and classicPrefixInRemoteStorage is false.
  • ReplicationControlManager.alterDisklessSwitch: reject any seal modification once classicPrefixInRemoteStorage is true, including lowering the seal. In phase 2, brokers delete their local prefix below the committed seal and start consolidating from it, so the seal can't move afterward.
  • ReplicationControlManager.leaderAcceptorFor (both overloads): use LeaderAcceptor (which requires an online replica directory) when isSwitchedWithLocalClassicPrefix(partition) is true, and keep isActive-only for born-diskless and phase-2 partitions.
  • ReplicationControlManager.changePartitionReassignment: take the staged addingReplicas/removingReplicas path when isSwitchedWithLocalClassicPrefix(part) is true, and take the one-step path for born-diskless and phase-2 partitions. This supersedes fix(inkless:switch): stage reassignments of switched partitions #757. When an administrator retargets a diskless partition that already has a reassignment in progress, the staged path reconciles the inherited adding and removing replicas against the new target, and the one-step path clears them even when the new target equals the current replicas.
  • ReplicationControlManager.expandIsrForDisklessManagedPartitions: skip unfence ISR expansion only while isSwitchedWithLocalClassicPrefix(partition) is true, so phase-2 partitions re-add unfenced replicas and complete in-flight reassignments like born-diskless partitions. Pass r -> r == brokerId || clusterControl.isActive(r) to leaderAcceptorFor so an in-flight reassignment that removes the current leader can elect the unfencing broker before its BrokerRegistrationChangeRecord is replayed.

Test plan

  • ReplicationControlManagerInklessTest.testAlterDisklessSwitchRejectedAfterClassicPrefixInRemoteStorage: verifies that lowering (0, 50), matching (100), raising (150), and negative (-1, -2) seal offsets all fail with InvalidRequestException once the flag is set.
  • ReplicationControlManagerInklessTest$DisklessManagedReplicasTests.testLeaderElectionRequiresOnlineDirectoryUntilClassicPrefixIsInRemoteStorage: verifies that a replica with DirectoryId.LOST can't win leader election in phase 1 or while the switch is pending (via both handleBrokerFenced and electLeader), and can win in phase 2.
  • ReplicationControlManagerInklessTest$DisklessManagedReplicasTests: covers staged reassignment completion and cancellation in phase 1 (testReassignSwitchedPartitionStagesTargetInsteadOfGrantingIsr, testCancelReassignmentOfSwitchedPartitionRevertsStagedState, testReassignConsolidatingSwitchedPartitionAlsoStages) and one-step reassignment in phase 2 (testReassignSwitchedPartitionWithClassicPrefixInRemoteStorageCompletesInOneStep).
  • ReplicationControlManagerInklessTest$DisklessManagedReplicasTests.testUnfenceExpandsIsrOfBornDisklessButNotSwitchedPartitions: verifies that unfencing expands ISR for born-diskless and phase-2 partitions and skips phase-1 and pending-switch partitions.
  • ReplicationControlManagerInklessTest$SetClassicPrefixInRemoteStorageTests.testSetClassicPrefixInRemoteStorageInFlightReassignmentCompletesOnLaterPhase2Unfence: verifies that a staged reassignment still in flight when the flag commits completes when the adding replica unfences in phase 2.

Known limitation

A consolidating follower can leave the classic fetcher at the seal without reporting itself in sync. Until a follow-up PR stops consolidation in phase 1, a phase-1 reassignment that adds such a follower can wait for the flag commit to finish it. The controller side in this PR handles that case: the flag commit completes the reassignment and elects the leader with phase-2 rules.

  • ReplicationControlManagerInklessTest$DisklessManagedReplicasTests.testRetargetAfterFlagKeepsRequestedReplicasWhenTargetKeepsRemovingReplica and testRetargetAfterFlagClearsStaleAddingReplica: verify that retargeting a partition whose phase-1 reassignment survived the flag commit uses exactly the requested replicas.
  • ReplicationControlManagerInklessTest$SetClassicPrefixInRemoteStorageTests.testSetClassicPrefixInRemoteStorageElectsLeaderWithLostDirectoryWhenCompletingReassignment: verifies that the flag commit elects a leader among the target replicas when they all have lost directories.
  • ReplicationControlManagerInklessTest$DisklessManagedReplicasTests.testRetargetAfterFlagToExactCurrentReplicasClearsUnfinishedReassignment: verifies that requesting exactly the current replicas clears an unfinished phase-1 move after the flag commits.
  • ReplicationControlManagerInklessTest$DisklessManagedReplicasTests.testRetargetStagedReassignmentDropsAddingReplicaThatIsNoLongerRequested and testRetargetStagedReassignmentKeepsCatchUpForAddingReplicaStillRequested: verify that retargeting a staged phase-1 reassignment drops adding replicas that are no longer requested and keeps catch-up for the ones that still are.

Assisted-by: Gemini 4 Argon

@giuseppelillo
giuseppelillo force-pushed the giuseppelillo/kc-586-switch-phase-rules branch from a853b50 to a01dbe1 Compare October 21, 2026 12:39
@giuseppelillo
giuseppelillo requested a balanced review from Copilot October 21, 2026 12:54

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved transition handling can corrupt reassignment targets, leave partitions leaderless, or stall follower catch-up.

3 open issues
What changed in this PR

This PR makes controller behavior follow the classic-to-diskless switch phase, preserving classic replica requirements until remote storage holds the prefix.

Changes:

  • Gate ISR expansion, reassignment, and leader eligibility on switch phase.
  • Reject seal changes after the remote-prefix flag commits.
  • Add tests for phase-specific behavior and transitions.
File Description
metadata/​src/​test/​java/​org/​apache/​kafka/​controller/​ReplicationControlManagerInklessTest.java Adds switch-phase and reassignment regression tests.
metadata/​src/​main/​java/​org/​apache/​kafka/​controller/​ReplicationControlManager.java Applies phase-aware controller rules and protects committed seals.

Review effort: Balanced

@giuseppelillo
giuseppelillo force-pushed the giuseppelillo/kc-586-switch-phase-rules branch from 12bfb37 to 84cf998 Compare October 21, 2026 13:11
@giuseppelillo giuseppelillo changed the title feat(inkless:controller): gate ISR, reassignment, and leader election on switch phase [KC-586] feat(inkless:controller): gate ISR, reassignment, and leader election on switch phase Oct 21, 2026
…iered

Once classicPrefixInRemoteStorage is set, brokers delete their local prefix below the committed seal and start consolidating from it. Reject any seal modification (including lowering) once the flag is set.

Assisted-by: Gemini 4 Argon
…election

In phase 1 (seal present, classicPrefixInRemoteStorage unset), the classic prefix still lives on the replicas' local disks, so a broker whose replica directory is offline cannot safely become leader. Use LeaderAcceptor in both leaderAcceptorFor overloads while isSwitchedWithLocalClassicPrefix(partition) holds, and keep isActive-only for born-diskless and phase-2 partitions.

Assisted-by: Gemini 4 Argon
@giuseppelillo
giuseppelillo force-pushed the giuseppelillo/kc-586-switch-phase-rules branch from 84cf998 to 6c46be6 Compare October 21, 2026 13:15
@giuseppelillo
giuseppelillo requested a balanced review from Copilot October 21, 2026 13:18

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Retargeting can leave reassignments unable to complete or drop explicitly requested replicas.

2 open issues
3 resolved since last review

Review effort: Balanced

…itions

In phase 1 (seal present, classicPrefixInRemoteStorage unset), records below the seal exist only on the replicas' local disks, so a reassignment must use the staged adding/removing path until the new replicas catch up. Born-diskless and phase-2 partitions keep the one-step reassignment path.

Retargeting a reassignment that is already in progress reconciles the inherited adding and removing replicas against the new target. A replica that is no longer requested leaves adding, a requested replica that hasn't caught up stays in adding, and the one-step path clears the previous move even when the new target equals the current replicas.

Assisted-by: Gemini 4 Argon
@giuseppelillo
giuseppelillo force-pushed the giuseppelillo/kc-586-switch-phase-rules branch from 6c46be6 to c2fc447 Compare October 21, 2026 13:28
@giuseppelillo
giuseppelillo requested a balanced review from Copilot October 21, 2026 13:33

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Replication safety across phase transitions requires human validation alongside the documented broker-side handoff limitation.

No open issues 🎉

2 resolved since last review

Review effort: Balanced

giuseppelillo and others added 2 commits October 21, 2026 16:16
…artitions

Once classicPrefixInRemoteStorage is set, a switched partition's classic prefix lives in remote storage and no replica relies on its local copy. Skip unfence ISR expansion only while isSwitchedWithLocalClassicPrefix(partition) holds, so phase-2 partitions re-add unfenced replicas (and complete in-flight reassignments) like born-diskless partitions.

Assisted-by: Gemini 4 Argon
Assisted-by: Claude Sonnet 5.5
Co-authored-by: Cursor <cursoragent@cursor.com>
The phase rules change how a switched partition handles reassignment, ISR expansion on unfence, and `AlterDisklessSwitch`. The docs still say that diskless reassignment is always immediate and that a returning broker rejoins the ISR at once. They don't describe the phases. The `seal` command also validates only the offset range, so `--dry-run` reports an action that the controller rejects in phase 2.

Add a "Switch phases" section to `CLASSIC_TO_DISKLESS_SWITCH.md` and link it from `FEATURES.md`, `FAQ.md`, and `DISKLESS_CONSOLIDATION.md`. Document that the controller rejects every seal change once `classicPrefixInRemoteStorage` is set, in the operator override section, the `seal --offset` help, and the `Admin.alterDisklessSwitch` Javadoc.

`kafka-topic-switch.sh seal` now fails with that reason when it reads the partition, which covers `--dry-run`. The `-1` and `-2` offsets don't read the partition, because controllers don't serve `DescribeTopicPartitions`. They keep working with `--bootstrap-controller`.

Testing

Run `TopicSwitchCommandTest`, the `clients` and `tools` checkstyle tasks, and `make vale`.

Assisted-by: Claude Sonnet 5.5
Co-authored-by: Cursor <cursoragent@cursor.com>

@jeqo jeqo left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found few minor changes, otherwise LGTM.

Comment thread docs/inkless/FAQ.md Resolved
… command

The controller rejects a seal offset that exceeds the committed seal, but
`kafka-topic-switch.sh seal` only checked the classic log range. After a
switch, the log end offset includes diskless data, so `--dry-run` reported
"Would set" for a request the controller then rejects. The command now fails
before it sends the request, and the help text and switch doc state the rule.

The unfence ISR expansion comment now covers both cases where the unfencing
broker is accepted as leader: a leaderless partition and a reassignment that
removes the current leader. A new test pins the leaderless case for
born-diskless partitions.

The FAQ answer about a returning replica broker notes that a phase-1
switched topic rejoins the ISR through `AlterPartition`.

## Test plan

- `ReplicationControlManagerInklessTest$DisklessManagedReplicasTests.testUnfenceElectsUnfencingBrokerForLeaderlessBornDisklessPartition`:
  fails with the previous `clusterControl::isActive` predicate and passes
  with the current one.
- `TopicSwitchCommandTest`: `testSealRejectsExplicitOffsetAboveCommittedSeal`,
  `testSealDryRunRejectsExplicitOffsetAboveCommittedSeal`, and
  `testSealCommitsExplicitOffsetEqualToCommittedSeal`.

Assisted-by: Claude Opus 5.5
Co-authored-by: Cursor <cursoragent@cursor.com>
jeqo
jeqo previously approved these changes Oct 23, 2026
@giuseppelillo
giuseppelillo force-pushed the giuseppelillo/kc-586-switch-phase-rules branch from 6931a21 to 8888c31 Compare October 23, 2026 11:59
@giuseppelillo

Copy link
Copy Markdown
Contributor Author

Re-pushed as the first 2 commits were missing the signature

…n docs

Main inserts the untiered diskless migration guide in the same place as the
phase-1 reassignment exception. Keep both so switched partitions still
document staged reassignment while the prefix is local.

Assisted-by: Grok 4.7
Co-authored-by: Cursor <cursoragent@cursor.com>
@jeqo
jeqo merged commit 27b691d into main Oct 23, 2026
7 checks passed
@jeqo
jeqo deleted the giuseppelillo/kc-586-switch-phase-rules branch October 23, 2026 13:53
viktorsomogyi pushed a commit that referenced this pull request Oct 29, 2026
… on switch phase (#833)

* fix(inkless:controller): reject alterDisklessSwitch after prefix is tiered

Once classicPrefixInRemoteStorage is set, brokers delete their local prefix below the committed seal and start consolidating from it. Reject any seal modification (including lowering) once the flag is set.

Assisted-by: Gemini 4 Argon

* fix(inkless:controller): require online directory for phase-1 leader election

In phase 1 (seal present, classicPrefixInRemoteStorage unset), the classic prefix still lives on the replicas' local disks, so a broker whose replica directory is offline cannot safely become leader. Use LeaderAcceptor in both leaderAcceptorFor overloads while isSwitchedWithLocalClassicPrefix(partition) holds, and keep isActive-only for born-diskless and phase-2 partitions.

Assisted-by: Gemini 4 Argon

* fix(inkless:controller): stage reassignments of phase-1 switched partitions

In phase 1 (seal present, classicPrefixInRemoteStorage unset), records below the seal exist only on the replicas' local disks, so a reassignment must use the staged adding/removing path until the new replicas catch up. Born-diskless and phase-2 partitions keep the one-step reassignment path.

Retargeting a reassignment that is already in progress reconciles the inherited adding and removing replicas against the new target. A replica that is no longer requested leaves adding, a requested replica that hasn't caught up stays in adding, and the one-step path clears the previous move even when the new target equals the current replicas.

Assisted-by: Gemini 4 Argon

* fix(inkless:controller): expand ISR on unfence for phase-2 switched partitions

Once classicPrefixInRemoteStorage is set, a switched partition's classic prefix lives in remote storage and no replica relies on its local copy. Skip unfence ISR expansion only while isSwitchedWithLocalClassicPrefix(partition) holds, so phase-2 partitions re-add unfenced replicas (and complete in-flight reassignments) like born-diskless partitions.

Assisted-by: Gemini 4 Argon
Assisted-by: Claude Sonnet 5.5
Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(inkless:switch): align docs and seal command with switch phases

The phase rules change how a switched partition handles reassignment, ISR expansion on unfence, and `AlterDisklessSwitch`. The docs still say that diskless reassignment is always immediate and that a returning broker rejoins the ISR at once. They don't describe the phases. The `seal` command also validates only the offset range, so `--dry-run` reports an action that the controller rejects in phase 2.

Add a "Switch phases" section to `CLASSIC_TO_DISKLESS_SWITCH.md` and link it from `FEATURES.md`, `FAQ.md`, and `DISKLESS_CONSOLIDATION.md`. Document that the controller rejects every seal change once `classicPrefixInRemoteStorage` is set, in the operator override section, the `seal --offset` help, and the `Admin.alterDisklessSwitch` Javadoc.

`kafka-topic-switch.sh seal` now fails with that reason when it reads the partition, which covers `--dry-run`. The `-1` and `-2` offsets don't read the partition, because controllers don't serve `DescribeTopicPartitions`. They keep working with `--bootstrap-controller`.

Testing

Run `TopicSwitchCommandTest`, the `clients` and `tools` checkstyle tasks, and `make vale`.

Assisted-by: Claude Sonnet 5.5
Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(inkless:switch): reject seals past the committed seal in the seal command

The controller rejects a seal offset that exceeds the committed seal, but
`kafka-topic-switch.sh seal` only checked the classic log range. After a
switch, the log end offset includes diskless data, so `--dry-run` reported
"Would set" for a request the controller then rejects. The command now fails
before it sends the request, and the help text and switch doc state the rule.

The unfence ISR expansion comment now covers both cases where the unfencing
broker is accepted as leader: a leaderless partition and a reassignment that
removes the current leader. A new test pins the leaderless case for
born-diskless partitions.

The FAQ answer about a returning replica broker notes that a phase-1
switched topic rejoins the ISR through `AlterPartition`.

## Test plan

- `ReplicationControlManagerInklessTest$DisklessManagedReplicasTests.testUnfenceElectsUnfencingBrokerForLeaderlessBornDisklessPartition`:
  fails with the previous `clusterControl::isActive` predicate and passes
  with the current one.
- `TopicSwitchCommandTest`: `testSealRejectsExplicitOffsetAboveCommittedSeal`,
  `testSealDryRunRejectsExplicitOffsetAboveCommittedSeal`, and
  `testSealCommitsExplicitOffsetEqualToCommittedSeal`.

Assisted-by: Claude Opus 5.5
Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
viktorsomogyi pushed a commit that referenced this pull request Oct 29, 2026
… on switch phase (#833)

* fix(inkless:controller): reject alterDisklessSwitch after prefix is tiered

Once classicPrefixInRemoteStorage is set, brokers delete their local prefix below the committed seal and start consolidating from it. Reject any seal modification (including lowering) once the flag is set.

Assisted-by: Gemini 4 Argon

* fix(inkless:controller): require online directory for phase-1 leader election

In phase 1 (seal present, classicPrefixInRemoteStorage unset), the classic prefix still lives on the replicas' local disks, so a broker whose replica directory is offline cannot safely become leader. Use LeaderAcceptor in both leaderAcceptorFor overloads while isSwitchedWithLocalClassicPrefix(partition) holds, and keep isActive-only for born-diskless and phase-2 partitions.

Assisted-by: Gemini 4 Argon

* fix(inkless:controller): stage reassignments of phase-1 switched partitions

In phase 1 (seal present, classicPrefixInRemoteStorage unset), records below the seal exist only on the replicas' local disks, so a reassignment must use the staged adding/removing path until the new replicas catch up. Born-diskless and phase-2 partitions keep the one-step reassignment path.

Retargeting a reassignment that is already in progress reconciles the inherited adding and removing replicas against the new target. A replica that is no longer requested leaves adding, a requested replica that hasn't caught up stays in adding, and the one-step path clears the previous move even when the new target equals the current replicas.

Assisted-by: Gemini 4 Argon

* fix(inkless:controller): expand ISR on unfence for phase-2 switched partitions

Once classicPrefixInRemoteStorage is set, a switched partition's classic prefix lives in remote storage and no replica relies on its local copy. Skip unfence ISR expansion only while isSwitchedWithLocalClassicPrefix(partition) holds, so phase-2 partitions re-add unfenced replicas (and complete in-flight reassignments) like born-diskless partitions.

Assisted-by: Gemini 4 Argon
Assisted-by: Claude Sonnet 5.5
Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(inkless:switch): align docs and seal command with switch phases

The phase rules change how a switched partition handles reassignment, ISR expansion on unfence, and `AlterDisklessSwitch`. The docs still say that diskless reassignment is always immediate and that a returning broker rejoins the ISR at once. They don't describe the phases. The `seal` command also validates only the offset range, so `--dry-run` reports an action that the controller rejects in phase 2.

Add a "Switch phases" section to `CLASSIC_TO_DISKLESS_SWITCH.md` and link it from `FEATURES.md`, `FAQ.md`, and `DISKLESS_CONSOLIDATION.md`. Document that the controller rejects every seal change once `classicPrefixInRemoteStorage` is set, in the operator override section, the `seal --offset` help, and the `Admin.alterDisklessSwitch` Javadoc.

`kafka-topic-switch.sh seal` now fails with that reason when it reads the partition, which covers `--dry-run`. The `-1` and `-2` offsets don't read the partition, because controllers don't serve `DescribeTopicPartitions`. They keep working with `--bootstrap-controller`.

Testing

Run `TopicSwitchCommandTest`, the `clients` and `tools` checkstyle tasks, and `make vale`.

Assisted-by: Claude Sonnet 5.5
Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(inkless:switch): reject seals past the committed seal in the seal command

The controller rejects a seal offset that exceeds the committed seal, but
`kafka-topic-switch.sh seal` only checked the classic log range. After a
switch, the log end offset includes diskless data, so `--dry-run` reported
"Would set" for a request the controller then rejects. The command now fails
before it sends the request, and the help text and switch doc state the rule.

The unfence ISR expansion comment now covers both cases where the unfencing
broker is accepted as leader: a leaderless partition and a reassignment that
removes the current leader. A new test pins the leaderless case for
born-diskless partitions.

The FAQ answer about a returning replica broker notes that a phase-1
switched topic rejoins the ISR through `AlterPartition`.

## Test plan

- `ReplicationControlManagerInklessTest$DisklessManagedReplicasTests.testUnfenceElectsUnfencingBrokerForLeaderlessBornDisklessPartition`:
  fails with the previous `clusterControl::isActive` predicate and passes
  with the current one.
- `TopicSwitchCommandTest`: `testSealRejectsExplicitOffsetAboveCommittedSeal`,
  `testSealDryRunRejectsExplicitOffsetAboveCommittedSeal`, and
  `testSealCommitsExplicitOffsetEqualToCommittedSeal`.

Assisted-by: Claude Opus 5.5
Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Assisted-by: Grok 4.7
gqmelo pushed a commit that referenced this pull request Nov 5, 2026
… on switch phase (#833)

* fix(inkless:controller): reject alterDisklessSwitch after prefix is tiered

Once classicPrefixInRemoteStorage is set, brokers delete their local prefix below the committed seal and start consolidating from it. Reject any seal modification (including lowering) once the flag is set.

Assisted-by: Gemini 4 Argon

* fix(inkless:controller): require online directory for phase-1 leader election

In phase 1 (seal present, classicPrefixInRemoteStorage unset), the classic prefix still lives on the replicas' local disks, so a broker whose replica directory is offline cannot safely become leader. Use LeaderAcceptor in both leaderAcceptorFor overloads while isSwitchedWithLocalClassicPrefix(partition) holds, and keep isActive-only for born-diskless and phase-2 partitions.

Assisted-by: Gemini 4 Argon

* fix(inkless:controller): stage reassignments of phase-1 switched partitions

In phase 1 (seal present, classicPrefixInRemoteStorage unset), records below the seal exist only on the replicas' local disks, so a reassignment must use the staged adding/removing path until the new replicas catch up. Born-diskless and phase-2 partitions keep the one-step reassignment path.

Retargeting a reassignment that is already in progress reconciles the inherited adding and removing replicas against the new target. A replica that is no longer requested leaves adding, a requested replica that hasn't caught up stays in adding, and the one-step path clears the previous move even when the new target equals the current replicas.

Assisted-by: Gemini 4 Argon

* fix(inkless:controller): expand ISR on unfence for phase-2 switched partitions

Once classicPrefixInRemoteStorage is set, a switched partition's classic prefix lives in remote storage and no replica relies on its local copy. Skip unfence ISR expansion only while isSwitchedWithLocalClassicPrefix(partition) holds, so phase-2 partitions re-add unfenced replicas (and complete in-flight reassignments) like born-diskless partitions.

Assisted-by: Gemini 4 Argon
Assisted-by: Claude Sonnet 5.5
Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(inkless:switch): align docs and seal command with switch phases

The phase rules change how a switched partition handles reassignment, ISR expansion on unfence, and `AlterDisklessSwitch`. The docs still say that diskless reassignment is always immediate and that a returning broker rejoins the ISR at once. They don't describe the phases. The `seal` command also validates only the offset range, so `--dry-run` reports an action that the controller rejects in phase 2.

Add a "Switch phases" section to `CLASSIC_TO_DISKLESS_SWITCH.md` and link it from `FEATURES.md`, `FAQ.md`, and `DISKLESS_CONSOLIDATION.md`. Document that the controller rejects every seal change once `classicPrefixInRemoteStorage` is set, in the operator override section, the `seal --offset` help, and the `Admin.alterDisklessSwitch` Javadoc.

`kafka-topic-switch.sh seal` now fails with that reason when it reads the partition, which covers `--dry-run`. The `-1` and `-2` offsets don't read the partition, because controllers don't serve `DescribeTopicPartitions`. They keep working with `--bootstrap-controller`.

Testing

Run `TopicSwitchCommandTest`, the `clients` and `tools` checkstyle tasks, and `make vale`.

Assisted-by: Claude Sonnet 5.5
Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(inkless:switch): reject seals past the committed seal in the seal command

The controller rejects a seal offset that exceeds the committed seal, but
`kafka-topic-switch.sh seal` only checked the classic log range. After a
switch, the log end offset includes diskless data, so `--dry-run` reported
"Would set" for a request the controller then rejects. The command now fails
before it sends the request, and the help text and switch doc state the rule.

The unfence ISR expansion comment now covers both cases where the unfencing
broker is accepted as leader: a leaderless partition and a reassignment that
removes the current leader. A new test pins the leaderless case for
born-diskless partitions.

The FAQ answer about a returning replica broker notes that a phase-1
switched topic rejoins the ISR through `AlterPartition`.

## Test plan

- `ReplicationControlManagerInklessTest$DisklessManagedReplicasTests.testUnfenceElectsUnfencingBrokerForLeaderlessBornDisklessPartition`:
  fails with the previous `clusterControl::isActive` predicate and passes
  with the current one.
- `TopicSwitchCommandTest`: `testSealRejectsExplicitOffsetAboveCommittedSeal`,
  `testSealDryRunRejectsExplicitOffsetAboveCommittedSeal`, and
  `testSealCommitsExplicitOffsetEqualToCommittedSeal`.

Assisted-by: Claude Opus 5.5
Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
gqmelo pushed a commit that referenced this pull request Nov 5, 2026
… on switch phase (#833)

* fix(inkless:controller): reject alterDisklessSwitch after prefix is tiered

Once classicPrefixInRemoteStorage is set, brokers delete their local prefix below the committed seal and start consolidating from it. Reject any seal modification (including lowering) once the flag is set.

Assisted-by: Gemini 4 Argon

* fix(inkless:controller): require online directory for phase-1 leader election

In phase 1 (seal present, classicPrefixInRemoteStorage unset), the classic prefix still lives on the replicas' local disks, so a broker whose replica directory is offline cannot safely become leader. Use LeaderAcceptor in both leaderAcceptorFor overloads while isSwitchedWithLocalClassicPrefix(partition) holds, and keep isActive-only for born-diskless and phase-2 partitions.

Assisted-by: Gemini 4 Argon

* fix(inkless:controller): stage reassignments of phase-1 switched partitions

In phase 1 (seal present, classicPrefixInRemoteStorage unset), records below the seal exist only on the replicas' local disks, so a reassignment must use the staged adding/removing path until the new replicas catch up. Born-diskless and phase-2 partitions keep the one-step reassignment path.

Retargeting a reassignment that is already in progress reconciles the inherited adding and removing replicas against the new target. A replica that is no longer requested leaves adding, a requested replica that hasn't caught up stays in adding, and the one-step path clears the previous move even when the new target equals the current replicas.

Assisted-by: Gemini 4 Argon

* fix(inkless:controller): expand ISR on unfence for phase-2 switched partitions

Once classicPrefixInRemoteStorage is set, a switched partition's classic prefix lives in remote storage and no replica relies on its local copy. Skip unfence ISR expansion only while isSwitchedWithLocalClassicPrefix(partition) holds, so phase-2 partitions re-add unfenced replicas (and complete in-flight reassignments) like born-diskless partitions.

Assisted-by: Gemini 4 Argon
Assisted-by: Claude Sonnet 5.5
Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(inkless:switch): align docs and seal command with switch phases

The phase rules change how a switched partition handles reassignment, ISR expansion on unfence, and `AlterDisklessSwitch`. The docs still say that diskless reassignment is always immediate and that a returning broker rejoins the ISR at once. They don't describe the phases. The `seal` command also validates only the offset range, so `--dry-run` reports an action that the controller rejects in phase 2.

Add a "Switch phases" section to `CLASSIC_TO_DISKLESS_SWITCH.md` and link it from `FEATURES.md`, `FAQ.md`, and `DISKLESS_CONSOLIDATION.md`. Document that the controller rejects every seal change once `classicPrefixInRemoteStorage` is set, in the operator override section, the `seal --offset` help, and the `Admin.alterDisklessSwitch` Javadoc.

`kafka-topic-switch.sh seal` now fails with that reason when it reads the partition, which covers `--dry-run`. The `-1` and `-2` offsets don't read the partition, because controllers don't serve `DescribeTopicPartitions`. They keep working with `--bootstrap-controller`.

Testing

Run `TopicSwitchCommandTest`, the `clients` and `tools` checkstyle tasks, and `make vale`.

Assisted-by: Claude Sonnet 5.5
Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(inkless:switch): reject seals past the committed seal in the seal command

The controller rejects a seal offset that exceeds the committed seal, but
`kafka-topic-switch.sh seal` only checked the classic log range. After a
switch, the log end offset includes diskless data, so `--dry-run` reported
"Would set" for a request the controller then rejects. The command now fails
before it sends the request, and the help text and switch doc state the rule.

The unfence ISR expansion comment now covers both cases where the unfencing
broker is accepted as leader: a leaderless partition and a reassignment that
removes the current leader. A new test pins the leaderless case for
born-diskless partitions.

The FAQ answer about a returning replica broker notes that a phase-1
switched topic rejoins the ISR through `AlterPartition`.

## Test plan

- `ReplicationControlManagerInklessTest$DisklessManagedReplicasTests.testUnfenceElectsUnfencingBrokerForLeaderlessBornDisklessPartition`:
  fails with the previous `clusterControl::isActive` predicate and passes
  with the current one.
- `TopicSwitchCommandTest`: `testSealRejectsExplicitOffsetAboveCommittedSeal`,
  `testSealDryRunRejectsExplicitOffsetAboveCommittedSeal`, and
  `testSealCommitsExplicitOffsetEqualToCommittedSeal`.

Assisted-by: Claude Opus 5.5
Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
gqmelo pushed a commit that referenced this pull request Nov 5, 2026
… on switch phase (#833)

* fix(inkless:controller): reject alterDisklessSwitch after prefix is tiered

Once classicPrefixInRemoteStorage is set, brokers delete their local prefix below the committed seal and start consolidating from it. Reject any seal modification (including lowering) once the flag is set.

Assisted-by: Gemini 4 Argon

* fix(inkless:controller): require online directory for phase-1 leader election

In phase 1 (seal present, classicPrefixInRemoteStorage unset), the classic prefix still lives on the replicas' local disks, so a broker whose replica directory is offline cannot safely become leader. Use LeaderAcceptor in both leaderAcceptorFor overloads while isSwitchedWithLocalClassicPrefix(partition) holds, and keep isActive-only for born-diskless and phase-2 partitions.

Assisted-by: Gemini 4 Argon

* fix(inkless:controller): stage reassignments of phase-1 switched partitions

In phase 1 (seal present, classicPrefixInRemoteStorage unset), records below the seal exist only on the replicas' local disks, so a reassignment must use the staged adding/removing path until the new replicas catch up. Born-diskless and phase-2 partitions keep the one-step reassignment path.

Retargeting a reassignment that is already in progress reconciles the inherited adding and removing replicas against the new target. A replica that is no longer requested leaves adding, a requested replica that hasn't caught up stays in adding, and the one-step path clears the previous move even when the new target equals the current replicas.

Assisted-by: Gemini 4 Argon

* fix(inkless:controller): expand ISR on unfence for phase-2 switched partitions

Once classicPrefixInRemoteStorage is set, a switched partition's classic prefix lives in remote storage and no replica relies on its local copy. Skip unfence ISR expansion only while isSwitchedWithLocalClassicPrefix(partition) holds, so phase-2 partitions re-add unfenced replicas (and complete in-flight reassignments) like born-diskless partitions.

Assisted-by: Gemini 4 Argon
Assisted-by: Claude Sonnet 5.5
Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(inkless:switch): align docs and seal command with switch phases

The phase rules change how a switched partition handles reassignment, ISR expansion on unfence, and `AlterDisklessSwitch`. The docs still say that diskless reassignment is always immediate and that a returning broker rejoins the ISR at once. They don't describe the phases. The `seal` command also validates only the offset range, so `--dry-run` reports an action that the controller rejects in phase 2.

Add a "Switch phases" section to `CLASSIC_TO_DISKLESS_SWITCH.md` and link it from `FEATURES.md`, `FAQ.md`, and `DISKLESS_CONSOLIDATION.md`. Document that the controller rejects every seal change once `classicPrefixInRemoteStorage` is set, in the operator override section, the `seal --offset` help, and the `Admin.alterDisklessSwitch` Javadoc.

`kafka-topic-switch.sh seal` now fails with that reason when it reads the partition, which covers `--dry-run`. The `-1` and `-2` offsets don't read the partition, because controllers don't serve `DescribeTopicPartitions`. They keep working with `--bootstrap-controller`.

Testing

Run `TopicSwitchCommandTest`, the `clients` and `tools` checkstyle tasks, and `make vale`.

Assisted-by: Claude Sonnet 5.5
Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(inkless:switch): reject seals past the committed seal in the seal command

The controller rejects a seal offset that exceeds the committed seal, but
`kafka-topic-switch.sh seal` only checked the classic log range. After a
switch, the log end offset includes diskless data, so `--dry-run` reported
"Would set" for a request the controller then rejects. The command now fails
before it sends the request, and the help text and switch doc state the rule.

The unfence ISR expansion comment now covers both cases where the unfencing
broker is accepted as leader: a leaderless partition and a reassignment that
removes the current leader. A new test pins the leaderless case for
born-diskless partitions.

The FAQ answer about a returning replica broker notes that a phase-1
switched topic rejoins the ISR through `AlterPartition`.

## Test plan

- `ReplicationControlManagerInklessTest$DisklessManagedReplicasTests.testUnfenceElectsUnfencingBrokerForLeaderlessBornDisklessPartition`:
  fails with the previous `clusterControl::isActive` predicate and passes
  with the current one.
- `TopicSwitchCommandTest`: `testSealRejectsExplicitOffsetAboveCommittedSeal`,
  `testSealDryRunRejectsExplicitOffsetAboveCommittedSeal`, and
  `testSealCommitsExplicitOffsetEqualToCommittedSeal`.

Assisted-by: Claude Opus 5.5
Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants