Repository navigation
Don't back up inactive local *.sql, don't restore an RBAC object twice - #1690
Open
devmalcevv wants to merge 4 commits into
Open
devmalcevv wants to merge 4 commits into
devmalcevv wants to merge 4 commits into
Conversation
…y#1666 createBackupRBAC copied <access path>/*.sql into the backup even when system.user_directories has no `local_directory`. clickhouse-server doesn't read these files then, they are leftovers of a switch from `local_directory` to `replicated` access storage, which GetAccessManagementPath still finds through its config.xml / system.disks fallback. Such a backup carried every RBAC object twice, the inactive copy as <uuid>.sql and the live one in the keeper dump, same name, different uuid. Since 2.8.1 restore converts *.sql to keeper on a server with only a `replicated` user directory, so it wrote both copies: the inactive definition replaced the live one (the old password worked, the current one didn't), and after the next clickhouse-server restart both were skipped with "due to conflicts with other access entities". Copy *.sql only when system.user_directories contains `local_directory`. The check runs only when the access path has *.sql files, and a local directory is still assumed when system.user_directories can't be read. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Backups created before the previous commit keep inactive <uuid>.sql next to the keeper dump with the same RBAC objects, see Altinity#1666, and such backups can't be re-created. Restore still wrote both copies when the server has only one of the two user directories: - only `replicated`: *.sql are converted to keeper after the dump is restored, the inactive definition replaces the live one and clickhouse-server skips both after restart; - only `local_directory`: the dump is converted to *.sql next to the copied ones, clickhouse-server keeps one file per object and removes the other. Find *.sql with the same kind and name as an object in the keeper dumps of the backup and add them to ignoredSQLFiles, so the keeper dump copy, which has the live objects, is the only one restored. Servers with both user directories are not affected, *.sql and the keeper dump go to different storages there as before. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
findRBACSQLFilesDuplicatedInKeeperDump compared the names returned by detectRBACObject, which cuts a name on the first space: users `john smith` from the keeper dump and `john doe` from *.sql are both `john` for it, so *.sql of a different user would be skipped as a duplicate. Add detectRBACObjectID, it reads a back quoted identifier as a whole and returns it without back quotes, policies also get their database and table. clickhouse-server 22.3 writes `ATTACH SETTINGS PROFILE name` and 26.8 writes the same name back quoted, both escape a back quote inside a name with a backslash. detectRBACObject is not changed, conflict resolution and keeper node names still use it. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Add the ChangeLog.md entry for Altinity#1666 and update "How to restore RBAC objects between different user_directories types" in Examples.md: *.sql are backed up only with a `local_directory` user directory, restore of older backups with duplicated objects takes the keeper dump copy on a server with one user directory, and what to do on a server with both, where the two copies still go to different user directories. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
devmalcevv
marked this pull request as ready for review
November 18, 2026 11:43
Collaborator
|
thanks for contribution will review little bit later |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1666
Summary
create --rbacskips<access path>/*.sqlwhensystem.user_directorieshas nolocal_directoryand logs how many files were skipped.restore --rbacon a server with one user directory no longer writes an object twice when the backup has it both as*.sqland in a Keeper dump. The Keeper copy is restored.TestRBACInactiveLocalDirectory, unit tests,ChangeLog.md,Examples.md.New backups won't have duplicates while restoring old backups will favour Keeper version.
Changes
createBackupRBAC: when*.sqlexist, the existingsplitUserDirectorieshelper decides whether to copy them. Ifsystem.user_directoriescan't be read, it assumes a local directory, so the old behaviour is kept.restoreRBAC: for backups made before the fix,findRBACSQLFilesDuplicatedInKeeperDumpfinds*.sqlwhose object (kind and full name, plus the table for row policies) is also in a Keeper dump, and adds them to the existingignoredSQLFiles.detectRBACObjectIDreads back-quoted names whole.detectRBACObjectcutsjohn smithtojohn; it is left unchanged.Tests
make testpass,make build*okTestRBACInactiveLocalDirectory(integration): inactive *.sql are not backed up, and an old backup with duplicates restores every object once.TestFindRBACSQLFilesDuplicatedInKeeperDump(unit): finds *.sql duplicated in a Keeper dump.TestDetectRBACObjectID(unit): parses the full name of an RBAC object, with or without special symbols - backquotes, escaped backquotes, spaces.Potential risks
*.sqlintolocal_directory. Unchanged behaviour, workaround inExamples.md.