Skip to content

Don't back up inactive local *.sql, don't restore an RBAC object twice - #1690

Open
devmalcevv wants to merge 4 commits into
Altinity:masterfrom
devmalcevv:rbac_user_directories_1666
Open

devmalcevv wants to merge 4 commits into
Altinity:masterfrom
devmalcevv:rbac_user_directories_1666

Conversation

@devmalcevv

Copy link
Copy Markdown

Fixes #1666

Summary

  • create --rbac skips <access path>/*.sql when system.user_directories has no local_directory and logs how many files were skipped.
  • restore --rbac on a server with one user directory no longer writes an object twice when the backup has it both as *.sql and in a Keeper dump. The Keeper copy is restored.
  • New integration test TestRBACInactiveLocalDirectory, unit tests, ChangeLog.md, Examples.md.

New backups won't have duplicates while restoring old backups will favour Keeper version.

Changes

  • createBackupRBAC: when *.sql exist, the existing splitUserDirectories helper decides whether to copy them. If system.user_directories can't be read, it assumes a local directory, so the old behaviour is kept.
  • restoreRBAC: for backups made before the fix, findRBACSQLFilesDuplicatedInKeeperDump finds *.sql whose object (kind and full name, plus the table for row policies) is also in a Keeper dump, and adds them to the existing ignoredSQLFiles.
  • detectRBACObjectID reads back-quoted names whole. detectRBACObject cuts john smith to john; it is left unchanged.

Tests

  • make test pass, make build* ok
  • TestRBACInactiveLocalDirectory (integration): inactive *.sql are not backed up, and an old backup with duplicates restores every object once.
  • TestFindRBACSQLFilesDuplicatedInKeeperDump (unit): finds *.sql duplicated in a Keeper dump.
  • TestDetectRBACObjectID (unit): parses the full name of an RBAC object, with or without special symbols - backquotes, escaped backquotes, spaces.

Potential risks

  • Server with both user directories: an older backup still restores stale *.sql into local_directory. Unchanged behaviour, workaround in Examples.md.

devmalcevv and others added 4 commits November 18, 2026 15:17
…y#1666

createBackupRBAC copied <access path>/*.sql into the backup even when
system.user_directories has no `local_directory`. clickhouse-server
doesn't read these files then, they are leftovers of a switch from
`local_directory` to `replicated` access storage, which
GetAccessManagementPath still finds through its config.xml /
system.disks fallback.

Such a backup carried every RBAC object twice, the inactive copy as
<uuid>.sql and the live one in the keeper dump, same name, different
uuid. Since 2.8.1 restore converts *.sql to keeper on a server with only
a `replicated` user directory, so it wrote both copies: the inactive
definition replaced the live one (the old password worked, the current
one didn't), and after the next clickhouse-server restart both were
skipped with "due to conflicts with other access entities".

Copy *.sql only when system.user_directories contains `local_directory`.
The check runs only when the access path has *.sql files, and a local
directory is still assumed when system.user_directories can't be read.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Backups created before the previous commit keep inactive <uuid>.sql
next to the keeper dump with the same RBAC objects, see
Altinity#1666, and such
backups can't be re-created. Restore still wrote both copies when the
server has only one of the two user directories:

- only `replicated`: *.sql are converted to keeper after the dump is
  restored, the inactive definition replaces the live one and
  clickhouse-server skips both after restart;
- only `local_directory`: the dump is converted to *.sql next to the
  copied ones, clickhouse-server keeps one file per object and removes
  the other.

Find *.sql with the same kind and name as an object in the keeper dumps
of the backup and add them to ignoredSQLFiles, so the keeper dump copy,
which has the live objects, is the only one restored. Servers with both
user directories are not affected, *.sql and the keeper dump go to
different storages there as before.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
findRBACSQLFilesDuplicatedInKeeperDump compared the names returned by
detectRBACObject, which cuts a name on the first space: users
`john smith` from the keeper dump and `john doe` from *.sql are both
`john` for it, so *.sql of a different user would be skipped as
a duplicate.

Add detectRBACObjectID, it reads a back quoted identifier as a whole
and returns it without back quotes, policies also get their database
and table. clickhouse-server 22.3 writes `ATTACH SETTINGS PROFILE name`
and 26.8 writes the same name back quoted, both escape a back quote
inside a name with a backslash.

detectRBACObject is not changed, conflict resolution and keeper node
names still use it.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Add the ChangeLog.md entry for Altinity#1666 and update "How to restore RBAC
objects between different user_directories types" in Examples.md:
*.sql are backed up only with a `local_directory` user directory,
restore of older backups with duplicated objects takes the keeper dump
copy on a server with one user directory, and what to do on a server
with both, where the two copies still go to different user directories.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@devmalcevv
devmalcevv marked this pull request as ready for review November 18, 2026 11:43
@Slach Slach added this to the 2.8.4 milestone Nov 18, 2026
@Slach

Slach commented Nov 18, 2026

Copy link
Copy Markdown
Collaborator

thanks for contribution will review little bit later

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

clickhouse-backup broken RBAC restore when inactive local access files exist but are not in user_directories list

2 participants