Skip to content

fix(backups): keep and restore Lens's files with each backup, not only the database - #385

Merged
adeelahmad merged 2 commits into
mainfrom
claude/project-thread-rj1jwj
Oct 15, 2026
Merged

adeelahmad merged 2 commits into
mainfrom
claude/project-thread-rj1jwj

Conversation

@adeelahmad

Copy link
Copy Markdown
Owner

Description

Before: a backup held only the database. Uploads, objects, media, frames, renditions, web pages, reports, exports, podcasts and certificates under data_dir were not in it, so restoring one brought back records whose files were missing.

After: every backup (manual, scheduled, before-upgrade, before-restore, lens backup) also keeps those files in <data_dir>/backups/.files/<name>/, and restoring puts them back. Files that haven't changed since the previous backup are hard links to its copy, so later backups take only the room of what changed. The download and the sealed copy on a storage connection carry the files too, as one .tar.gz holding <name> and <name>.files/, and Bring back restores them. Left out are things Lens fetches or rebuilds itself (models, bin, python, cache, the search index), temporary files (tmp, locks, logs, unfinished uploads) and secret.key. Older backups show database only and leave files alone when restored.

How: there is a new app/domain/datafiles.py with snapshot, restore, drop and tidy. migrations.backup snapshots after the database part. backups.restore restores the files first, puts them back from the before-restore backup if the database part fails, and delete/prune drop them. A backup is refused when the disk hasn't room for the changed files. Backup.files was added to the API.

Motivation and Context

Adeel: "Backup is only backing up db but not assets etc".

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change; ! in the title)

Checklist:

  • The title is a Conventional Commit.
  • My code follows the code style of this project (make hooks checks it on commit).
  • I added or updated tests (tests/api/test_backups.py: kept, linked, downloaded, restored, deleted, copied to a connection and brought back; embedded and SurrealDB server).
  • My change requires documentation updates, and I have updated docs/ accordingly (docs/database.md#backups, docs/deployment.md).
  • Users need more than the title to understand it, so I wrote it under ## Unreleased in CHANGELOG.md.

Done questions (AGENTS.md)

  • Goal: backups and restore now cover the files too; merges on green CI.
  • Agents: the same /api/v1/admin/backups API, with files on each backup.
  • Zero setup: nothing new to install or set.
  • Fewest clicks and keystrokes: no new steps.
  • Configurable: n/a, no new setting.
  • Light: hard links keep repeat backups small; the first backup takes as much room as the files, and is refused with a clear message when the disk is short.
  • Private and secure: files stay local; copies off the machine are sealed with the existing secret.key-derived key; secret.key is never included.
  • Cost: n/a, no model calls.
  • User decides: restore still needs the passkey admin and the typed name.
  • Upgrades: older backups keep working (database only), with no stored-data change.
  • Honest: the docs describe what is kept and what is left out.
  • Tests, docs and generated files (make openapi) updated.
  • Open standards; no new dependencies.
  • Result reported in the project thread. Refine later: show how much room the hard-linked copies really take, rather than their apparent size.

🤖 Generated with Claude Code

https://claude.ai/code/session_012qkVq3xnW9YJboxRYV4qSv


Generated by Claude Code

…y the database

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012qkVq3xnW9YJboxRYV4qSv
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012qkVq3xnW9YJboxRYV4qSv
@adeelahmad
adeelahmad merged commit 648ed31 into main Oct 15, 2026
10 checks passed
@adeelahmad
adeelahmad deleted the claude/project-thread-rj1jwj branch October 15, 2026 08:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants