Skip to content

fix(worker): scan watched folders and run routines from lens worker - #42

Merged
adeelahmad merged 2 commits into
mainfrom
claude/e2e-fixes-elucss
Oct 2, 2026
Merged

adeelahmad merged 2 commits into
mainfrom
claude/e2e-fixes-elucss

Conversation

@adeelahmad

Copy link
Copy Markdown
Owner

Requested by Adeel · project thread

Description

Before: in make dev, make run and every package (Synology, QNAP, Cloudron, Proxmox), a watched folder showed "Last scan never" forever, and routines never ran on their schedule.

After: the worker scans watched folders and runs routines when they are due, so new files come in and scheduled routines run.

Motivation and Context

Found while testing main end to end in Docker. Every Docker and package setup runs the API with RUN_BACKGROUND=false and a separate lens worker. But only the API's background work polled watched folders and ran routines. lens worker only ran jobs and sent notifications, and nothing ran lens watch.

How: the two scheduling threads move into routines.start(). The API's background work and lens worker both call it. A worker limited with --steps (a Mac mlx worker) doesn't run them, and neither does one started with --no-schedule. sources.poll_due now claims a folder scan before it starts, as routine runs already are, so two processes never scan the same folder at once.

Also fixes test_watch_preview_counts_files, which failed wherever LibreOffice is installed. A .txt file counts as a document there. CI skips this test because it has no rclone.

Steps to reproduce (if appropriate):

make dev, finish the setup wizard with a watched folder, then open Sources. Before this change it shows "Last scan never" indefinitely.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change; ! in the title)

Checklist:

  • The title is a Conventional Commit.
  • My code follows the code style of this project (make hooks checks it on commit).
  • I added or updated tests.
  • My change requires documentation updates, and I have updated docs/ accordingly.
  • Users need more than the title to understand it, so I wrote it under ## Unreleased in CHANGELOG.md.

Validated locally because GitHub Actions jobs on this repo currently fail within seconds, with no logs. I ran ruff, ruff format, mypy, the OpenAPI diff, and test_sources plus test_routines (with rclone) on both embedded SurrealDB and a SurrealDB 3.2.4 server.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Ja26f8Npk3M6ixFhE1WEfe


Generated by Claude Code

Docker, Synology, QNAP, Cloudron and Proxmox all run the API with
RUN_BACKGROUND=false and a separate lens worker, but only the API's
background work polled watched folders and ran routines, so neither
ever happened there. lens worker now starts both (unless limited with
--steps or --no-schedule). Folder scans are claimed before they run.

Also makes the watch preview test hold where LibreOffice is installed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ja26f8Npk3M6ixFhE1WEfe
@adeelahmad adeelahmad self-assigned this Oct 2, 2026

Copy link
Copy Markdown
Owner Author

Every check here fails within about 3 seconds and leaves no logs. The same happens on every push to main since the merges today, including the PR title check, so it isn't caused by this PR. It looks like the repository has run out of GitHub-hosted Actions minutes; #27 moves CI to the self-hosted runner.

I ran the CI steps locally on this head instead, and all of them passed:

  • ruff check, ruff format --check, mypy, and the OpenAPI schema diff
  • pre-commit from main to HEAD
  • the full backend suite on embedded SurrealDB (307 passed, with rclone and LibreOffice installed)
  • test_sources and test_routines against a SurrealDB 3.2.4 server

I also checked it in the dev Docker stack, where the watched folder now gets scanned. Merging on that basis, under the project's merge policy.


Generated by Claude Code

@adeelahmad
adeelahmad merged commit 49bfe25 into main Oct 2, 2026
0 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants