diff --git a/CHANGELOG.md b/CHANGELOG.md index f1f208a52..162f4e7f1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,13 @@ The backend (`fastapi_backend`) and the frontend (`nextjs-frontend`) are version ## Unreleased +- **Install on Proxmox VE with one command.** A [community helper script](https://community-scripts.org/docs/ct/detailed_guide) + (`proxmox/ct/lens.sh`, run in the Proxmox host's shell) creates a Debian 13 LXC container running Lens without + Docker: SurrealDB 3.2.4, the API, a job worker and the web app as systemd services, with fresh secrets and the web + app on port 3000. `lens-setup-code` in the container prints the first-admin setup code; after the admin, the setup + wizard (`/welcome`) asks for the first namespace, the model provider and storage. `update` moves to the + latest published GitHub release (the newest `main` until there is one), building it beside the running version + so Lens stays up until the switch (proxmox/README.md). - **Synology package.** `packaging/synology/build.sh` builds a self-contained `.spk` for DSM 7.2.1+ (x86_64, or 64-bit ARM with `ARCH=armv8`): Manual Install it in Package Center, answer a short wizard (port, address, setup code), and Lens runs. The package carries the API/worker, web app and SurrealDB images and hands them to Container diff --git a/proxmox/README.md b/proxmox/README.md new file mode 100644 index 000000000..c43f511c2 --- /dev/null +++ b/proxmox/README.md @@ -0,0 +1,55 @@ +# Lens on Proxmox VE + +A [Proxmox VE community helper script](https://community-scripts.org/docs/ct/detailed_guide) that creates a Debian 13 +LXC container running Lens without Docker: SurrealDB, the API, a job worker and the web app as systemd services. + +Run it in the Proxmox VE host's shell: + +```bash +bash -c "$(curl -fsSL https://raw.githubusercontent.com/adeelahmad/lense/main/proxmox/ct/lens.sh)" +``` + +The script uses the community-scripts engine (`community-scripts/core`) for the container, and takes its own +`ct/lens.sh` and `install/lens-install.sh` from this folder. When it finishes, open `http://:3000` and +create the first admin with the setup code (run `lens-setup-code` in the container). The setup wizard at `/welcome` +then asks for the first namespace, the model provider and storage; every step can be skipped. + +Defaults: 4 cores, 4 GB RAM (the web app's build needs it), 20 GB disk, unprivileged, amd64 or arm64. Override them in +the script's advanced settings, or with `var_cpu=… var_ram=… var_disk=…` before the command. + +## What's in the container + +| | | +|---|---| +| Code | `/opt/lens`, a link to `/opt/lens-`: a shallow clone of the latest GitHub release, or of `main` while there is none | +| Settings | `/etc/lens/lens.env` (API and worker), `/etc/lens/web.env` (web app), `/etc/lens/archive.yaml` | +| Data | `/var/lib/lens` (archive data and uploads; folders to watch go in `media/`), `/var/lib/surrealdb` | +| Services | `surrealdb` (127.0.0.1:8001), `lens-api` (127.0.0.1:8000), `lens-worker`, `lens-web` (port 3000) | + +Only the web app listens on the network; it proxies the API's paths. The secrets in `/etc/lens/*.env` are generated +once at install and kept by updates. Logs: `journalctl -u lens-api -f` (and `lens-worker`, `lens-web`, `surrealdb`). + +The container runs the lean image's tools (ffmpeg, Tesseract, poppler, rclone). To read Word, PowerPoint and +spreadsheet files, text, web pages and emails too, install what the `full` image adds +(`apt install libreoffice-writer-nogui libreoffice-calc-nogui libreoffice-impress-nogui chromium fonts-liberation2`); +see [Deployment](../docs/deployment.md) for HTTPS, allowed hosts and backups, which apply here as well. + +## Updating + +Run `update` in the container's console. It moves to the latest published GitHub release (the newest commit of `main` +while the repository has no release). It builds the new version beside the running one, so Lens stays up during the +build and a failed build leaves it as it was; then it stops the services, switches `/opt/lens` over, starts them again +and removes the old version. It does nothing when the container is already there. Publishing a release is all it takes +for containers to pick it up: there are no release assets to build for Proxmox. SurrealDB is upgraded when the pinned +version in `ct/lens.sh` changes. + +## Testing a branch + +Point the engine at a branch of this repository: + +```bash +_CS_DEFAULT_URL=https://raw.githubusercontent.com/adeelahmad/lense//proxmox \ + bash -c "$(curl -fsSL https://raw.githubusercontent.com/adeelahmad/lense//proxmox/ct/lens.sh)" +``` + +This picks the scripts from that branch; the container still installs the latest release of Lens (or `main`). diff --git a/proxmox/ct/lens.sh b/proxmox/ct/lens.sh new file mode 100755 index 000000000..1936ee7b5 --- /dev/null +++ b/proxmox/ct/lens.sh @@ -0,0 +1,119 @@ +#!/usr/bin/env bash +# Scripts (this file and install/lens-install.sh) come from the Lens repository; the engine from community-scripts/core. +_CS_DEFAULT_URL="${_CS_DEFAULT_URL:-https://raw.githubusercontent.com/adeelahmad/lense/main/proxmox}" +_cs_boot="${COMMUNITY_SCRIPTS_CORE_DIR:-$(dirname "${BASH_SOURCE[0]}")/../../core}/core/build.func" +source "$_cs_boot" 2>/dev/null || source <(curl -fsSL "${COMMUNITY_SCRIPTS_CORE_URL:-https://raw.githubusercontent.com/community-scripts/core/main}/core/build.func") +# Copyright (c) 2021-2026 community-scripts ORG +# Author: adeelahmad +# License: MIT | https://github.com/community-scripts/ProxmoxVE/raw/main/LICENSE +# Source: https://github.com/adeelahmad/lense + +APP="Lens" +var_tags="${var_tags:-archive;media;ai}" +var_cpu="${var_cpu:-4}" +var_ram="${var_ram:-4096}" +var_disk="${var_disk:-20}" +var_os="${var_os:-debian}" +var_version="${var_version:-13}" +var_arm64="${var_arm64:-yes}" +var_unprivileged="${var_unprivileged:-1}" + +header_info "$APP" +variables +color +catch_errors + +function update_script() { + header_info + check_container_storage + check_container_resources + if [[ ! -d /opt/lens/.git ]]; then + msg_error "No ${APP} Installation Found!" + exit + fi + + SURREALDB_VERSION="3.2.4" + + ensure_dependencies jq + msg_info "Checking for update: ${APP}" + # The latest published GitHub release; main until there is one + LENS_REF="$(curl -fsSL https://api.github.com/repos/adeelahmad/lense/releases/latest 2>/dev/null | jq -r '.tag_name // empty' || true)" + LENS_REF="${LENS_REF:-main}" + if [[ "$LENS_REF" == "main" ]]; then + REFSPEC="refs/heads/main" + else + REFSPEC="refs/tags/${LENS_REF}" + fi + CURRENT="$(git -C /opt/lens rev-parse HEAD)" + # An annotated tag lists its commit last, as ^{} + LATEST="$(git -C /opt/lens ls-remote origin "${REFSPEC}" "${REFSPEC}^{}" | tail -1 | cut -f1)" + if [[ -z "$LATEST" ]]; then + msg_error "Could not reach the Lens repository" + exit + fi + if [[ "$CURRENT" == "$LATEST" ]]; then + msg_ok "No update available: ${APP} ${LENS_REF} (${CURRENT:0:7})" + exit + fi + msg_ok "Update available: ${APP} ${CURRENT:0:7} → ${LENS_REF} (${LATEST:0:7})" + + # Build the new version beside the running one; Lens keeps running until it has built + NEW_DIR="/opt/lens-${LATEST:0:12}" + OLD_DIR="$(readlink -f /opt/lens)" + msg_info "Fetching Lens ${LENS_REF}" + rm -rf "$NEW_DIR" + $STD git clone --depth 1 --branch "${LENS_REF}" "$(git -C /opt/lens remote get-url origin)" "$NEW_DIR" + msg_ok "Fetched Lens ${LENS_REF}" + + msg_info "Building Lens API (patience)" + cd "$NEW_DIR/fastapi_backend" + export UV_PYTHON="3.12" + $STD uv sync --frozen --no-dev + msg_ok "Built Lens API" + + msg_info "Building Lens web app (patience)" + cd "$NEW_DIR/nextjs-frontend" + export NEXT_TELEMETRY_DISABLED=1 CI=true AUTH_SECRET=build-only + $STD pnpm install --frozen-lockfile + $STD pnpm build + unset AUTH_SECRET + msg_ok "Built Lens web app" + + msg_info "Stopping Services" + systemctl stop lens-web lens-worker lens-api + msg_ok "Stopped Services" + + if [[ "$(cat ~/.surrealdb 2>/dev/null)" != "$SURREALDB_VERSION" ]]; then + msg_info "Updating SurrealDB to ${SURREALDB_VERSION}" + systemctl stop surrealdb + curl -fsSL "https://github.com/surrealdb/surrealdb/releases/download/v${SURREALDB_VERSION}/surreal-v${SURREALDB_VERSION}.linux-$(arch_resolve amd64 arm64).tgz" | + tar -xz -C /usr/local/bin surreal + echo "${SURREALDB_VERSION}" >~/.surrealdb + systemctl start surrealdb + msg_ok "Updated SurrealDB to ${SURREALDB_VERSION}" + fi + + msg_info "Switching to the new version" + ln -sfn "$NEW_DIR" /opt/lens + git -C /opt/lens rev-parse HEAD >~/.lens + if [[ "$OLD_DIR" != "$NEW_DIR" ]]; then + rm -rf "$OLD_DIR" + fi + msg_ok "Switched to Lens ${LENS_REF} (${LATEST:0:7})" + + msg_info "Starting Services" + systemctl start lens-api lens-worker lens-web + msg_ok "Started Services" + msg_ok "Updated successfully!" + exit +} + +start +build_container +description + +msg_ok "Completed successfully!\n" +echo -e "${CREATING}${GN}${APP} setup has been successfully initialized!${CL}" +echo -e "${INFO}${YW}Access it using the following URL:${CL}" +echo -e "${GATEWAY}${BGN}http://${IP}:3000${CL}" +echo -e "${INFO}${YW}First admin: run 'lens-setup-code' in the container for the setup code, then follow the setup steps at /welcome.${CL}" diff --git a/proxmox/install/lens-install.sh b/proxmox/install/lens-install.sh new file mode 100755 index 000000000..21b00d9fb --- /dev/null +++ b/proxmox/install/lens-install.sh @@ -0,0 +1,202 @@ +#!/usr/bin/env bash + +# Copyright (c) 2021-2026 community-scripts ORG +# Author: adeelahmad +# License: MIT | https://github.com/community-scripts/ProxmoxVE/raw/main/LICENSE +# Source: https://github.com/adeelahmad/lense + +source /dev/stdin <<<"$FUNCTIONS_FILE_PATH" +color +verb_ip6 +catch_errors +setting_up_container +network_check +update_os + +LENS_REPO="https://github.com/adeelahmad/lense.git" +SURREALDB_VERSION="3.2.4" + +msg_info "Installing Dependencies" +# ffmpeg: audio and video; tesseract: text on screen; poppler: PDF fallback (as in fastapi_backend/Dockerfile) +$STD apt install -y \ + git \ + jq \ + ca-certificates \ + ffmpeg \ + tesseract-ocr \ + poppler-utils +# antiword reads legacy .doc files; skip it where the release no longer ships it +if apt-cache show antiword &>/dev/null; then + $STD apt install -y antiword +fi +msg_ok "Installed Dependencies" + +fetch_and_deploy_gh_release "rclone" "rclone/rclone" "binary" +PYTHON_VERSION="3.12" setup_uv +NODE_VERSION="22" NODE_MODULE="pnpm@10.7.1" setup_nodejs + +msg_info "Installing SurrealDB ${SURREALDB_VERSION}" +curl -fsSL "https://github.com/surrealdb/surrealdb/releases/download/v${SURREALDB_VERSION}/surreal-v${SURREALDB_VERSION}.linux-$(arch_resolve amd64 arm64).tgz" | + tar -xz -C /usr/local/bin surreal +chmod +x /usr/local/bin/surreal +echo "${SURREALDB_VERSION}" >~/.surrealdb +mkdir -p /var/lib/surrealdb +msg_ok "Installed SurrealDB ${SURREALDB_VERSION}" + +msg_info "Fetching Lens" +# The latest published GitHub release; main until there is one +LENS_REF="$(curl -fsSL https://api.github.com/repos/adeelahmad/lense/releases/latest 2>/dev/null | jq -r '.tag_name // empty' || true)" +LENS_REF="${LENS_REF:-main}" +# Each version lives in /opt/lens-; /opt/lens points at the running one, so `update` can build the next +# version beside it and switch over only once the build succeeded. +$STD git clone --depth 1 --branch "${LENS_REF}" "${LENS_REPO}" /opt/lens-new +git -C /opt/lens-new rev-parse HEAD >~/.lens +mv /opt/lens-new "/opt/lens-$(cut -c1-12 ~/.lens)" +ln -sfn "/opt/lens-$(cut -c1-12 ~/.lens)" /opt/lens +msg_ok "Fetched Lens ${LENS_REF} ($(cut -c1-7 ~/.lens))" + +msg_info "Configuring Lens" +LENS_IP="${LOCAL_IP:-$(hostname -I | awk '{print $1}')}" +SURREAL_PASS="$(openssl rand -hex 16)" +mkdir -p /etc/lens /var/lib/lens /var/lib/lens/media +cat </etc/lens/surrealdb.env +SURREAL_USER=root +SURREAL_PASS=${SURREAL_PASS} +SURREAL_BIND=127.0.0.1:8001 +SURREAL_LOG=info +EOF +cat </etc/lens/lens.env +# Lens API and worker. Keep the secrets stable: ACCESS_SECRET_KEY signs sessions, +# ARCHIVE_SECRET_KEY encrypts stored source credentials and LLM keys. +ACCESS_SECRET_KEY=$(openssl rand -hex 32) +ARCHIVE_SECRET_KEY=$(openssl rand -hex 32) +ARCHIVE_CONFIG=/etc/lens/archive.yaml +SURREAL_URL=ws://127.0.0.1:8001 +SURREAL_USER=root +SURREAL_PASS=${SURREAL_PASS} +RUN_BACKGROUND=false +FRONTEND_URL=http://${LENS_IP}:3000 +CORS_ORIGINS=["http://${LENS_IP}:3000"] +# Password reset mail. Unset MAIL_SERVER: reset links are written to the log instead. +# MAIL_SERVER=smtp.example.org +# MAIL_PORT=587 +# MAIL_USERNAME= +# MAIL_PASSWORD= +# MAIL_FROM=lens@example.org +EOF +cat </etc/lens/web.env +# Lens web app (Next.js). It proxies /api/v1, /iiif, /embed, /s, /reports and /static to the API. +API_BASE_URL=http://127.0.0.1:8000 +AUTH_SECRET=$(openssl rand -base64 32) +AUTH_TRUST_HOST=true +NODE_ENV=production +NEXT_TELEMETRY_DISABLED=1 +EOF +cat </etc/lens/archive.yaml +# Lens processing configuration; settings saved in the web app take precedence. See archive.example.yaml. +# No namespaces here: the setup wizard asks for the first one after the admin account is created. +data_dir: /var/lib/lens +server: + allowed_hosts: [127.0.0.1, localhost] +workers: + inline: 0 # lens-worker.service does the work +sources: + local_roots: [/var/lib/lens/media] # the setup wizard (/welcome) can watch a folder inside it +EOF +chmod 600 /etc/lens/*.env +msg_ok "Configured Lens" + +msg_info "Building Lens API (patience)" +cd /opt/lens/fastapi_backend +export UV_PYTHON="3.12" +$STD uv sync --frozen --no-dev +msg_ok "Built Lens API" + +msg_info "Building Lens web app (patience)" +cd /opt/lens/nextjs-frontend +export NEXT_TELEMETRY_DISABLED=1 CI=true AUTH_SECRET=build-only +$STD pnpm install --frozen-lockfile +$STD pnpm build +unset AUTH_SECRET +msg_ok "Built Lens web app" + +msg_info "Creating Services" +cat </etc/systemd/system/surrealdb.service +[Unit] +Description=SurrealDB for Lens +After=network.target + +[Service] +EnvironmentFile=/etc/lens/surrealdb.env +ExecStart=/usr/local/bin/surreal start surrealkv:/var/lib/surrealdb/lens.db +Restart=always + +[Install] +WantedBy=multi-user.target +EOF + +cat </etc/systemd/system/lens-api.service +[Unit] +Description=Lens API +After=network.target surrealdb.service +Requires=surrealdb.service + +[Service] +WorkingDirectory=/opt/lens/fastapi_backend +EnvironmentFile=/etc/lens/lens.env +ExecStart=/opt/lens/fastapi_backend/.venv/bin/fastapi run app/main.py --host 127.0.0.1 --port 8000 --proxy-headers +Restart=always + +[Install] +WantedBy=multi-user.target +EOF + +cat </etc/systemd/system/lens-worker.service +[Unit] +Description=Lens Worker +After=network.target surrealdb.service lens-api.service +Requires=surrealdb.service + +[Service] +WorkingDirectory=/opt/lens/fastapi_backend +EnvironmentFile=/etc/lens/lens.env +ExecStart=/opt/lens/fastapi_backend/.venv/bin/lens worker +Restart=always + +[Install] +WantedBy=multi-user.target +EOF + +cat </etc/systemd/system/lens-web.service +[Unit] +Description=Lens Web +After=network.target lens-api.service + +[Service] +WorkingDirectory=/opt/lens/nextjs-frontend +EnvironmentFile=/etc/lens/web.env +ExecStart=/opt/lens/nextjs-frontend/node_modules/.bin/next start -H 0.0.0.0 -p 3000 +Restart=always + +[Install] +WantedBy=multi-user.target +EOF + +cat <<'EOF' >/usr/local/bin/lens-setup-code +#!/usr/bin/env bash +# The first-admin setup code, from the API log (printed only until the first account exists) +journalctl -u lens-api --no-pager | grep -i "setup code" | tail -1 +EOF +chmod +x /usr/local/bin/lens-setup-code + +systemctl enable -q --now surrealdb +for _ in $(seq 1 30); do + /usr/local/bin/surreal is-ready --endpoint http://127.0.0.1:8001 &>/dev/null && break + sleep 1 +done +systemctl enable -q --now lens-api lens-worker lens-web +msg_ok "Created Services" + +motd_ssh +customize +cleanup_lxc