Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
176 changes: 176 additions & 0 deletions .github/workflows/images.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,176 @@
name: Images

# Lens's own images, prebuilt for amd64 and arm64 (a Raspberry Pi, Apple silicon, Graviton), pushed to GHCR for each
# release so an install can pull them instead of building (docs/deployment.md#prebuilt-images):
#
# ghcr.io/<owner>/lens-backend:X.Y.Z-full (also X.Y.Z, latest-full, latest) the API and workers, full edition
# ghcr.io/<owner>/lens-backend:X.Y.Z-lean (also latest-lean) the same without LibreOffice and Chromium
# ghcr.io/<owner>/lens-frontend:X.Y.Z (also latest) the web app
#
# Each architecture builds on a runner of its own (GitHub's arm64 runner for arm64, no emulation), pushes by digest,
# and a last job joins the two into one tag. A release starts it with `tag` (a release published with GITHUB_TOKEN
# fires no release event, so release.yml runs it); by hand, give a release tag to (re)publish that version's images.
# A pull request that changes this file builds them all without pushing. Only the built-in GITHUB_TOKEN is used.
on:
release:
types: [published]
pull_request:
paths: [.github/workflows/images.yml]
workflow_dispatch:
inputs:
tag:
description: "Release tag to build and push (vX.Y.Z)"
required: true
type: string

concurrency:
group: images-${{ github.event.release.tag_name || inputs.tag || github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}

env:
TAG: ${{ github.event.release.tag_name || inputs.tag }}
PUSH: ${{ github.event_name != 'pull_request' }}

jobs:
build:
name: Build ${{ matrix.image.name }} (${{ matrix.arch }})
strategy:
fail-fast: false
matrix:
image:
- { name: backend-full, package: lens-backend, context: fastapi_backend, file: fastapi_backend/Dockerfile, target: full }
- { name: backend-lean, package: lens-backend, context: fastapi_backend, file: fastapi_backend/Dockerfile, target: lean }
- { name: frontend, package: lens-frontend, context: nextjs-frontend, file: nextjs-frontend/Dockerfile.prod, target: "" }
arch: [amd64, arm64]
runs-on: ${{ matrix.arch == 'arm64' && 'ubuntu-24.04-arm' || 'ubuntu-latest' }}
timeout-minutes: 120
permissions:
contents: read
packages: write
steps:
- name: Only a release tag
if: env.PUSH == 'true'
run: |
[[ "$TAG" =~ ^v[0-9]+\.[0-9]+\.[0-9]+$ ]] || { echo "::error::'$TAG' isn't a release tag (vX.Y.Z)"; exit 1; }

- uses: actions/checkout@v7
with:
ref: ${{ env.TAG || github.ref }}

- name: Image name
id: meta
env:
PACKAGE: ${{ matrix.image.package }}
run: echo "image=ghcr.io/${GITHUB_REPOSITORY_OWNER,,}/$PACKAGE" >> "$GITHUB_OUTPUT"

- uses: docker/setup-buildx-action@v4

- if: env.PUSH == 'true'
uses: docker/login-action@v4
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Build
id: build
uses: docker/build-push-action@v7
with:
context: ${{ matrix.image.context }}
file: ${{ matrix.image.file }}
target: ${{ matrix.image.target }}
platforms: linux/${{ matrix.arch }}
# the source repository on the package's page, which also links the package to it
labels: |
org.opencontainers.image.source=${{ github.server_url }}/${{ github.repository }}
org.opencontainers.image.version=${{ env.TAG }}
org.opencontainers.image.revision=${{ github.sha }}
org.opencontainers.image.licenses=MIT
org.opencontainers.image.title=${{ matrix.image.package }}
outputs: ${{ env.PUSH == 'true' && format('type=image,name={0},push-by-digest=true,name-canonical=true,push=true', steps.meta.outputs.image) || 'type=cacheonly' }}
cache-from: type=gha,scope=${{ matrix.image.name }}-${{ matrix.arch }}
cache-to: type=gha,mode=max,scope=${{ matrix.image.name }}-${{ matrix.arch }}

- name: Keep the digest
if: env.PUSH == 'true'
env:
DIGEST: ${{ steps.build.outputs.digest }}
run: |
mkdir -p digests
touch "digests/${DIGEST#sha256:}"

- if: env.PUSH == 'true'
uses: actions/upload-artifact@v7
with:
name: digest-${{ matrix.image.name }}-${{ matrix.arch }}
path: digests/*
retention-days: 1

publish:
name: Publish ${{ matrix.image.name }}
needs: build
if: github.event_name != 'pull_request'
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
strategy:
fail-fast: false
matrix:
image:
- { name: backend-full, package: lens-backend, tags: "{v}-full {v} latest-full latest" }
- { name: backend-lean, package: lens-backend, tags: "{v}-lean latest-lean" }
- { name: frontend, package: lens-frontend, tags: "{v} latest" }
steps:
- uses: actions/download-artifact@v8
with:
path: digests
pattern: digest-${{ matrix.image.name }}-*
merge-multiple: true

- uses: docker/setup-buildx-action@v4

- uses: docker/login-action@v4
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: One tag for both architectures
env:
PACKAGE: ${{ matrix.image.package }}
TAGS: ${{ matrix.image.tags }}
GH_TOKEN: ${{ github.token }}
run: |
image="ghcr.io/${GITHUB_REPOSITORY_OWNER,,}/$PACKAGE"
version="${TAG#v}"
# latest only for the newest release, not for a backport or an older version published again
newest=$(gh release list --repo "$GITHUB_REPOSITORY" --exclude-drafts --exclude-pre-releases --limit 100 \
--json tagName --jq '.[].tagName' | { cat; echo "$TAG"; } | sort -V | tail -1)
args=()
for t in $TAGS; do
case "$t" in latest*) [ "$newest" = "$TAG" ] || continue ;; esac
args+=(-t "$image:${t//\{v\}/$version}")
done
digests=()
for d in digests/*; do digests+=("$image@sha256:$(basename "$d")"); done
[ "${#digests[@]}" = 2 ] || { echo "::error::expected 2 architectures, got ${#digests[@]}"; exit 1; }
docker buildx imagetools create "${args[@]}" "${digests[@]}"
{
echo "### $image"
for a in "${args[@]}"; do [ "$a" = -t ] || echo "- \`$a\`"; done
} >> "$GITHUB_STEP_SUMMARY"

- name: Both architectures are there
env:
PACKAGE: ${{ matrix.image.package }}
TAGS: ${{ matrix.image.tags }}
run: |
image="ghcr.io/${GITHUB_REPOSITORY_OWNER,,}/$PACKAGE"
first="${TAGS%% *}"
ref="$image:${first//\{v\}/${TAG#v}}"
platforms=$(docker buildx imagetools inspect "$ref" --format '{{range .Manifest.Manifests}}{{.Platform.OS}}/{{.Platform.Architecture}} {{end}}')
echo "$ref: $platforms"
for p in linux/amd64 linux/arm64; do
[[ " $platforms " == *" $p "* ]] || { echo "::error::$ref has no $p image"; exit 1; }
done
14 changes: 7 additions & 7 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,12 @@ name: Release
# Before it cuts anything, the last release, holding data, is upgraded to the code being released and checked
# (upgrade.yml): when anything is lost or an upgrade step misbehaves, nothing is cut. A pushed tag isn't checked again.
#
# Publishing starts the package workflows (cloudron.yml, qnap.yml, synology.yml, launchers.yml, browser-extension.yml,
# safari-clipper.yml, browser-extension-firefox.yml, desktop.yml). A release published with the workflow's own token
# doesn't trigger `release: published` in other workflows, so this one starts them itself, for the tag (`gh workflow run
# <file> --ref vX.Y.Z -f tag=vX.Y.Z`). With a RELEASE_TOKEN secret (a fine-grained token with Contents: write on this
# repository) the release is published with it instead, `release: published` fires as usual, and nothing is started
# here. RELEASE_TOKEN is also what pushes past branch protection on main.
# Publishing starts the package workflows (images.yml, cloudron.yml, qnap.yml, synology.yml, launchers.yml,
# browser-extension.yml, safari-clipper.yml, browser-extension-firefox.yml, desktop.yml). A release published with
# the workflow's own token doesn't trigger `release: published` in other workflows, so this one starts them itself,
# for the tag (`gh workflow run <file> --ref vX.Y.Z -f tag=vX.Y.Z`). With a RELEASE_TOKEN secret (a fine-grained
# token with Contents: write on this repository) the release is published with it instead, `release: published` fires
# as usual, and nothing is started here. RELEASE_TOKEN is also what pushes past branch protection on main.
on:
workflow_dispatch:
inputs:
Expand Down Expand Up @@ -169,7 +169,7 @@ jobs:
# RELEASE_TOKEN's release fires `release: published` in them already
if: env.EXISTING != 'true' && env.HAS_RELEASE_TOKEN != 'true'
run: |
for workflow in cloudron.yml qnap.yml synology.yml launchers.yml browser-extension.yml safari-clipper.yml browser-extension-firefox.yml desktop.yml; do
for workflow in images.yml cloudron.yml qnap.yml synology.yml launchers.yml browser-extension.yml safari-clipper.yml browser-extension-firefox.yml desktop.yml; do
if [ ! -f ".github/workflows/$workflow" ]; then
echo "$workflow isn't in $TAG; skipped"
elif gh workflow run "$workflow" --ref "$TAG" -f tag="$TAG"; then
Expand Down
4 changes: 3 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,9 @@ in PowerShell; or skip the terminal and double-click the launcher for your syste
[Lens.app](docs/get-started.md#the-mac-app) from the same release installs Lens, runs it without Docker and keeps it in
the menu bar (built, early: not signed yet).
[Deployment](docs/deployment.md) covers servers and Cloudron; there are also packages for
[Synology](packaging/synology/README.md), [QNAP](packaging/qnap/README.md) and [Proxmox](proxmox/README.md).
[Synology](packaging/synology/README.md), [QNAP](packaging/qnap/README.md) and [Proxmox](proxmox/README.md), and each
release publishes Lens's images for amd64 and arm64 (a Raspberry Pi) to GHCR ([prebuilt
images](docs/deployment.md#prebuilt-images)).
The [checks](docs/requirements.md#install) say which of these CI proves.

To work on the code instead, `make run` builds and runs everything from a checkout, `make setup-code` prints the
Expand Down
17 changes: 17 additions & 0 deletions docs/deployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,23 @@ docker compose -f docker-compose.prod.yml up -d --build
docker compose -f docker-compose.prod.yml logs backend | grep "setup code"
```

## Prebuilt images

Each release publishes Lens's own images to GHCR, built for amd64 and arm64 (a Raspberry Pi, a Mac with Apple silicon,
an arm64 server), so a machine can download them instead of building
([images.yml](https://github.com/adeelahmad/lense/blob/main/.github/workflows/images.yml)):

| Image | Tags | What it is |
|---|---|---|
| `ghcr.io/adeelahmad/lens-backend` | `X.Y.Z-full` (also `X.Y.Z`, `latest-full`, `latest`) | the API and workers, full edition: reads Office files, web pages and emails too |
| `ghcr.io/adeelahmad/lens-backend` | `X.Y.Z-lean` (also `latest-lean`) | the same without LibreOffice and Chromium |
| `ghcr.io/adeelahmad/lens-frontend` | `X.Y.Z` (also `latest`) | the web app |

`latest` follows the newest release. Pin a version (`X.Y.Z`) in anything you run, and run that release's
`docker-compose.prod.yml` with it, since a version's images and its compose file go together. The images carry no
optional extras (`EXTRAS`): to bake one in, build from source as above. The images are built in CI from the release's
tag with the repository's own token; nothing else signs or hosts them.

## Checklist

* **HTTPS in front of the web app**: [Remote access](remote-access.md) (a Cloudflare Tunnel Lens runs itself, set up in
Expand Down
1 change: 1 addition & 0 deletions docs/requirements.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ request, to another check that proves the same thing; loosening a requirement or
| Double-click launchers for each system | [launchers.yml](https://github.com/adeelahmad/lense/blob/main/.github/workflows/launchers.yml) `Build the launchers`, `Check them` |
| Lens.app on a Mac with Apple silicon installs Lens and runs it without Docker | [desktop.yml](https://github.com/adeelahmad/lense/blob/main/.github/workflows/desktop.yml) `Build Lens.app`, `The bundled msb runs`, `The app carries its helper and microsandbox` |
| Packages for Synology, QNAP and Cloudron | [synology.yml](https://github.com/adeelahmad/lense/blob/main/.github/workflows/synology.yml) `Build`; [qnap.yml](https://github.com/adeelahmad/lense/blob/main/.github/workflows/qnap.yml) `Build the package`; [cloudron.yml](https://github.com/adeelahmad/lense/blob/main/.github/workflows/cloudron.yml) `build` |
| Each release publishes Lens's images for amd64 and arm64 to GHCR | [images.yml](https://github.com/adeelahmad/lense/blob/main/.github/workflows/images.yml) `Build`, `One tag for both architectures`, `Both architectures are there` |
| A Proxmox container script | No check yet ([#486](https://github.com/adeelahmad/lense/issues/486)) |
| The first admin signs up with a passkey from the setup link, which fills in the setup code | [journeys.mjs](https://github.com/adeelahmad/lense/blob/main/ci/clicks/journeys.mjs) `first-admin`; [baseline.json](https://github.com/adeelahmad/lense/blob/main/ci/clicks/baseline.json) `first-admin` ≤ 1 click; [test_passkeys.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_passkeys.py) `test_first_admin_with_a_passkey_and_no_password`; [setupForm.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/setupForm.test.tsx) `fills in the code from the setup link and starts at the email` |

Expand Down
Loading