Skip to content

smoke tests: 27 packages (a–l, batch 1) - #1366

Merged
bryan-minimal merged 1 commit into
mainfrom
smoke/bin-a-l-1
Nov 7, 2026
Merged

bryan-minimal merged 1 commit into
mainfrom
smoke/bin-a-l-1

Conversation

@bryan-minimal

Copy link
Copy Markdown
Member

Adds a Standalone smoke test to 27 packages that ship an OutputBin and had no Standalone test (a–l list, batch 1 of 3).

package test
alex /usr/bin/alex --version
alsa-lib bash -c '/usr/bin/aserver -h; rc=$?; [ $rc -lt 126 ]' (test_deps: bash-bootstrap)
ast-grep /usr/bin/ast-grep --version
attr /usr/bin/getfattr --version
atuin /usr/bin/atuin --version
autoconf /usr/bin/autoconf --version
automake /usr/bin/automake --version
bandwhich /usr/bin/bandwhich --version
bash-language-server /usr/bin/bash-language-server --version
bat /usr/bin/bat --version
binutils /usr/bin/ld --version
binutils-arm-none-eabi /usr/bin/arm-none-eabi-ld --version
broot /usr/bin/broot --version
btop /usr/bin/btop --version
bzip3 /usr/bin/bzip3 --version
cabal /usr/bin/cabal --version
caddy /usr/bin/caddy version
check /usr/bin/checkmk --version
claude-code /usr/bin/claude --version
cloud-sql-proxy /usr/bin/cloud-sql-proxy --version
cmake /usr/bin/cmake --version
cosign /usr/bin/cosign version
crane /usr/bin/crane version
cython /usr/bin/cython --version
delta /usr/bin/delta --version
difftastic /usr/bin/difft --version
duf /usr/bin/duf --version

How these were verified

Locally, in a fresh min session activate . --loadout pkgbuild on this tree (a new session name for every run): min package build <pkgs> (cached artifacts, no rebuild; tests are not in the spec hash), then min check --packages <pkgs> with the names space-separated. All 81 tests across the three a–l batches report standalone tests...Pass, and no other check fails. pkgs CI shows standalone tests...Skip for these because nothing rebuilds, so the local run is the real evidence.

Negative control: in a fresh session with all the tests applied, hyperfine's test was swapped for /usr/bin/hyperfine --version; exit 7. min check reported smoketest: ... had exit code 7 / standalone tests...Fail. So the tests do run and a failure is reported. The real failures below (exit 127, loader errors) also came through as Fail.

Wrapper-style tests: where a command needs a shell (a sh script like freetype-config, or a tool whose help flag exits non-zero by design, wrapped as cmd; rc=$?; [ $rc -lt 126 ] to prove only that exec and the loader work), the test uses test_deps = [bash-bootstrap], not base. bash-bootstrap's only runtime_dep is glibc. base drags in zlib, zstd, libmagic, libstdc++ and others, and that hid real missing runtime_deps: libgd and diffoscope both passed with base and failed without it.

Skipped (no test added)

package reason
bash-bootstrap, gawk-bootstrap, ghc-bootstrap, llvm-bootstrap bootstrap packages
binutils-2.46-glibc, gcc-15.2.0-glibc, glibc-bedrock-2.42, go-1.4, jikes-1.22 bootstrap rungs (} target multi-arch shape)
hex0 stage0
curl, file, htop, ghostscript, libidn2, libgd, diffoscope, dejagnu, llama.cpp the smoke test fails because of a real packaging bug (findings below). The test should land with the fix.

Findings (separate PRs, since runtime_deps are hashed and trigger a rebuild)

The missing libraries were found by an in-test ldd run under test_deps = [bash-bootstrap], with the result read back through exit codes, because min check drops test output.

package symptom missing
curl curl --version exit 127 libzstd.so.1 (zstd), same as batch 1 (#820)
file file --version exit 127 libzstd.so.1, libz.so.1 (zstd, zlib), same as batch 1
libidn2 idn2 --version exit 127 libunistring.so.5 (libunistring), same as batch 1
htop htop --version exit 127 libcap.so.2 (libcap)
ghostscript gs --version exit 127 libbz2.so.1.0 (bzip2)
libgd gdtopng exit ≥126 without base libstdc++.so.6, libgcc_s.so.1 (subsetOf gcc ["libgcc","libstdcpp"])
diffoscope --version exit 1 without base: ImportError: failed to find libmagic file (libmagic)
dejagnu runtest --version exit 1: ERROR: unable to find expect expect
llama.cpp llama-cli --version exit 127 its OWN libs: libmtmd.so.0, libllama.so.0, libllama-common.so.0, libggml*.so.0. build.sh runs install -vm755 build/bin/*.so, which ships only the unversioned names, so the SONAME files are never installed. llama-cli and llama-server cannot start.

Tests ran on the Mac's VM (arm64). amd64 is not exercised locally.

🤖 Generated with Claude Code

Add a Standalone smoke test (--version or equivalent) to 27 packages that
ship an OutputBin and had none. Tests are not in the spec hash, so nothing
rebuilds. Verified locally with min check in a fresh session.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bryan-minimal
bryan-minimal requested a review from a team as a code owner November 6, 2026 21:09
@coderabbitai

coderabbitai Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

  • Run on-demand review

This review includes 27 billable files and costs up to $6.75.

  • Ask an admin to make reviews automatic

Open in CodeRabbit

Reviews can continue after your included limit without a manual trigger. An admin must approve usage-based billing.

Or wait 58 minutes for your next included review.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available. Your 65 included PR review attempts over the past 7 days set your current allowance at 1 review per hour.

Learn how review limits work.

Review configuration:

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 7a198f0 and 61f8316.

Files selected for processing: 27

  • packages/alex/build.ncl
  • packages/alsa-lib/build.ncl
  • packages/ast-grep/build.ncl
  • packages/attr/build.ncl
  • packages/atuin/build.ncl
  • packages/autoconf/build.ncl
  • packages/automake/build.ncl
  • packages/bandwhich/build.ncl
  • packages/bash-language-server/build.ncl
  • packages/bat/build.ncl
  • packages/binutils-arm-none-eabi/build.ncl
  • packages/binutils/build.ncl
  • packages/broot/build.ncl
  • packages/btop/build.ncl
  • packages/bzip3/build.ncl
  • packages/cabal/build.ncl
  • packages/caddy/build.ncl
  • packages/check/build.ncl
  • packages/claude-code/build.ncl
  • packages/cloud-sql-proxy/build.ncl
  • packages/cmake/build.ncl
  • packages/cosign/build.ncl
  • packages/crane/build.ncl
  • packages/cython/build.ncl
  • packages/delta/build.ncl
  • packages/difftastic/build.ncl
  • packages/duf/build.ncl
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

@norrietaylor norrietaylor left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Scope matches the intent. Each of the 27 files adds one smoketest and the import that it needs. Nothing else changes. The title says a–l, but this batch only covers a–d. The PR body says so ("batch 1 of 3"). The Skipped and Findings tables in the body describe all three batches, not only this diff.

Clobber none. The diff touches only the import line and adds a tests block at the end of each spec. git log on the touched files shows only routine version bumps before this commit.

Correctness 26 tests use standaloneTest "<bin> <flag>". This is the same shape as the 115 existing uses, for example packages/redis/build.ncl:64 and packages/netcat/build.ncl:63. In attr and binutils, the block goes on the outer spec, not on the replace_on_cycle prebuilt. That is correct. The alsa-lib wrapper follows the rc -lt 126 precedent in packages/edgedelta/build.ncl:68. It uses bash-bootstrap, which ships usr/bin/bash. The script-based tools take their interpreter from their runtime closure: autoconf/automake from perl, cython from python, checkmk from bash+gawk, and bash-language-server from node.

Not checked These tests do not change the spec hash, so CI reports standalone tests...Skip. The only evidence that they pass is the author's local arm64 run. I did not run them, and I did not check amd64. I did not check that aserver -h exits non-zero, so I cannot say if the wrapper is necessary.

@@ -1,4 +1,5 @@
let { Attrs, BuildSpec, Local, OutputBin, OutputData, OutputLib, Source, .. } = import "minimal.ncl" in
let { Attrs, BuildSpec, Local, OutputBin, OutputData, OutputLib, Source, Test, .. } = import "minimal.ncl" in
let bash-bootstrap = import "../bash-bootstrap/build.ncl" in

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit · The new bash-bootstrap import is between the minimal.ncl line and the config.ncl line. In the other files with this pattern (attr, binutils), config.ncl comes second and the package imports come after it. Move this import below let { target, .. } = import "config.ncl" in.

@bryan-minimal
bryan-minimal added this pull request to the merge queue Nov 7, 2026
Merged via the queue into main with commit 98c9e87 Nov 7, 2026
15 checks passed
@bryan-minimal
bryan-minimal deleted the smoke/bin-a-l-1 branch November 7, 2026 23:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants