Repository navigation
postgres: build contrib extensions (pgcrypto, pg_trgm, ...) and enable lz4 - #1203
Conversation
…e lz4 Build world-bin so the server ships the contrib modules from its own release tarball, compiled in the same configured tree, so they can never drift from the server version. With the current configure flags that is 52 modules (pgcrypto, pg_trgm, hstore, citext, pg_stat_statements, postgres_fdw, ...). In a trial build they added ~4.7 MB to a ~35 MB package and no runtime deps: pgcrypto links libcrypto and libz, which postgres already depends on. An extension does nothing until someone runs CREATE EXTENSION. Configure --with-lz4 now that lz4 is packaged, resolving the TODO in build.sh: LZ4 for TOAST, WAL and backup compression. pglz stays the default, so existing behaviour does not change. contrib skips pgcrypto and sslinfo silently when configure finds no OpenSSL, so build.sh now fails if either is missing. New standalone tests each start a throwaway server: every available extension can be created; pg_trgm similarity and a trigram GIN index serving LIKE; pgcrypto digest, random bytes, bcrypt, PGP and AES; raw Blowfish through the OpenSSL legacy provider opt-in; and lz4 column and WAL compression. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Warning Review limit reached
This review includes 2 billable files and costs up to $0.50.
Reviews can continue after your included limit without a manual trigger. An admin must approve usage-based billing. Or wait 47 minutes for your next included review. View limit details
No actionable comments were generated in the recent review.
|
| Layer / File(s) | Summary |
|---|---|
Build configuration and installation packages/postgres/build.sh, packages/postgres/build.ncl |
The build enables LZ4, uses world-bin build and install targets, adds LZ4 as a runtime dependency, and checks for installed pgcrypto.so and sslinfo.so. |
Server test harness and coverage packages/postgres/build.ncl |
A shared helper initializes, starts, and cleans up a temporary PostgreSQL server. Tests cover available extensions, pg_trgm similarity and GIN index use, pgcrypto operations including legacy-provider encryption, and LZ4 column and WAL compression. |
Priority:
Estimated code review effort: 3 (Moderate) | ~20 minutes
Change: Feature
Possibly related issues
gominimal/pkgs#1149: This issue calls for behavioral standalone package tests, and this change adds standalone PostgreSQL server tests.
Possibly related PRs
- gominimal/pkgs#1140: The legacy-provider test uses OpenSSL configuration and provider modules, which are affected by this PR’s OpenSSL package changes.
Suggested reviewers: bryan-minimal
Merge Risk: 
Minimal · previous review e75fe
-
No concrete merge-blocking risk was identified in the build or server tests.
Pre-merge checks: 
5
-
Full details: Docstring Coverage
-
Explanation
No eligible functions found in the changed files. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 0 functions across 1 files. (1 skipped: 1 unsupported.)
-
Finishing Touches
-
📝 Generate docstrings
-
- Commit to this branch
- Create a new PR
🧪 Generate unit tests (beta)
-
- Commit to this branch
- Create a new PR
-
- Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Comment @coderabbitai help to get the list of available commands.
norrietaylor
left a comment
There was a problem hiding this comment.
The change looks correct and bundling contrib into postgres is the right call. Two things need doing before merge.
Blocking
- x86_64 hasn't been built or tested. The PR notes say x86_64 is covered by CI, but PR CI only runs static checks. In the
minimal-checklog for postgres,standalone tests,output types valid,missing runtime_depsandenumerate binsall show Skip. So the five new tests, theworld-binbuild and the--with-lz4link have run only on one aarch64 machine. Please do an x86_64 patched build and runmin checkon it. - Reproducibility needs re-checking on the final change. The double build was done with
make -C contrib, before the switch toworld-binand--with-lz4. Please do two--rebuildbuilds of the head commit and diff the two$OUTPUT_DIRtrees.
Suggestions (non-blocking)
build.sh: a baretest -efails the build without saying why. Something like|| { echo "pgcrypto.so missing: configure did not find OpenSSL" >&2; exit 1; }would make the failure clear.runtime_deps:lz4brings in the whole package (CLI tools, man pages, headers) when postgres only needsliblz4. It's consistent with howzstdandzlibare done today, so fine for this PR. Moving tosubsetOflater would also mean addinglz4tobuild_depsso configure still finds the headers.pgcrypto_legacy_provider: this mostly tests the openssl package, i.e. thatlegacy.sois where OpenSSL's MODULESDIR expects it. A comment saying so would help whoever sees it fail after an openssl layout change.contrib_extensions_create: a short comment on why the threshold is> 40(contrib currently gives about 49) would help.
Checked and fine
- No accidental
%{interpolation in the Nickel multiline strings. - The bcrypt
$hsubstitution intoqis safe. - A failure inside
for e in $(q …)is still caught by the count check that follows. - The TOAST value is large enough that the
pg_column_compression = lz4check really tests compression. wal_compression = lz4is rejected if the build lacks lz4, so that check is meaningful.- The existing output globs cover everything contrib installs.
|
Reproducibility check on arm64 (aarch64): reproducible. I built this PR's
The hashing step was appended only on a scratch branch and isn't part of this PR. This was done in-sandbox with |
Summary
Make PostgreSQL's bundled extensions, the contrib modules, available by building them into the
postgrespackage instead of adding separate per-extension packages. Starts from the request for pgcrypto and pg_trgm. Also enables--with-lz4, which resolves the TODO inbuild.shnow thatlz4is packaged.Why bundle rather than split into packages like
postgres-pgcrypto:pgcrypto.soNEEDs onlylibcrypto.so.4andlibz.so.1, andpostgresalready depends on both.CREATE EXTENSION.world/install-world), Debian since PG 10, Arch, Homebrew and the official Docker image all ship contrib with the server.Extensions from outside the PostgreSQL tree (pgvector, PostGIS, ...) should still be separate packages that depend on
postgresat runtime. That's out of scope here.Changes
build.sh:make world-bin/make install-world-bin. That's the server plus contrib without the docs, PostgreSQL's documented target for this.build.sh: fail the build ifpgcrypto.soorsslinfo.sois missing. Contrib skips both silently when configure doesn't find OpenSSL.build.sh/build.ncl:--with-lz4, withlz4inruntime_depsalongsidezstd. LZ4 becomes available for TOAST, WAL and backup compression.pglzstays the default, so existing behaviour doesn't change.build.ncl: new standalone tests. Each starts its own throwaway server through a smallserver_testhelper and does one job:contrib_extensions_create: every available extension can be created (CASCADE), which loads each library.pg_trgm:similarity(), and the planner uses a trigram GIN index forLIKE '%abc%'.pgcrypto:digest(known SHA-256 vector),gen_random_bytes,crypt/gen_salt('bf'),pgp_sym_*, raw AESencrypt/decrypt.pgcrypto_legacy_provider: raw Blowfish works whenOPENSSL_CONFactivates the OpenSSL legacy provider.lz4_compression: aCOMPRESSION lz4column is stored as lz4, andwal_compression = lz4is accepted.Outputs are unchanged: the existing globs already cover everything contrib installs. The four
contrib/spi*.exampledocs underusr/share/docaren't captured, same as before, since nothing captures docs.Checklist
min checkpasses for the affected packages/harnesses.min patched-build <name>succeeds for any package I added or modified.Notes for reviewers
min checkwith all six standalone tests passing, the existinginitdb_c_utf8included. To confirm the new tests aren't vacuous, I flipped one assertion and checked thatmin checkfailed and namedlz4_compression. x86_64 is covered by CI only.build.shtwice, with an uncommitted probe appended that hashes$OUTPUT_DIR. A marker that differed between the two runs forced two real builds instead of a cache hit; both did the full compile (1,566gccinvocations each) and configure found LZ4. The two output trees were identical: all 2,028 entries (1,924 files, 9 symlinks, 95 directories) had the same type, mode, symlink target and sha256. I couldn't run the host-sidemip --no-cache --rebuildfrom the sandbox, and I haven't run this check on x86_64.encrypt()/decrypt()withbf,desorcast5fail by default, because the shippedopenssl.cnfdoesn't activate the legacy provider. That's the same on every OpenSSL 3+/4 distro, and upstream documents it in pgcrypto's notes.crypt()with bf/md5/xdes uses pgcrypto's built-in code and works either way. The new test covers the opt-in path rather than enabling legacy ciphers globally.uuid-ossp(needs--with-uuid; mostly replaced by the built-ingen_random_uuid()/uuidv4()/uuidv7()),xml2,sepgsql, and the plperl/plpython transforms, which would make perl/python runtime deps of postgres. Left for a follow-up if anyone asks.libsoutput. The output glob isusr/lib/**, so libpq and the server modules live in one output, and contrib adds 49.sofiles to it. Nothing depends onpostgrestoday. If a libpq-only consumer appears, that output is worth splitting.min checkreports which test failed but not its output, which is why the helper keeps one concern per test.🤖 Generated with Claude Code