Skip to content

gcc-arm-none-eabi: build common embedded archs instead of everything (~6x faster) - #20

Merged
twitchyliquid64 merged 1 commit into
mainfrom
tom/gcc-arm-none-eabi
Mar 27, 2026
Merged

twitchyliquid64 merged 1 commit into
mainfrom
tom/gcc-arm-none-eabi

Conversation

@twitchyliquid64

Copy link
Copy Markdown
Contributor

So the gcc-arm-none-eabi takes like an hour and a half, which mostly is fine as it runs concurrently, but the reason its doing that is because its building newlib / libstdc++ for like 70+ different architectures. This is a bit much for now.

This change builds only for 6-7 common embedded architectures that this package plans to support. Adding more is trivial. This should speed up the build and the amount of work that needs to get done.

@twitchyliquid64

Copy link
Copy Markdown
Contributor Author

Builds fine on buildbot (25mins on an 8 core machine)

@msample msample left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@twitchyliquid64
twitchyliquid64 merged commit 02b74e0 into main Mar 27, 2026
1 check passed
@twitchyliquid64
twitchyliquid64 deleted the tom/gcc-arm-none-eabi branch March 27, 2026 22:24
motdotla pushed a commit to motdotla/pkgs that referenced this pull request Aug 27, 2026
* ci: add stable runtime-closure check (inbox#21)

A report-only helper + workflow: given a package set (or the PR diff), compute
the transitive runtime closure and report whether branch `stable` already
contains it (missing = closure - stable - co-promoted), de-risking the "stable
must stay dependency-closed" promotion invariant.

Replicates the engine's needed_for_internet injection (transitives.rs pulls
ca-certificates into the runtime closure of any needs.internet package) so the
BFS doesn't under-count and false-PASS (e.g. promoting gh/curl before
ca-certificates). Runtime-only, name-presence-only -- both honest caveats for gominimal#20.

Additive, contents:read, continue-on-error + --no-fail; keep off required checks
until gominimal#20 turns it into a gate (no code change needed to flip it).

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* stable-closure-check: exit 0 on no-seeds in --changed/--no-fail mode

A report-only run that touches no packages (the workflow's --changed mode on a
PR with no packages/* changes, e.g. this PR's own CI) found no seeds and exited
2, which under set -o pipefail turned the closure job red. Report-only checks
must not fail on no-op PRs. Treat no-seeds as "nothing to check" (exit 0) in
--changed or --no-fail mode; keep exit 2 only for an explicit bare invocation.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* stable-closure: close two fail-open false-PASS holes + fix --no-fail contract

An adversarial verification against the engine's runtime-closure semantics
(transitives.rs) confirmed the good news first: the gated needed_for_internet /
ca-certificates injection is faithfully replicated, so curl/gh/go correctly
require ca-certificates -- no false-PASS from the closure math itself.

But the guard could still go green without verifying anything, via two
fail-OPEN input paths:

- An unknown / namespace-mismatched seed only warned, then its closure =={itself}
  and was subtracted away -> empty missing -> exit 0 PASS. A typo or a natural
  name (python3, nodejs) silently verified nothing. Now: verdict UNVERIFIED,
  exit 2 (blocking); --no-fail still exits 0 but prints UNVERIFIED, never PASS.
- An empty-but-valid dump ([]) made every closure == its seeds -> vacuous PASS.
  Now guarded in build_runtime_map (covers the --stable-worktree path too) ->
  exit 2.

Also:
- --no-fail now genuinely never blocks: operational failures (missing minimal
  binary, unfetched origin/stable, dump error) route through a no-fail-aware
  fail() that returns 0 with an explicit UNVERIFIED verdict instead of exiting
  2 before the guard. Exit 1 stays reserved for a computed non-empty `missing`.
- git calls now honor --repo (cwd=args.repo), not just `minimal dump`.
- A missing stable ref is detected up front (git rev-parse) with the actionable
  fetch hint, instead of a raw git error.

Verified against the real catalog: ca-certificates injection is load-bearing
(gh's closure includes it only with the injection); PASS / NOT CLOSED /
UNVERIFIED / empty-dump exit codes are 0 / 1 / 2 / 2.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
motdotla pushed a commit to motdotla/pkgs that referenced this pull request Aug 27, 2026
…mal#279)

* ci: add version-age report-only check (inbox#21 / gominimal#20)

A non-blocking pull_request check that, for each changed packages/*/build.ncl,
resolves the upstream release date (attrs.released_at override -> GitHub
releases/tags -> ftp.gnu.org Last-Modified -> unknown), computes age, and posts
a job-summary table flagging anything younger than a 7-day reference (the gominimal#20
minimum-soak criterion).

Report-only and additive: contents:read, no secrets, never fails the job, and
MUST be kept off branch-protection required checks. Its high-value output is the
live census of which packages have no derivable date and need attrs.released_at.

The attrs.released_at schema (minimal) + example + minimum_version bump are a
deliberate, deferred follow-up gated on sizing the backfill (see gominimal#20); the
script already handles released_at's absence gracefully, so this lands standalone.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* version-age: add generic source-URL Last-Modified date tier

The ~82 no-source_provenance packages (toolchains, X11 libs, pinned binaries)
still have an http(s) source tarball -- so generalize the GNU-specific
Last-Modified HEAD to any source URL ("date on the file"). New tier 4 (after
GitHub/GNU, before UNKNOWN), and a fall-through fallback for GitHub/GNU packages
whose tag/tarball didn't resolve. (Repology was evaluated and dropped -- its API
exposes versions but no release dates.)

Last-Modified is availability/upload time, not strictly upstream release; close
enough for a soak gate, and dwell (git time) backstops it. http(s) only; the URL
is taken whole from the spec (no SSRF-via-version), with a code comment flagging
the fork-PR HEAD surface for the reviewer.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* version-age: fix two census-found bugs in date resolution

Running the resolver over the full catalog (373 pkgs) surfaced two real
bugs that would have errored or under-resolved the live check:

1. github_date crashed on tags-only repos: `git/refs/tags/{t}` (plural)
   returns a LIST of prefix-matching refs, and the code called `.get` on
   it -> AttributeError. Switch to singular `git/ref/tags/{t}` (one
   object) and guard against a list anyway, accepting only an exact match.

2. source_url read the wrong key: source deps carry their fetch spec
   under `from` ({type, url, sha256}), not a flat `url`, so the new
   source-URL tier matched nothing. Read `from.url` (tolerate flat shape).

Census effect: dated 229 -> 263 / 373 (the source-URL fix reclaimed 34
http(s)-sourced packages); 0 runtime errors across the catalog.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* version-age: empirical tag/path matcher + resolver hardening

Probed all 20 dateless-but-has-provenance packages against their real
upstream (GitHub/GitLab tags APIs, GNU ftp listings) and taught the
resolver the formats found, instead of guessing:

- candidate_tags: keep the historical 8 forms (no regression) + add
  repo-derived families (bun-v{v}, varlock@{v}, lib-stripped fuse-/
  xkbcommon-) and a per-repo override map for bespoke schemes
  (R_2_7_5, REL_18_4, core-8-6-16, V_10_3_P1, llvmorg-, version-,
  rust-v, gopls/v, cabal-install-v, lcms).
- gnu_date: try flat + nested per-version dir (gcc lives in
  /gnu/gcc/gcc-X/) + aliased project dir (libidn2 under /gnu/libidn/).
- new self-dated tier: versions carrying a trailing YYYYMMDD (ncurses
  weekly snapshots, other Dickey projects) self-date with no network.

Census effect (full catalog, live): dated 263 -> 279 / 373; the 20
fixable resolver-misses drop to 3 (libsvtav1=GitLab, ngspice=SourceForge,
linux_headers=stable-tree-in-gregkh/linux) which need new host tiers.

Hardening (from an adversarial review the census motivated):
- _curl_head_date shared helper: https-only with --proto/--proto-redir
  =https + bounded redirects (close the fork-PR SSRF/downgrade surface),
  and validate the FINAL hop is 2xx so an error page's stale
  Last-Modified can't yield a bogus "release date" (this removed
  libx265's 403-page date). Naive datetimes treated as UTC.
- gnu_date now follows redirects (was curl -sI, missed 302s).
- source_ext reads nested from.url (same bug class as the prior two)
  and strips ?query before the ext match; add zst.
- gh_json: add a 30s timeout (was unbounded).
- md_cell: escape |/newlines/backticks in the summary table + unknown
  list (untrusted build.ncl -> GITHUB_STEP_SUMMARY injection).
- by_name skips non-string names (unhashable -> crash guard).

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants