For records with no existing file, walk homebrew-core git history
(newest-first) via FormulaVersions and use the pkg_version at the oldest
revision where the CVE still appears in resolved_ids as the `fixed`
boundary, instead of the current pkg_version. Existing records preserve
their on-disk ranges (per the merge behaviour in the previous commit),
so this only fires once per newly-annotated (formula, CVE) pair.
The FormulaVersions instance and its rev_list output are cached per
formula so subsequent CVEs for the same formula reuse the ~90s
path-filtered git rev-list and per-revision formula loads. OsvExport.run
now runs inside the SimulateSystem block so historical loads are
runner-OS-independent. Historical revisions are evaluated under the base
simulation only; a resolves inside on_linux/on_intel falls through to
the current pkg_version (documented; no such annotation exists in core
today).
Spot-checked against real homebrew-core: unzip CVE-2014-8139 -> 6.0_6
(current is 6.0_8), CVE-2022-0529 -> 6.0_8; libquicktime CVE-2016-2399
-> 1.2.4_5 (the resolves-added-later case; hand-correct to _4).
Currently tests fail on all runners (Tier 1 Linux and Tier 3 macOS). It
is difficult to align the setups used between Homebrew/brew and
Homebrew/core so can limit to basic sanity of the cask and formula
tests.
When migrating a record written before `published` existed, fall back to
its `modified` timestamp rather than today's so first-rewrite doesn't
jump the publication date forward.
Narrow the upstream cache value with `is_a?(Hash)` at the call site
instead of `T.cast`, and collapse the failed-fetch guard to a single
`next if` now that reassignment isn't needed.
Records now carry `published` and `database_specific: {source:
"generated"}`. `.run` reads any existing `<id>.json`, preserves its
`published` timestamp and `affected[].ranges` (so the `fixed` boundary
reflects when the annotation was first observed rather than drifting to
the current pkg_version), and only writes when the remaining content has
changed. Records for annotations no longer in core are simply not
visited, so they persist on the default branch.
A transient OSV.dev failure leaves an existing enriched record untouched
rather than stripping its summary/severity/references. The unchanged
comparison uses parsed hashes so key ordering in a hand-formatted
existing file does not cause spurious rewrites.
Add a pointer to ossf.github.io/osv-schema above SCHEMA_VERSION and note
where the Homebrew ecosystem/prefix were registered. patch_ref now
returns T::Hash[Symbol, T.any(String, T::Array[String])] via a PatchRef
alias; the input side stays T.untyped to match Formula#serialized_patches.
- Bootsnap compiles Ruby files into instruction sequence and YAML
caches on first load, so an update that changes files leaves the
next command to recompile them at startup, on the user's time.
- The cost is largest for updates under `Library/Homebrew/vendor`:
portable Ruby bumps rotate the whole cache key (it hashes the
Ruby version and gem directory listing), cold-starting every
file at once, and vendored gem bumps rewrite entire gem trees,
much of which `update-report`'s own run never loads and so
cannot recompile as a side effect.
- Loading the `brew install` and `brew fetch` command graph with a
fully cold cache took 0.91s vs 0.56s warm on an M-series Mac in
this checkout, so ~350ms of pure cache compilation; the gap is a
multiple of that on older machines.
- After an explicit `brew update` that changed
`Library/Homebrew/vendor`, `update-report` now spawns a detached
background Ruby that requires `global`, `cmd/install`,
`cmd/fetch` and `cmd/upgrade`, recompiling the caches for the
most common next commands while the user reads the update
report.
- Code-only updates skip the prewarm: `update-report` has already
recompiled most of what the next command loads (the residual
was ~20-40ms here), so a background process is not worth it.
- Auto-update runs also skip it: the command that triggered them
loads the same files immediately afterwards, so a parallel
prewarm would only duplicate its work.
- `Homebrew::Bootsnap.prewarm!` reuses the existing `enabled?`
gate, is skipped under `HOMEBREW_TESTS` and spawns the
interpreter from `HOMEBREW_RUBY_EXEC_ARGS` with the parent's
`$LOAD_PATH`, stdio on `/dev/null` and its own process group,
then detaches, so the update exits immediately and a prewarm
failure cannot break it.
- Verified: across a vendored gem bump a detached child appears
once `update-report` exits and compiles whatever its run did
not load (736 -> 1055 cache files after a minimal report run),
while code-only updates, auto-update runs and
`HOMEBREW_NO_BOOTSNAP=1` spawn no child.
When multiple formulae resolve the same CVE, fetch it from OSV.dev once
per run rather than once per (formula, id) pair. Uses Hash#fetch with a
block so a nil (fetch-failed) result is also cached.
- The parallel fetch loop paused a fixed 50ms per iteration, even
when every download had already finished: `Kernel#sleep` was 27%
of wall samples (~100ms) in `brew prof --stackprof fetch` for an
already-cached formula, the largest single frame ahead of
`Kernel#require`.
- Warm-cache downloads resolve in microseconds, so almost all of
that sleeping was wasted on every `brew fetch`, `brew install`
and `brew upgrade`, interactive or scripted, and even the final
iteration slept after the last download had finished.
- Warm-cache `brew fetch` of a cached formula drops from
0.63-0.70s to 0.45-0.52s (~200ms, ~25-30%), measured both on a
TTY and piped, and `Kernel#sleep` disappears from its profile
entirely.
- Completed downloads are reported as they finish instead of on
the next 50ms poll tick.
- Skip the end-of-iteration pause entirely when the partition left
no downloads pending; previously even the final iteration slept.
- Wait on a `Concurrent::Event` signalled by an `on_resolution!`
callback on every download future instead of sleeping, so warm
caches never pay a poll interval.
- On animated TTYs the wait times out after 50ms, purely to keep
redrawing the spinner and progress bars at a steady cadence
while downloads are in flight; otherwise after 1s as a safety
net.
- The event is reset before each wait and the remaining futures
are re-checked in between, so a download resolving between the
partition and the reset cannot cause a lost wakeup.
Replaces the tap's brew-vulns --osv-export DIR with a hidden dev-cmd
following the generate-formula-api pattern. Iterates homebrew/core under
with_no_api_env + SimulateSystem, unions each formula's serialized_patches
across all OS/arch variations from to_hash_with_variations so
platform-gated resolves annotations are not silently dropped, and hands
[formula, patches] pairs to Homebrew::Vulns::OsvExport.
--dry-run lists record ids without writing or querying OSV.dev.
Consumed by Homebrew/homebrew-advisory-database's regenerate workflow.
Ports the OSV record exporter from Homebrew/homebrew-brew-vulns. Emits
one BREW-<formula>-<CVE> record per security id declared in a formula's
patch resolves list, under the registered Homebrew ecosystem with
pkg:brew purls, marking the currently shipped pkg_version as the fixed
boundary. Upstream summary/severity/references are copied from OSV.dev
where available.
This is the generator for Homebrew/homebrew-advisory-database; the
dev-cmd entry point follows in the next commit.
Compute stable_repo_url and stable_tag once at the top of build_target
and reuse them in both the installed and non-installed branches, rather
than repeating the same target_repo_url/tag chain twice.
- Start the requested bottle after `prelude` checks but before
recursive dependency enqueueing so network I/O overlaps Ruby work.
- Reuse the cached-location future in `DownloadQueue` and defer
staging until dependency enqueueing completes, avoiding duplicate
transfers and early extraction.
- Preserve direct fetch ordering and existing attestation, source, local
bottle and only-dependencies paths.
- This favours cold parallel downloads; warm or serial downloads gain
little and later dependency failures can waste bandwidth.
Previously --brewfile silently replaced any named arguments; combine
them instead so `brew deps foo --brewfile` includes foo alongside the
Brewfile entries, and dedupe the merged list.
Move the verbatim source/head URL fallback into a Scanner.target_repo_url
class method so build_target's three call sites share one implementation
written in the ||= style suggested in review, and add a spec for the
installed-keg SBOM branch where the SBOM downloadLocation host is not a
supported forge.
Read brew and cask entries from a Brewfile and treat them as if they
had been passed as named arguments, so --brewfile composes with --tree,
--graph, --for-each, --union and the default flat listing. --formula and
--cask filter which entry types are loaded. A bare --brewfile reads
./Brewfile and --brewfile=<path> reads a specific file, matching
brew vulns.
repo_url only normalises GitHub, GitLab and Codeberg, so formulae whose
stable, head and homepage URLs are all hosted elsewhere (bash, gmp,
gnu-tar, go, make, openconnect, zsh) were skipped outright. Restore the
tap's last-resort fallback: query OSV with the source URL verbatim when
its path yielded a tag, otherwise the head URL verbatim. OSV's GIT
ecosystem returns nothing for these hosts today, so this is parity with
the tap and future-proofing rather than a detection change.
Follow-up to #23080.
- Use the released `ruby-macho` gem now that the code signing API
is available in `6.0.0`.
- Vendor the released runtime files so `brew` can load `macho`
without relying on the PR git checkout.
- Keep `MachO.codesign!` for patched keg binaries to avoid the
`codesign` signing path this branch is replacing.
Extracted from the typecheck-deadcode branch (Homebrew/brew#22733 was the
first extraction), which uses Spoom to find code with no static callers.
Remove four methods with no callers:
- Homebrew::API.cached_formula_json_file_path
- Homebrew::API::Cask.all_renames
- Homebrew::API::Formula.all_renames
- APIHashable#generated_hash!
Renames are resolved through Homebrew::API.formula_renames and
Homebrew::API.cask_renames, which read the internal API, so the
all_renames pair on the JSON API classes is unused. The only remaining
references were vestigial RSpec stubs, so drop those too.
APIHashable#generated_hash! reverts the global monkeypatches applied by
generating_hash!. The generate-*-api commands exit once they are done and
never call it, so it is only needed to stop tests leaking global state.
Move it to a test helper rather than keeping it in production code.
