feat: SBOM generation and OmniBOR build provenance (CRA compliance) - #10343
Conversation
There was a problem hiding this comment.
Pull request overview
Adds build-integrated supply-chain transparency outputs to the Autotools build: SBOM generation (make sbom) and OmniBOR/Bomsh build provenance tracing (make bomsh) to support CRA-style compliance workflows.
Changes:
- Add
scripts/gen-sbomPython generator for CycloneDX 1.6 + SPDX 2.3 JSON SBOM outputs (plus tag-value conversion viapyspdxtools). - Wire new
sbom,bomsh, and install/uninstall targets into Autotools (configure.ac, 10000Makefile.am), with tool discovery viaAC_PATH_PROG. - Add end-user documentation (
doc/SBOM.md,doc/CRA.md) and reference it fromREADME.md/INSTALL.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
scripts/gen-sbom |
New Python SBOM generator producing CycloneDX/SPDX JSON with optional dependency metadata. |
Makefile.am |
Adds sbom/bomsh targets, install/uninstall rules, and clean integration. |
configure.ac |
Detects required tools (python3, pyspdxtools, git, bomtrace3, bomsh scripts) and exports needed vars. |
doc/include.am |
Ships new docs in the dist tarball; cleans generated OmniBOR directory. |
doc/SBOM.md |
Primary documentation for SBOM + OmniBOR workflows and outputs. |
doc/CRA.md |
Downstream integrator guidance for CRA-oriented SBOM/provenance usage. |
README.md |
Adds brief pointers to the new SBOM/CRA and OmniBOR features. |
INSTALL |
Adds new sections documenting make sbom / make bomsh usage and prerequisites. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
Note that we are in the process of removing liblms and libxmss (see #10292), which will simplify the dependency analysis in the python script. |
|
Three CI failures to address: 1. 2. 3. PRB-master-job failure appears to be a reporting artifact (no test results found), not a real failure. |
FYI, this will be resolved with #10387 |
c6d9f94 to
39aefe7
Compare
There was a problem hiding this comment.
Fenrir Automated Review — PR #10343
No scan targets match the changed files in this PR. Review skipped.
|
retest this please |
There was a problem hiding this comment.
Pre-merge self-review
Ran a structured pre-merge review across scripts/gen-sbom, scripts/test_gen_sbom.py, .github/workflows/sbom.yml, Makefile.am, and the docs. Five commits result, each listing its closed findings in the trailer:
test(sbom): expand coverage; CI hard-requires pcpp
fix(sbom): correctness fixes in gen-sbom
build(sbom): Makefile.am hygiene
ci(sbom): pin deps, harden token, partial-clone bomsh
docs(sbom): correct NTIA/SPDX claims; fix broken anchor
Real defects fixed (fix(sbom))
parse_options_hURL corruption.re.split(r'/\*|//', raw)[0]truncatedPACKAGE_URL "https://..."at the first//inside the URL, emittingPACKAGE_URL = "https:in the SBOMwolfssl:build:*properties. Replaced with a string-literal-aware state machine.detect_licensemissed the canonical FSF preamble.or\s+(any\s+)?laterfailed onor (at your option) any later version(the FSF GPLv2 preamble used in millions of upstream COPYING files); silently mis-detected asGPL-X.0-only. Broadened with a sentence-bounded filler match.- Empty
--libaccepted. A misconfigured build pointing--libat/dev/nullor an empty.asilently emitted the well-known empty-file SHA-256 (e3b0c44…b855) as the wolfSSL component checksum; both SPDX and CDX validators accepted it. Hard-fail gate on size 0 with an actionable error;--srcszero-byte placeholders get a soft warning (cross-compile setups legitimately use them).
Coverage gaps closed (test(sbom))
+42 tests covering detect_license (had zero tests), sha256_file's chunked-read path (the loop body wasn't exercised by the only existing empty-file vector), all four SBOM document generators (generate_cdx / generate_spdx / cdx_dep_component / spdx_dep_package), and the LicenseRef-without-license-text CLI rejection. Independent oracles throughout — SPDX licence list, hashlib one-shot SHA-256, CDX 1.6 / SPDX 2.3 schema field semantics.
TestParseUserSettings.setUp previously called self.skipTest on missing pcpp. The unit job didn't install pcpp, so 9 standalone-path tests silently skipped on every PR run. Now hard-fails with an actionable install hint, and the unit job installs pcpp==1.30.*.
Build hygiene (build(sbom))
Five Makefile.am improvements — none changes user-facing behaviour for any correct invocation; each closes a specific failure mode:
- bomsh artifact glob quoted for spaces in
abs_builddir SOURCE_DATE_EPOCHonly exported whengit log -1 --format=%ctreturned non-empty (silent shallow-clone case)--license-override/--license-textonly passed when set (decouples Makefile from gen-sbom's argparse default)install-bomshswitched from a two-tar pipeline (which masked source-side errors due to POSIX sh's pipeline exit-status semantics) tocp -R src/. dst/uninstall-hookdepends onuninstall-sbom+uninstall-bomshinstead of duplicating their bodies
CI hardening (ci(sbom))
pcpp==1.30.* pinned, permissions: contents: read at workflow level (defence-in-depth — workflow does no API writes), git clone --filter=blob:none for the bomsh fetch (saves ~10s per run; pinned SHA makes blob filter safe).
Doc corrections (docs(sbom))
README.mdpreviously said "Both produce SPDX 2.3 + CycloneDX 1.6 JSON validated against NTIA minimum elements". Neither path runs an NTIA-minimum-elements checker. Onlymake sbomrunspyspdxtools(SPDX-spec, not NTIA-specific). Corrected.doc/CRA.mdnamedntia-conformance-checkeras enforcing SPDX §10.1hasExtractedLicensingInfos; it does not (that is pyspdxtools' job). Corrected with explicit text on whatntia-conformance-checkeractually validates so a reader cannot be misled into running a malformed LicenseRef SBOM through the wrong tool and getting a false pass.doc/SBOM.mdanchor link#3-make-bomshresolved to the wrong section; fixed to the GFM auto-anchor for §3.1.
Re the Copilot inline review
The Copilot review raised 8 concerns. 7 are addressed: the NOASSERTION CDX wording in SBOM.md §2.4, make bomsh → make sbom rebuild ordering, hardcoded library paths replaced with the WOLFSSL_LIB_DSO_BASENAMES glob, scripts/gen-sbom added to EXTRA_DIST, trap-based staging cleanup, and removing the hardcoded licenseListVersion.
The remaining concern — install-bomsh using a non-INSTALL_DATA copy mechanism — is unchanged in intent (this round uses cp -R rather than the previous tar pipeline for clean error propagation but did not adopt find ... | xargs $(INSTALL_DATA)). The OmniBOR tree is a directory of content-addressed binary objects; using $(INSTALL_DATA) per-file is the more automakey approach but adds complexity for a directory whose layout is meant to be preserved verbatim. Open to revisiting if maintainers prefer.
Acknowledgement
@sameehj — the standalone --user-settings / --srcs entry point, SPDX 2.3 LicenseRef compliance, library-discovery glob, and CI workflow shape this PR from "works on my machine" into a production-ready, embedded-customer-friendly tool. The test infrastructure you built was solid. Thanks.
cc @Frauschi for visibility — make clean removes both the doxygen artefacts and BOMSH_OMNIBORDIR via doc/include.am's clean-local:.
525918d to
cf0bdd9
Compare
|
retest this please |
|
retest this please |
578fbee to
9d49a5d
Compare
There was a problem hiding this comment.
Skoll Code Review
Scan type: reviewOverall recommendation: COMMENT
Findings: 4 total — 4 posted, 0 skipped
3 finding(s) posted as inline comments (see file-level comments below)
Posted findings
- [Medium] make sbom does not thread --dep-version; liboqs/zlib version becomes NOASSERTION without pkg-config —
Makefile.am:409-475 (sbom target) - [Low] Redundant/dead branch logic in FIPS product bucket computation —
scripts/gen-advisory:product_model (FIPS block) - [Low] Duplicate pkg-config invocation per dependency —
scripts/gen-sbom:cdx_dep_component / spdx_dep_package - [Info] Executable-bit inconsistency between gen-sbom (755) and gen-advisory (644) —
scripts/gen-advisory:1
Review generated by Skoll
|
Good catch on the --dep-version gap. Yes, let's expose a SBOM_DEP_VERSIONS make variable so autotools packagers without pkg-config can still record dep versions. @sameehj can you pick this up along with the other Skoll findings? |
7c6dc94 to
25e9bd8
Compare
Add tooling to produce Software Bills of Materials and build provenance for wolfSSL, supporting EU Cyber Resilience Act (CRA) obligations. SBOM generation: - New `make sbom` target producing SPDX 2.3 output with NTIA minimum elements, urn:uuid document namespaces, and SPDX LicenseRef compliance. - Reproducible library discovery across autotools and CMake builds, with liboqs recorded as a linked artefact. - Standalone `scripts/gen-sbom` for embedded / RTOS / custom-builder flows that do not use the main build system, plus --srcs-file, --no-artifact-hash, and hash-source options. Build provenance (OmniBOR / bomsh): - End-to-end bomsh tracing of the built binaries with ArtifactID insertion, snapshotting the traced library before libtool relink and hashing the bomsh-traced binary. - `scripts/bomsh_verify.py` to validate provenance against the traced gitoid. Security advisories: - `scripts/gen-advisory` generating CSAF 2.0 and CycloneDX VEX, with a `make` target, VEX overlay schema/example, and CWE name data. Docs, tests, and CI: - doc/SBOM.md and doc/CRA.md, plus README/INSTALL updates. - Unit and regression tests for gen-sbom and gen-advisory. - New sbom.yml and advisory.yml workflows: SPDX validation via pyspdxtools, CSAF validation, bomsh provenance verification, SBOM artifact archiving, macOS coverage, and actions pinned to SHAs. Signed-off-by: Sameeh Jubran <sameeh@wolfssl.com>
Recognize the "GPLv3"/"GPLv3+" short form so downstream LICENSING files map to GPL-3.0-only instead of NOASSERTION. Add --dep-wolfssl, a wolfssl DEP_META entry, and --name-derived project URLs. Update tests. Signed-off-by: Sameeh Jubran <sameeh@wolfssl.com>
Add an openssl entry to DEP_META (Apache-2.0, OpenSSL 3.x git-tag purl) and a --dep-openssl flag so OpenSSL-compat products (wolfProvider, wolfEngine) can record OpenSSL as a dependency component alongside wolfSSL. Update tests. Signed-off-by: Sameeh Jubran <sameeh@wolfssl.com>
- gen-advisory: honour explicit per-version status when defaultStatus is "affected", so unaffected/fixed releases are no longer marked vulnerable - gen-advisory: fail loudly when a CVE record has no non-empty English description (CSAF/CycloneDX note text is required, minLength 1) - gen-advisory: note that --cve-id fetches from the CVE Services API - bomsh_verify: scope the object-store shape check to sha1, matching the sha1 gitoid hashing (drop the unreachable sha256-length branch) - Makefile.am: fail `make bomsh` early when python3/pyspdxtools are absent; quote $(ENABLED_LIBZ)/$(ENABLED_LIBOQS); consolidate clean-local so the omnibor/ and advisories/out/ build dirs are removed on clean - tests: cover the defaultStatus fix, the _bucket_for unknown-state hard-fail, and a csaf_validate.mjs runner self-test wired into CI Signed-off-by: Sameeh Jubran <sameeh@wolfssl.com>
Add the canonical scripts/sbom.am shared Automake SBOM recipe that downstream wolfSSL-stack products vendor, and ship it via EXTRA_DIST. Capture AM_CFLAGS/CFLAGS (not just AM_CPPFLAGS) and make the config header path overridable (SBOM_CONFIG_H) so products that carry feature -D flags in AM_CFLAGS or place config.h in a subdirectory record their real build configuration. Signed-off-by: Sameeh Jubran <sameeh@wolfssl.com>
Use Automake's $(docdir) for sbomdir so a --docdir override is honoured, match tab/space in the wolfSSL version parse ([[:space:]]), document the GNU-make requirement and the intentional install/uninstall-sbom asymmetry, and widen the SBOM workflow pull_request filter to '**' so PRs onto release/** base branches also run. Signed-off-by: Sameeh Jubran <sameeh@wolfssl.com>
Signed-off-by: Sameeh Jubran <sameeh@wolfssl.com>
Make the canonical SBOM fragment a true superset of the vendored copies so it can be re-vendored to every product without regressions. Products whose feature macros live in config.h (via AC_DEFINE) keep the existing behaviour: the recipe derives them from a CC -dM -E dump with AM_CPPFLAGS/AM_CFLAGS/CFLAGS and a force-included SBOM_CONFIG_H. Products whose feature flags are NOT in config.h (wolfMQTT, wolfTPM, wolfscep, which record them in a generated options.h) can now set SBOM_OPTIONS_H to point gen-sbom at that header directly. When unset the compiler-dump path is used exactly as before. This folds the options.h capture that had diverged into the wolfMQTT/ wolfTPM/wolfscep copies back into the single source of truth, while retaining the SBOM_CONFIG_H override, AM_CFLAGS/CFLAGS capture and $(docdir) install those copies were missing. Signed-off-by: Sameeh Jubran <sameeh@wolfssl.com>
4a9aa68 to
c6a336f
Compare
wolfSSL removed liboqs: Falcon is now provided natively by wolfCrypt, and --with-liboqs is a deprecated no-op (configure.ac). A build therefore no longer links liboqs, so recording it as an SBOM dependency is dead code and the SBOM integration CI (which asserted a liboqs dep package) failed. Remove the liboqs dependency throughout: - scripts/gen-sbom: drop DEP_META['liboqs'] and the --dep-liboqs flag. - Makefile.am / configure.ac: drop --dep-liboqs "$(ENABLED_LIBOQS)" and the now-unused AC_SUBST([ENABLED_LIBOQS]). - .github/workflows/sbom.yml: drop the liboqs install / --with-liboqs steps and the liboqs dep assertion; keep the native-Falcon build so the HAVE_FALCON build-property capture is still exercised. - scripts/test_gen_sbom.py: drop the liboqs-specific tests, guard against the key reappearing, and use openssl as the example dep elsewhere. - doc/SBOM.md: drop the --dep-liboqs / liboqs dependency references.
|
merge conflicts are resolved |
|
conflicts resolved and there are no whitespace/tabs/ctrl/non-ASCII |
conflicts resolved and there are no whitespace/tabs/ctrl/non-ASCII
There was a problem hiding this comment.
Only commenting rather than reviewing as I'm missing a whole middle part that I couldn't find in the comments of the PR.
My main question is: why is this all living here and not in other repos? I presume there is some kind of standard that dictates it, but I can't find it mentioned in the PR description.
|
"Why here and not other repos": it is in the other repos. This PR is the base implementation; the per-product targets are separate PRs:
As for why it's a build target at all rather than a standalone tool: wolfssl is compile-time configured, so what's actually in the shipped library depends on the The standard is the EU Cyber Resilience Act (Regulation (EU) 2024/2847), which requires manufacturers to produce an SBOM covering at least the top-level dependencies of the product. Main obligations apply from December 2027. That's the driver. "Missing a whole middle part": true. It also contains All that stuff will be moving to wolfGlass, but this PR has to land first so we have SBOM generation that will work for customers. |
|
@LinuxJedi the background you're missing is written up in wolfSSL/wolfGlass, which is public: https://github.com/wolfSSL/wolfGlass/blob/master/docs/PLAN.md Section 3 covers why the tools currently live in the wolfssl repo and why that's a problem. Section 4 covers what stays central versus what gets vendored per product. Section 19 phase 3 is the wolfssl migration specifically: wolfGlass becomes canonical for the generator and
|
wolfGlass was seeded from a pre-merge snapshot of wolfSSL/wolfssl#10343. Upstream then removed liboqs in be9963f3 ("sbom: drop liboqs dependency support (removed from wolfSSL)") before the PR merged, because Falcon is now provided natively by wolfCrypt and wolfSSL no longer links liboqs. The vendored copy therefore re-introduced a dependency that upstream had deliberately dropped, and would have emitted an unresolvable liboqs component into any product SBOM generated with --dep-liboqs. The test suite had been edited in the same wrong direction: it asserted liboqs belongs in DEP_META, where upstream asserts it must never reappear. share/gen-sbom is now byte-identical to wolfssl master scripts/gen-sbom. tests/test_gen_sbom.py differs only in the path adaptations wolfGlass needs (share/gen-sbom, provenance/bomsh_verify.py, selftest.yml). Products that have already vendored share/ (wolfBoot #830, wolfHSM #414) must re-sync and update their .wolfglass-rev pin. Signed-off-by: Sameeh Jubran <sameeh@wolfssl.com>
Add `sbom`, `install-sbom` and `uninstall-sbom` targets to both build systems, producing CycloneDX and SPDX output for CRA compliance. The recipe is the canonical scripts/sbom.am fragment from wolfSSL (wolfSSL/wolfssl#10343), vendored byte-identical; Makefile.am only declares the wolfMQTT-specific inputs. Feature macros come from the generated wolfmqtt/options.h via SBOM_OPTIONS_H rather than config.h, since wolfMQTT uses no AC_DEFINE. Both paths stage an install to hash the real installed library, record wolfSSL as a dependency component, validate the SPDX with pyspdxtools, and are exercised in CI. Requires a wolfssl source tree via WOLFSSL_DIR for scripts/gen-sbom, plus python3 and spdx-tools on the build host.
Sync scripts/sbom.am byte-for-byte with the canonical copy in the wolfSSL repository (wolfSSL/wolfssl#10343) so the SBOM recipe cannot fork from the single source of truth. wolfProvider carried the oldest variant of the fragment, missing the SBOM_OPTIONS_H branch, the SBOM_CONFIG_H override, the AM_CFLAGS/CFLAGS macro capture, and sbomdir = $(docdir). These are inert for wolfProvider today (its feature flags come through config.h and it does not set SBOM_OPTIONS_H), so this is a maintenance sync, not a behavior change. `diff scripts/sbom.am <wolfssl>/scripts/sbom.am` is now empty. The SBOM_DEP_WOLFSSL = yes / SBOM_DEP_OPENSSL = yes settings in Makefile.am are unaffected (they override the fragment's ?= defaults), so the SBOM still records both wolfSSL and OpenSSL as dependencies.
Replace the copied scripts/sbom.am plus a live wolfSSL gen-sbom checkout with the vendored wolfGlass toolkit under tools/sbom/. The generator no longer depends on wolfSSL/wolfssl#10343. The full SBOM job builds the OpenSSL + wolfSSL + wolfProvider stack, so it runs on the nightly suite and on PRs labeled ci:sbom rather than on every pull request. Signed-off-by: Sameeh Jubran <sameeh@wolfssl.com> Co-authored-by: Cursor <cursoragent@cursor.com>
Replace the copied scripts/sbom.am plus a live wolfSSL gen-sbom checkout with the vendored wolfGlass toolkit under tools/sbom/. The generator no longer depends on wolfSSL/wolfssl#10343. The full SBOM job builds the OpenSSL + wolfSSL + wolfProvider stack, so it runs on the nightly suite and on PRs labeled ci:sbom rather than on every pull request. Signed-off-by: Sameeh Jubran <sameeh@wolfssl.com>
wolfSSL/wolfssl#10343 merged on 2026-07-27, but the SBOM workflow still checked out wolfssl at refs/pull/10343/head. GitHub keeps PR refs alive after merge, so CI stayed green while validating a pre-merge snapshot rather than master. Point wolfssl_ref and its fallback at master. Distinguish the two gen-sbom lookup failures. A set-but-wrong WOLFSSL_DIR previously printed the "re-run with WOLFSSL_DIR=..." advice to someone who had already set it; it now names the path that was probed and states that WOLFSSL_DIR is a source tree, not an install prefix. Refresh the README and in-recipe notes that still told users to track the #10343 branch until it merged.
The gate skipped SBOM generation and exited 0 whenever wolfssl's scripts/gen-sbom was absent, so the job could report success having never exercised `make sbom`. That was deliberate while the script was unmerged, but it landed on master in wolfSSL/wolfssl#10343. On master, a missing gen-sbom or one too old for --dep-wolfssl is a regression, so fail instead of skipping: a green run now always means the target was exercised and the wolfssl-dependency assertions ran. Only a deliberately pinned older wolfssl_ref still degrades to a skip. The ref is passed through env rather than interpolated into the shell script, to keep the workflow expression out of the command line.
Summary
Adds two complementary supply chain transparency targets to the wolfSSL autotools build:
make sbom— generates a Software Bill of Materials (SPDX 2.3 + CycloneDX 1.6) for EU Cyber Resilience Act (CRA) compliancemake bomsh— generates an OmniBOR artifact dependency graph (cryptographic source-to-binary traceability) using the Bomsh projectmake bomshenriches the SPDX document with aPERSISTENT-ID gitoidExternalRef, bridging component identity and build provenance in a single filemake sbom
Produces three files:
wolfssl-<version>.cdx.jsonwolfssl-<version>.spdx.jsonwolfssl-<version>.spdxSBOM contents: name, version, supplier, license (parsed from
LICENSINGat generation time), copyright, SHA-256 of installed library, CPE, PURL, download location, build configuration defines, and optional third-party dependency versions (liboqs, libz, libxmss, liblms).Implementation:
scripts/gen-sbom(Python 3, stdlib only) stages amake installinto a temp directory, hashes the installed library, generates both formats, removes staging.configure.acdetectspython3,pyspdxtools, andgitviaAC_PATH_PROG.make sbomfails with a clear error if either required tool is missing.make bomsh
Runs a full clean rebuild under
bomtrace3(patched strace — userspace only, no kernel modifications) to produce an OmniBOR artifact graph inomnibor/. Ifbomsh_sbom.pyis available andwolfssl-<version>.spdx.jsonexists, annotates the SPDX with aPERSISTENT-ID gitoidExternalRef.configure.acdetectsbomtrace3,bomsh_create_bom.py, andbomsh_sbom.pyviaAC_PATH_PROG. The raw logfile and conf file are written to the build directory (not/tmp/) to avoid concurrent-build collisions. All generated files removed bymake clean.Install targets
make install-sbom 8000 # installs SBOM files to $(datadir)/doc/wolfssl/ make install-bomsh # installs omnibor/ and enriched SPDX to $(datadir)/doc/wolfssl/Documentation
doc/SBOM.md— unified reference covering both targets, combined workflow, all output files, implementation notesdoc/CRA.md— product-integrator guide: how to incorporate wolfSSL's SBOM artefacts into a downstream product SBOM (SPDXExternalDocumentRef, CycloneDX component reference), commercial license guidance, OmniBOR gitoid meaning, auditor handoff checklist, links to OpenSSF CRA and SBOM Everywhere SIG guidanceINSTALL: sections 21 and 22README.md: brief SBOM/CRA and OmniBOR/Bomsh sectionsdoc/include.am:SBOM.mdandCRA.mdadded todist_doc_DATATest plan
./autogen.sh && ./configure && make && make sbom— verify three SBOM files produced and SPDX validatesmake install-sbom/make uninstall-sbom— verify files install and remove cleanlymake clean— verify all generated files removedbomtrace3andbomsh_create_bom.pyin PATH:make sbom && make bomsh— verifyomnibor/produced andomnibor.wolfssl-<ver>.spdx.jsoncontainsPERSISTENT-ID gitoidExternalRefbomtrace3in PATH:make bomsh— verify clear error message, non-zero exitmake sbom:make bomsh— verify OmniBOR graph produced, NOTE printed about missing SPDX, no failure