This repo hosts a large-scale CodeQL demo database for **Firefox**. Purpose: to demonstrate realistic CodeQL performance and scaling. Smaller demo repos understate costs and mislead about practical usage.
This is work in progress.
Base URL: https://github.com/hohn/codeql-for-firefox/releases
| Filename | Size | Description | URL |
|---|---|---|---|
| firefox-db-bm-none.tar.zst | 1.66 GB | Full CodeQL DB (build-mode=none) | link |
| firefox-db-cpp-scan-bm-none.formatted.sarif.zst | 72.1 MB | SARIF results, C++ scan (bm=none) | link |
| firefox-db-cpp-scan.formatted.sarif.zst | 986 KB | SARIF results, C++ scan (with bm) | link |
| firefox-db.tar.zst | 756 MB | Full CodeQL DB (trace build mode) | link |
The **bm** abbreviation is for build mode. In particular:
- bm=none → extraction only, no actual build
- plain → traced build
cd ~/large-local-only/firefox/firefox
./mach clobber
export PATH=/home/hohn/large-local-only/codeql:$PATH
source /home/hohn/.cargo/env
codeql database init \
--language=cpp \
--source-root=. \
firefox-db
./mach configure
# Run under tracing (OOM at -j20, succeeded with -j10)
codeql database trace-command firefox-db -- ./mach build -j10
codeql database finalize firefox-db**Timings**
- Plain Firefox build: ~10 minutes
- Build with CodeQL trace: ~57 minutes
- Finalize: ~50 minutes
| Phase | Directory | Size | Notes |
|---|---|---|---|
| During finalize | trap/ | 16.5G | TRAP facts |
| db-cpp/ | 7.0G | Relational store | |
| log/ | 2.8G | Build + extractor logs | |
| src/ | 561M | Source snapshot | |
| After finalize | db-cpp/ | 2.5G | Relational store |
| log/ | 2.9G | Logs | |
| diagnostic/ | 32K | Scratch | |
| Final DB size | firefox-db/ | 5.5G | Usable DB |
| Distribution file | tar.zst | 757M | Compressed archive |
Note: numbers differ between “interim DB size” (~27 GB) and “final size” (5.5 GB).
cd ~/large-local-only/firefox/firefox
./mach clobber
rm -rf obj-x86_64-pc-linux-gnu/ firefox-db*
export PATH=/home/hohn/large-local-only/codeql:$PATH
source /home/hohn/.cargo/env
./mach configure
codeql database create \
--language=cpp \
--source-root=. \
--threads 20 \
--ram=50000 \
--build-mode=none \
firefox-db-bm-none
tar --use-compress-program="zstd -19 -T0" -cvf \
firefox-db-bm-none.tar.zst firefox-db-bm-none**Results**
- Extraction time: ~17 minutes (init → finalize)
- TRAP import: 11m14s
- Final DB size: ~2–3 GB relational store + 137 MB source archive
- Compressed archive: 1.66 GB
Available suites (CodeQL 1.4.6 cpp-queries):
| Suite | Queries |
|---|---|
| cpp-code-scanning.qls | 60 |
| cpp-lgtm.qls | 108 |
| cpp-lgtm-full.qls | 178 |
| cpp-security-and-quality.qls | 181 |
| cpp-security-experimental.qls | 134 |
| cpp-security-extended.qls | 97 |
- DB: firefox-db
- Suite: cpp-code-scanning (60 queries)
- Host: Mac Studio (28c / 256 GB RAM, CodeQL 2.22.4)
| Walltime | CPU% | User CPU (s) | Sys CPU (s) | Max RSS (GB) | Maj PF | Min PF | Invol CS | Vol CS |
|---|---|---|---|---|---|---|---|---|
| 22:16 | 1065 | 13775 | 464 | 124 | 124776 | 8.2 M | 48.3 M | 230647 |
Notes:
- ~22 minutes for “short” suite (60 queries)
- ~10.6 cores saturated
- Peak RAM: 124 GB, no swap
- Very high context switch activity
- Wall time: 23m41s
- CPU time: 13 555 s (≈953% CPU utilization, ~9.5 cores avg)
- Max RAM: 126 GB
- Page faults: 294 k major, 10 M minor
- Context switches: 25.9 M invol, 248 k vol
- SARIF output: 2.9 GB (!), vs 33 MB for build-traced DB
| DB | SARIF size | Log size |
|---|---|---|
| firefox-db | 33 MB | 32 KB |
| firefox-db-bm-none | 2.9 GB | 32 KB |
Using minimize-sarif.py:
- firefox-db-cpp-scan-bm-none: 4.06 GB → 805 KB, 104 results
- firefox-db-cpp-scan: 53 MB → 2.1 MB, 732 results