Daily, plain-language GTFS quality scorecards for small transit agencies: see the current grade, the three things to fix, and why each one matters.
GTFS Scorecard is a daily data-quality service for small transit agencies and the program staff, maintainers, and vendors who support them. It publishes dated grades, prioritized fixes, history, program views, practitioner tools, and open data. It does not implement a competing GTFS validator. Correctness findings come from MobilityData's canonical validator.
Requires Python 3.12+, uv, and Java 17+ (the validator jar is downloaded automatically on first run).
cd pipeline
uv sync
uv run scorecard run --allThis fetches today's snapshot of every current configured feed, validates and
scores it, and writes artifacts to data/artifacts/<agency>/<date>.json plus a
latest.json and a cross-agency index.json. Retired aliases are excluded
from the batch; an explicit historical rescore writes a dated record without
recreating mutable current files. Re-running a day is idempotent. Checks (from
the repo root; mirrors the CI gate):
make verifyThe frontend reads the JSON artifacts over HTTP. Serve the repo root and open
the page through http://, not by double-clicking the file:
cd .. # repo root, so data/artifacts/ is reachable
python3 -m http.server 8000
# then open http://localhost:8000/web/index.htmlOpening web/index.html as a file:// URL leaves the page stuck on
"Loading scorecards…": browsers block ES module loading and fetch over
file://, so the app never runs. Any static server works; the only requirement
is that data/artifacts/ sits one level above web/, which the
../data/artifacts fallback in web/src/app.js expects.
GTFS Scorecard is the evidence and triage layer:
- Mobility Database and Transitland help identify public feeds and their sources or versions.
- The MobilityData GTFS validator supplies canonical specification findings.
- An agency, vendor, editor, or support program changes and publishes the feed through its existing workflow.
- A 90-day GTFS Scorecard pilot is testing the final handoff: link an accepted action to the intended feed, run a comparable recheck, and preserve the result.
The California GTFS Quality Dashboard inspired the project's daily scorecard and support-program framing. GTFS Scorecard is independent and is not an official compliance determination.
Pilot agencies: Unitrans (ASUCD / City of
Davis) and Yolobus (Yolo County Transportation
District). Beyond the pilots, the registry contains more than 2,100 curated
feed records, mostly in the United States and Canada, plus reviewed canaries
across Europe, Asia-Pacific, and South America. Scoring is scheduled daily for
every registered feed, and the public status page reports the exact current
configured and published counts along with when the latest run completed. Read
the published total there rather than from this repository: automation stopped
committing generated data at the S3 cutover (see
docs/follow-ups.md), so what git carries is a fallback
snapshot that moves only when it is deliberately re-materialized from the live
corpus (last refreshed 2026-08-07),
with more than 2,100 numeric scorecards published. CI can only gate the
snapshot figure, because make verify runs offline. A feed record is not
always a distinct transit agency: regional feeds and modal variants are counted
separately while the identity registry is reconciled. A record the Mobility
Database has retired in favour of another is no longer one of them: it keeps
its dated artifacts and redirects its old URL, and only its successor publishes
a current grade, so one agency does not carry two
(docs/feed-supersessions.md). That leaves the published count lower than it
was, and right.
Live: gtfsscorecard.org — with the latest completed-run evidence at gtfsscorecard.org/status.
Status: Beta. The public scorecards and daily evidence service are live.
The remediation workflow is being tested and is not yet a proven service.
Realtime quality is scored only when a usable realtime feed is configured and
measured; otherwise it is shown neutrally as not yet measured. Any agency can
be added through registry/intake.yaml.
- Check a feed: browse gtfsscorecard.org, or request a one-off score.
- Track an agency: use the self-serve submission form, or follow the repository walkthrough to open a pull request.
- Gate a feed in CI: add the GTFS Scorecard Marketplace Action to a feed repository.
- Test what happens after the scorecard: read the remediation pilot request if you can take a finding through an accepted request and same-feed recheck.
- Contribute: choose a bounded open issue or read CONTRIBUTING.md. Feed corrections, accessibility review, and practitioner feedback are useful without changing scoring code.
- Get help, or fund the work: SUPPORT.md is the front door for both. Scoring is free for every agency and stays that way; the support page explains what running the service costs, what sponsorship would pay for, and what it never buys.
- An overall grade from the categories that can be measured: correctness, freshness, rider experience completeness, and, when available, realtime quality.
- "Top 3 things to fix", in plain language with effort hints. Findings are framed as fixes, never as failures.
- For U.S. agencies only, an NTD GTFS-readiness read (published, valid,
current, and
agency_idpresent) plus a neutral check of whetheragency_idhappens to equal the five-digit NTD ID. Equality is optional; RY2026 presence and the P-50 crosswalk are not. - Trend history, one JSON artifact per agency per day.
- An embeddable grade badge (
<agency>/badge.svg) the agency can put on its own developer page.
The scoring methodology, with citations to the California Transit Data
Guidelines v4.0 and the validator's rule taxonomy, is in
docs/rubric.md. Methodology changes are governed: a
validator-version bump must attach the shadow-run impact report from
scorecard canary before it lands (rubric.md, "Governed upgrades").
The grade, weights, thresholds, and fix order are GTFS Scorecard policy choices,
not an official MobilityData grade or a universal quality standard.
Feed sources and licenses are in
docs/feeds.md. Forward planning is split in two: the
infrastructure and scaling plan is in docs/roadmap.md, and
what the product becomes for its users is in
docs/product-roadmap.md.
Each agency gets a scorecard page, plus three companion pages written for the
different seats at an agency check-in: a board one-pager
(/agency/<id>/board/), a call-prep brief (/brief/), and, when comparable
finding-clearance records exist, a clearance log (/fixes/). Around those sit:
- Coverage views — the coverage overview (
/pulse/), most common problems (/problems/), realtime reliability (/realtime/), optional-feature adoption and accessibility data coverage (/adoption/), a consumer feature finder with translation-language and accessibility filters (/app/#/?view=features), and worldwide agency and route maps (/map/,/routes/). U.S.-specific NTD readiness (/ntd/) and equity (/equity/) stay available as regional modules. - Program pages (
/program/<state>/) for 46 states plus DC and named cohorts fromrollups.yaml, each with the fixes shared across the group. - Practitioner tools — request a one-off score through GitHub or run it
locally (
/try.html), check a feed before publishing (/check/), compare two agencies (/compare/), query the dataset with SQL in the browser (/query/), and put feed quality in a vendor contract (/procurement/). - Board-ready outputs — each agency has a printable one-pager; see the live Unitrans example, or generate a self-contained, custom-branded offline report with the board-report tool.
- A fix knowledge base (
/fix/<rule>/, one plain-language page per common validator finding) and the standards crosswalk (/crosswalk/). - Machine-readable surfaces — the versioned read API (docs/api.md), a Parquet table for bulk SQL, per-agency badges and conformance marks, Atom change feeds, monthly citable dataset releases, and a read-only MCP server (docs/mcp.md).
The country controls describe only the current covered set. They are not a census of a country or region. The live evidence and readiness result for a bounded European GTFS beta are on the status page; the release gate and scope limits are in docs/global-expansion.md.
Gate your own pipeline on feed quality with the published action. It scores the
feed, prints the grade and the top fixes in the job log, and fails the build if
the feed drops below min-grade or expires within min-days-to-expiry:
- uses: ChelseaKR/gtfs-scorecard@v1
with:
feed-url: https://your-agency.example/google_transit.zip
country: CA
min-grade: C
min-days-to-expiry: 14country is the feed's assigned ISO 3166-1 alpha-2 code; it defaults to US
for existing workflows. Both thresholds are optional; leave one blank to skip that check. Full input
reference and a complete workflow are in docs/ci-action.md.
Beyond run, the CLI carries the commands the rollout plan needs:
scorecard sync --country US --state California # propose keyless, untracked
# Mobility Database entries
scorecard shards --count 4 # JSON fan-out plan for CI
scorecard reindex # rebuild index.json from disk
scorecard rollups # publish program rollup artifacts
scorecard render-site # crawlable static pages + sitemap
scorecard alerts --out digest.md # expiry/regression digest
scorecard notify # per-subscriber digest (dry run)Add --source-metadata-out <path> to scorecard sync for a source receipt
that accounts for every recognized Mobility Database Schedule row as excluded,
filtered, already tracked, duplicate, conflicted, or proposed for review. It
never edits the registry and does not treat a catalog proposal as permission to
republish.
notify builds a feed-health email for each opt-in subscriber in
subscriptions.yaml, containing only the agencies they
follow and only when one needs attention. It prints the emails by default; the
daily workflow sends them via SES once an operator verifies a sender, applies
the SES grant in infra/artifacts, and sets the SES_FROM repo variable.
The daily workflow fans agencies out across a parallel matrix and can mirror
artifacts to a CloudFront-backed S3 bucket once infra/artifacts is applied
(ADR 0002). Program rollups are configured in rollups.yaml as
named cohorts (a liaison's own agencies, a district, the whole state) and shown
at #/programs; an agency "needs attention" when its feed is expiring or its
grade regressed, not merely when it scores below a B. The published JSON is a
documented read API (docs/api.md); a flat catalog of every agency
(grade, score, feed URL, days-to-expiry, top fix) is served at /catalog.json
and /catalog.csv so a consumer needs one request, not one per agency.
The roadmap plans the path from two pilot feeds to a worldwide-capable service. The Year 1 software is built, tested, and mostly deployed; the deploy runbook walks through the AWS stacks and carries the current deployment status.
| Roadmap piece | In the repo | State |
|---|---|---|
| Mobility Database sync | scorecard sync (mobilitydb.py) |
run on demand |
| Scheduled scoring run | scorecard shards + CI matrix |
daily schedule live in Actions; observed completion is on /status/ |
| Expiry/regression alerts | scorecard alerts, notify --send, infra/alerts |
applied and live; SES sender verified, digest sends daily |
| Artifacts on S3 + CloudFront | infra/artifacts |
applied and live; daily mirror on, site still serves from Pages |
| Self-serve submission form | web/submit.html, infra/submit |
applied and live; submissions open reviewable pull requests |
| Instant scoring | web/try.html, infra/instant-score |
built; falls back to the issue-form path until applied (ADR 0029) |
| Fan-out compute (Year 2) | infra/compute (SQS + worker) |
built; apply when the daily run outgrows the Actions matrix |
The cohort drafted from the Mobility Database has grown well past the first
California pass: the manifest-backed registry now
carries more than 2,100 curated feed records, mostly across the US and Canada,
with a 528-record reviewed European cohort across 26 countries.
It now includes a geographically diverse reviewed canary cohort, scored daily
(a 2026-07 dedupe pass removed ~350 records that duplicated an already-listed
feed).
Any agency with a public GTFS feed can be added with one YAML block in
registry/intake.yaml and a pull request; the walkthrough is
docs/add-your-agency.md. The live web form at
web/submit.html does the same without YAML through the
deployed infra/submit endpoint; every submission still opens a pull request
for human review before publication.
The public scorecards, data, API, local pre-publish check, and request-backed one-off scoring remain free. The support page explains the project's infrastructure needs and accepts sponsorship inquiries; it does not advertise a payment rail that is not ready.
Programs and agencies that want a facilitated audit, workshop, or implementation help can review the current consulting services and pricing. Paid help is separate from scoring and never changes a grade or removes anything from the free tier.
This repo is governed by the shared portfolio standards vendored at
docs/standards/ (pinned to a tag — never edited locally;
see the integrity note in that directory's history). Per
docs/standards/README.md's conformance rule, every applicable standard is
declared here; none is silently skipped.
| Standard | State | Standard document |
|---|---|---|
| Code Quality | Applies (Python; TS/Node N/A — web/ is no-build vanilla JS) |
CODE-QUALITY-STANDARD.md |
| Security & Supply-Chain | Applies (ASVS L1 shape: no auth, no PII store) | SECURITY-AND-SUPPLY-CHAIN-STANDARD.md |
| CI/CD | Applies | CI-CD-STANDARD.md |
| Observability | Applies — Tier B (frontend) + Tier C (batch pipeline); see ADR 0031 | OBSERVABILITY-STANDARD.md |
| Accessibility | Applies fully — civic content, self-declared WCAG 2.2 AAA (see docs/accessibility.md, docs/vpat.md) | ACCESSIBILITY-STANDARD.md |
| Internationalization | Applies — civic transit data, public-facing; exemption path unavailable | INTERNATIONALIZATION-STANDARD.md |
| Performance | Applies — the static frontend has blocking Lighthouse budgets on code changes; the scheduled data-refresh path retains its documented advisory exception | PERFORMANCE-STANDARD.md |
| AI Evaluation | N/A — no LLM/model component: no model inference in any user-facing or decision-making path (AI-EVALUATION-STANDARD §0); the MCP server (server.json) is read-only data retrieval, no LLM SDK. Flips to APPLIES on first LLM SDK use. |
AI-EVALUATION-STANDARD.md |
| AI Development Measurement | Applies — delivery and quality-debt outcomes are measured at repository level; local AI-tool telemetry is never a merge gate | AI-DEVELOPMENT-MEASUREMENT-STANDARD.md |
| Quality & Metrics | Applies (data-quality/lineage named for this repo explicitly) | QUALITY-AND-METRICS-STANDARD.md |
| Documentation | Applies | DOCUMENTATION-STANDARD.md |
| Release & Versioning | Applies — reusable Action tags (v1/v1.5.0), monthly dataset releases; the MCP server manifest is written but not published to the registry (see Versioning) |
RELEASE-AND-VERSIONING-STANDARD.md |
| Responsible-Tech Framework | Applies (audits A-F; AI-governance rows N/A — no AI system) | RESPONSIBLE-TECH-FRAMEWORK.md |
| Incident Response | Applies — deployed static site and scheduled pipeline; incidents use the shared severity, label, postmortem, and secret-leak conventions | INCIDENT-RESPONSE-STANDARD.md |
| Data Governance | Applies — public civic datasets, feed provenance, licenses, retention, and publication lineage are core product concerns | DATA-GOVERNANCE-STANDARD.md |
Open gaps per standard, as of the most recent conformance audit, are tracked in docs/standards-conformance-gaps.md rather than restated here.
Tier B (frontend) + Tier C (batch pipeline) — not the
OBSERVABILITY-STANDARD's default Tier-A mapping for this repo's class;
see ADR 0031 for why. In short:
the deployed system is a scheduled Actions batch job plus a static site, not
a long-lived hosted service (infra/compute, the piece that would make this
Tier A, is built but not applied). Tier C's health/tracing/SLO controls are
N/A (no network surface to probe). For Tier B, every fresh site build passes a
blocking structural SEO check. Lighthouse gates code changes, and a weekly
synthetic run checks representative production routes. Build-time SEO reports
are retained for 14 days and production Lighthouse reports for 90 days.
The public site has no analytics loader, visitor-tracking code, or real-user
monitoring beacon. Search Console ownership verification and sitemap submission
remain external domain-owner tasks; this repository stores no Search Console
credentials or configuration. See the ADR and deploy runbook.
SemVer. The public API surface is the published JSON schema
(schema_version), the GitHub Action's inputs (action.yml), and the CLI
(docs/api.md). pipeline/pyproject.toml's [project].version is the single
source of truth; CITATION.cff and server.json mirror it, and
pipeline/scripts/check_versions.py fails make verify if they drift.
Supported-version policy: latest major only — this is a single-deployment
civic tool, not a library with multiple consumers pinned to old majors.
This repo also produces releases: the marketplace GitHub Action (tagged
v1) and monthly citable dataset releases (dataset-release.yml). It also
carries an MCP server manifest (server.json), which is not published to
the MCP Registry — publishing needs an interactive operator login and the
manifest still has no packages[] entry, so io.github.chelseakr/gtfs-scorecard
does not resolve there. docs/mcp.md has the install recipe that
does work and the reason the packages entry was removed.
A state DOT, national RTAP, or country program can fork this repo and stand up its own branded instance — own domain, own agency registry, own organization name — from config and a deploy, no code change: docs/fork-quickstart.md.
registry/ manifest-backed agency shards; add yours to intake.yaml
instance.example.yaml fork branding template; copy to instance.yaml to rebrand
rollups.yaml program rollups (portfolio views across many agencies)
pipeline/ Python pipeline: fetch -> validate -> score -> publish
web/ static frontend; reads only published JSON
infra/ Terraform: artifacts CDN, submission function, fan-out compute
docs/ feeds, rubric, roadmap, api, fixes/ (KB), decisions/ (ADRs)
data/ raw snapshots (ignored) and published artifacts (committed)
Three rules hold everywhere in this project: every metric is published with a
plain-language explanation of what it means; accessibility (WCAG 2.2 AAA) is
non-negotiable in the web app; and an agency without a realtime feed is shown
neutrally, never shamed. Contributor and agent-facing build rules live in
CLAUDE.md; current direction beyond the shipped roadmap is worked in the
open as labeled notes in docs/ideation/.
Built and maintained by Chelsea Kelly-Reif, starting from the transit systems in Davis, California. Contributions and corrections are welcome; agencies can ask to be added or removed under the listing and removal policy.