## https://sploitus.com/exploit?id=5E898517-A886-5617-9E04-4F1EDBF6022B
# Vulnerability Intelligence Pipeline
A real ETL pipeline against live data: pulls security advisories from [GitHub's Security Advisory Database](https://github.com/advisories) (the same source `npm audit`, `pip-audit`, and Dependabot draw from), loads them into a local SQLite database with idempotent upserts, and ranks them with a custom priority score that's meant to answer a more useful question than raw CVSS does: what should actually get looked at first.
This is the odd one out in this set of projects on purpose. The other three are "train a model on a dataset." This one is a pipeline: extract, transform, load, and a schema designed to be queried, not a notebook that runs once and produces a chart.
## Why CVSS alone isn't enough
CVSS scores how bad a vulnerability is in isolation. It doesn't know whether the affected package is used by five people or five million, and it doesn't know that an advisory published an hour ago is a different kind of urgent than one from two years ago that the ecosystem has already patched around. The priority score here is:
```
priority = cvss_score ร ecosystem_weight ร recency_decay
```
- **ecosystem_weight** โ a coarse popularity ordering across package ecosystems (npm and PyPI weighted highest, niche ecosystems lower), since GitHub's API doesn't expose live download counts per ecosystem to weight this more precisely.
- **recency_decay** โ halves every 30 days since publication, floored at 0.05 so an old, still-unpatched advisory doesn't vanish from the ranking entirely, it just stops dominating it.
Full formula and reasoning in `src/transform.py`.
## Pipeline stages
```
src/extract.py GitHub Advisory API -> raw JSON (paginated, live data)
src/transform.py raw JSON -> clean AdvisoryRecord + priority score
src/load.py AdvisoryRecord -> SQLite, idempotent upsert on ghsa_id
src/pipeline.py orchestrates all three, logs each run to pipeline_runs
src/report.py CLI to query the current top-priority advisories
```
Re-running `python -m src.pipeline` later doesn't duplicate anything, it updates existing advisories in place and adds newly published ones, which is what actually makes this a pipeline instead of a one-off script. Withdrawn advisories are dropped rather than kept around with a stale score.
## Running it
```bash
pip install -r requirements.txt
# optional but recommended โ raises the GitHub API rate limit from 60/hr to 5,000/hr.
# no special scopes needed, this only reads public advisory data.
export GITHUB_TOKEN=your_token_here
python -m src.pipeline --pages 5 # fetch ~500 most recently updated advisories
python -m src.report --limit 15 # show the current top 15 by priority
python -m src.report --ecosystem npm # filter to one ecosystem
pytest tests/ # 21 tests, all offline (no live API calls)
```
## Sample output
Pulled live on 2026-07-15, top of the current priority ranking:
| Priority | CVSS | Severity | Ecosystem | Summary |
|---|---|---|---|---|
| 9.78 | 9.9 | critical | npm | n8n-MCP cross-tenant access to workflow version backups |
| 9.49 | 9.6 | critical | npm | TidGi Desktop RCE via malicious TiddlyWiki repository |
| 9.07 | 9.9 | critical | pip | DIRAC RCE in FileCatalog DatasetManager via SQL injection |
| 9.06 | 9.9 | critical | pip | DIRAC RCE in RequestManager via eval on untrusted input |
This will look different every time it's run, that's the point, it's live data, not a snapshot.
## Tests
21 tests across `tests/test_transform.py` and `tests/test_load.py`, covering the scoring formula (recency decay at known checkpoints, ecosystem weighting, CVSS fallback logic) and the database layer (idempotent upserts, ordering, filtering). None of them hit the network โ they run against fixture data, so the suite runs in milliseconds and doesn't depend on GitHub's API being up or rate limits being available.
## What this deliberately doesn't do
- **No exploit code, no PoCs, no attack tooling.** This is a defensive triage tool, it aggregates and ranks *public, already-disclosed* advisories to help someone patch faster. It doesn't touch anything offensive.
- **Ecosystem weights are a coarse manual ordering, not real usage data.** GitHub's advisory API doesn't expose download counts, so "npm > PyPI > Maven > ..." is a reasonable approximation, not measured popularity. A future version could pull real download stats per ecosystem (npm registry API, PyPI BigQuery dataset) and replace this.
- **No CVE/NVD cross-referencing beyond what GitHub already links.** A more complete pipeline would also reconcile against the NVD and CISA's Known Exploited Vulnerabilities catalog to flag advisories with confirmed in-the-wild exploitation, which is a stronger signal than recency alone.
## License
MIT, see `LICENSE`.