## https://sploitus.com/exploit?id=A2534545-D334-5921-BAF4-0B9D04D53E19
# TawasolPay — AI-Powered Cyber Risk Assistant
Takes TawasolPay's data pack (60 assets, 114 open vulnerabilities, 40 threat intel records,
20 business services, an MDR advisory) and produces a ranked, evidence-backed top-5 risk list
with remediation guidance retrieved from the real NIST SP 800-53 Rev. 5 catalog.
**Live system:** _(deployment URL)_
---
## What it does
**Ranks risks on evidence, not CVSS.** Six weighted components produce a 0-100 score per
vulnerability. CVSS is capped at 10 of those 100 points on purpose, so a CVSS 10 on an
isolated dev box cannot outrank a CVSS 8 on an internet-facing payment gateway with a live
ransomware campaign against it. There is a test asserting exactly that property against the
real dataset (`tests/test_pipeline.py::test_exposure_beats_raw_cvss`).
| Component | Max | What moves it |
|---|---|---|
| Threat campaign match | 25 | A named actor campaign in `threat_intelligence.csv` matching this CVE, weighted by confidence, ransomware association, exploit maturity |
| Active exploitation | 20 | The CSV's `exploit_available` flag, corroborated against the live CISA KEV catalog, plus KEV's own ransomware-campaign flag |
| Business criticality | 20 | Asset criticality, the business service's revenue impact, customer-facing status, compliance scope breadth, RTO |
| Internet exposure | 15 | Whether the asset is reachable from the internet |
| Missing controls | 10 | No EDR, exploitable without auth, stale `last_seen`, no owning team |
| CVSS baseline | 10 | Raw severity, deliberately capped as one input of six |
The weighting order mirrors the prioritisation guidance the MDR advisory itself states
(exposure → active exploitation → ransomware association → business criticality → missing
compensating controls), rather than weights I invented.
**No LLM is involved in the ranking or the retrieval.** Scoring is pure pandas joins and
arithmetic; control selection is embedding similarity. An LLM is used only to rephrase
already-computed facts into fluent prose, and the system runs completely without one — with
no API key set it produces the same ranking and the same retrieved NIST guidance, using
deterministic templates for the wording. That is why the live deployment works whether or not
a free-tier key is present.
**Each of the top 5 entries carries:** the asset, the vulnerability and CVE, the matched threat
intel (actor, campaign, confidence, ransomware association), the business service at risk, a
plain-English explanation of why it ranks where it does, the retrieved NIST control, the
relevant section of this morning's MDR advisory, and the internal runbook hint where one
applies safely.
## How it fits together
```
data/source/*.csv ─────────► pandas joins ──┐
assets, vulns, services, (asset_id, │
threat intel cve, service) │
├──► scoring.py ──► ranked risks
CISA KEV catalog ──────────► exact CVE ──────┘ (6 weighted │
(cisagov/kev-data) lookup components) │
▼
NIST SP 800-53 Rev. 5 ─────► embed 300 ─────► cosine ─────────► top control + text
(usnistgov/oscal-content) base controls similarity │
(bge-small, ▼
MDR advisory (.md) ────────► ONNX, local) ──► campaign section ──► narrative.py
(template, or
LLM rephrase)
│
▼
FastAPI /api/report
+ web UI
```
## Running it locally
```bash
git clone https://github.com/spearb0lt/tawasolpay-risk-assistant.git
cd tawasolpay-risk-assistant
python -m venv .venv
source .venv/bin/activate # Windows: .venv\Scripts\activate
pip install -r requirements.txt
uvicorn app.main:app --reload
```
Open . No API key is required — the app serves the full report out of
the box.
To enable LLM-polished prose, copy `.env.example` to `.env` and set any one of `GROQ_API_KEY`,
`GEMINI_API_KEY`, or `OPENAI_API_KEY`. Provider selection, model pinning, and any
OpenAI-compatible gateway (OpenRouter, a local Ollama) are all environment variables; no code
changes.
Run the smoke tests with `python tests/test_pipeline.py` (no pytest required).
### Refreshing the upstream reference data
The parsed NIST corpus, its embeddings, and the trimmed KEV lookup are committed so a fresh
deploy needs no network access and boots in under a second. To rebuild them from source:
```bash
python scripts/fetch_sources.py # downloads NIST OSCAL catalog + CISA KEV
python scripts/build_kev_lookup.py # -> data/processed/kev_lookup.json
python scripts/build_nist_corpus.py # -> data/processed/nist_controls.json
python scripts/build_nist_embeddings.py # -> data/processed/nist_embeddings.npy
```
## Layout
```
app/
services/scoring.py the risk engine: joins + the six weighted components
services/report.py assembles the final top-N entries
services/narrative.py prose, template-first with optional LLM rephrase
services/hints.py matches the internal runbook hint, with a product guard
rag/retriever.py NIST control index + MDR advisory index (fastembed + numpy)
rag/query_builder.py builds the retrieval query from a scored risk
llm/ provider-neutral LLM layer (Groq / Gemini / OpenAI-compatible)
scripts/ fetch and build the reference data artifacts
data/source/ the data pack, plus raw upstream dumps (gitignored)
data/processed/ parsed NIST corpus, embeddings, KEV lookup (committed)
web/ single-page frontend
```
---
## Supporting question 1 — the data split
**Embedded (semantic retrieval):** the NIST SP 800-53 Rev. 5 control catalog and the MDR
advisory. Both are free-form prose, and both have to answer a question that has no join key.
Nothing in the data connects `CVE-2024-21762` to `SI-2 Flaw Remediation`; the link is a
judgement about meaning — "unauthenticated RCE on an internet-facing appliance with no EDR"
resembles "identify, report, and correct system flaws" — which is exactly what an embedding
index is for. The catalog is static, so the 300 base controls are embedded once
(`bge-small-en-v1.5`, ONNX, runs locally, no API key) and searched with cosine similarity over
a 300×384 matrix. At that size a vector database would be infrastructure with no benefit, so
retrieval is a numpy dot product.
**Queried as structured records:** the five CSVs and the CISA KEV catalog. These have exact
join keys (`asset_id`, `cve`, `business_service`), tiny cardinality, and every question I ask
of them is a filter, a join, or an arithmetic weight — "is this asset internet-exposed", "does
this CVE appear in KEV", "what is this service's RTO". Embedding them would convert exact,
auditable lookups into approximate ones and make the ranking impossible to explain or test; a
vector search that returns *roughly* the right asset is a bug, not a feature. The CISA KEV
catalog is 1,709 rows of mostly structured fields keyed by CVE ID, so it joins the same way.
The split is per-field, not per-file. The one place I initially got this wrong was
`remediation_guidance.csv`: its `finding_type` is a short free-text label, so I tried matching
it semantically, and both token overlap **and** embedding similarity confidently mapped
"Atlassian Jira Server-Side Template Injection" to "Confluence RCE via Template Injection"
(0.771) — which would have printed *"patch Confluence; disable OGNL template evaluation"* for a
Jira server. Neither matcher was the answer; a guard was. The system now rejects any hint
naming a product the vulnerability does not name, and falls back to showing no hint rather than
a wrong one (`app/services/hints.py`, with a regression test).
## Supporting question 2 — where it goes wrong
**1. 47 of the 114 vulnerabilities can never be KEV-confirmed, and the UI does not say so.**
Those rows carry synthetic IDs (`CVE-SYN-2026-xxxx`) that by construction will never appear in
the CISA KEV catalog, so they can never earn the 12 points that a KEV match plus KEV's
ransomware flag contribute. `CVE-SYN-2026-0004` is a concrete casualty: the MDR advisory names
it as RedMantis's *initial access vector*, yet it ranks 15th of 114 (score 69.7), below real
CVEs it is chained with. Worse, the report renders "no KEV badge" identically for *"we checked
KEV and this CVE is absent"* and *"this ID is not the kind of thing KEV indexes"*, which are
very different claims. **What catches it today:** a missing KEV entry is never treated as
evidence of *non*-exploitation — the CSV's own `exploit_available` flag and the threat-intel
match still contribute up to 33 points, which is why synthetic CVEs still reach the upper
ranks at all, and a test asserts they are scored rather than dropped. **What I would add:** a
three-state provenance field (`confirmed` / `checked-absent` / `not-checkable`) surfaced in the
UI, so a reader never mistakes an un-checkable ID for a clean bill of health.
**2. The threat-intel join is exact-string on CVE, so 10 of the 40 intel records can never
match anything.** `threat_intelligence.csv`'s key column is `matched_cve_or_control`, and ten
rows key on a *control* or configuration identifier instead of a CVE — `CICD-SYN-001`,
`CLOUD-SYN-001`, `K8S-SYN-001`, `INSIDER-SYN-001` and others. My join compares that column to
`vulnerabilities.cve`, so those rows score zero on the 25-point campaign component no matter
what. Two of them are ransomware-associated, including NightHarbor's "Backup Hunter"
(`CLOUD-SYN-001`) — the exact campaign the advisory's own *Intelligence gaps* section warns is
profiling backup infrastructure. A live ransomware campaign aimed at our backups is currently
invisible to the ranking. **What I would do:** route non-CVE intel keys down a second matching
path against asset attributes and control gaps (asset type, `edr_installed`, exposure) rather
than the CVE column, and report unmatched intel rows explicitly as coverage gaps instead of
silently dropping them.
**3. The top control is decided by margins far smaller than the retriever's precision.** Risk
#4 (Jira template injection) returns SI-2 at 0.7065 with RA-5 at 0.7035 — three thousandths
apart. Risk #5 (Confluence OGNL injection, the same weakness class, the same campaign) returns
the *opposite* order: RA-5 at 0.6965, SI-2 at 0.6954, about one thousandth apart. Two
near-identical findings therefore get different headline controls, and the difference is not
cosmetic: RA-5 is *Vulnerability Monitoring and Scanning*, which tells you to go find
vulnerabilities you have already found, while SI-2 *Flaw Remediation* tells you to patch. RA-5
leads three of the five entries, so the report's most common recommendation is arguably its
least actionable one. **What catches it today:** the two runners-up and all similarity scores
are shown in the UI, so a reader can see the call was close rather than trusting a single
control. **What I would do:** stop treating this as a pure similarity problem — a
known-exploited, patch-available finding should prefer the remediation control by rule, with
retrieval ranking *within* the controls appropriate to the finding's state.
## Supporting question 3 — the one thing I would change next
I would replace the single-control, similarity-ranked answer with a small evaluation set and a
finding-state-aware retrieval step, because failure mode 3 is not a tuning problem — it is the
system's core claim being unverified. Right now "the most applicable NIST control" rests
entirely on cosine similarity over one query formulation, and I have no ground truth to say
whether SI-2 or RA-5 is *right* for a patch-available, actively-exploited, internet-facing RCE.
I would hand-label the correct control for perhaps 20 representative findings, measure
precision@1 against that set, and only then tune — query construction, chunking the control
statement separately from its discussion, or re-ranking the top-5 candidates with the LLM I
already have wired in. That would also let me safely re-introduce the 714 control enhancements
I currently exclude from the corpus, since I would be able to measure whether they help or
just add noise. Everything else here is either measured (the ranking has a property test, the
KEV join has assertions) or degrades safely (no API key, no hint match); the retrieval quality
is the one load-bearing claim I am currently asserting rather than demonstrating.
---
## Stack
FastAPI, pandas, fastembed (ONNX `bge-small-en-v1.5`, local, no key), numpy for the vector
search, vanilla JS frontend. LLM providers are optional and swappable by environment variable:
Groq, Google Gemini, or any OpenAI-compatible endpoint. Deployed free-tier on Render via
`render.yaml`.
Reference data: [CISA KEV catalog](https://github.com/cisagov/kev-data) and
[NIST SP 800-53 Rev. 5 OSCAL catalog](https://github.com/usnistgov/oscal-content), both fetched
by `scripts/fetch_sources.py`.