Share 
## https://sploitus.com/exploit?id=0C0056F1-FBAE-530D-8F20-FFA3E79D41DB
# Vulnerability Hunting Agent

> An LLM agent that **reads code, forms hypotheses, writes a PoC, and executes it in a Docker sandbox** โ€” then ships only the bugs it could actually trigger.

[**ํ•œ๊ตญ์–ด README**](README.ko.md)  ยท  Apache-2.0  ยท  Python 3.11+

---


  


**What this repo demonstrates**

> The README and the bundled specialists are written **Python-first** โ€” that's where the externally-validated CVE was found and where the experiment table below was measured. The scaffold itself is language-agnostic: C/C++ and JavaScript starter specialists are already shipped (see [config/prompts/hunters/](config/prompts/hunters/)), and Java / Go / Rust starters are planned. Adding a language is the same one-Markdown-file-per-CWE workflow described under [Customize](#customize-add-your-own-specialist).

- A reproduction of the **Anthropic "Project Mythos" scaffold** (file-level parallel hunters + a reviewer stage) โ€” see the original write-up at [red.anthropic.com/2026/mythos-preview](https://red.anthropic.com/2026/mythos-preview/) โ€” on a **publicly accessible frontier model** (we ran it on something in the 4.7-class), no preview model required.
- We don't have the preview model or infra Mythos used. To close the gap on a publicly available model we lifted three things explicitly: the **prompt**, the **agent**, and the **model**. The numbers below are the lift each layer added.
- One scan against an open-source target produced a finding the upstream maintainer accepted and published as a public advisory (details at the bottom of this README). Other findings exist on other targets; disclosure for those is in progress.

This is a PoC. It is small (~150 LOC per file), boring on purpose, and easy to fork.

---

## Closing the gap to Mythos โ€” 3 layers

Mythos works because of **(a) a frontier model**, **(b) a tool-using agent loop**, and **(c) carefully tuned prompts** โ€” together. Strip any one and the rest collapses. We don't get to use Mythos's model or infra, so we rebuilt the same skeleton on a public model and measured what each layer is worth.

The shape:

| | Mythos | Ours |
|---|---|---|
| Layer 1 โ€” Prompt | one generic prompt | **5 specialists** (auth ยท SSRF ยท deser ยท path ยท injection) โ€” fan-out by CWE |
| Layer 2 โ€” Agent | one Hunter | **per-file ร— per-specialty** independent sessions + a separate Reviewer |
| Layer 3 โ€” Model | one frontier LLM | **swappable** (Opus 4.7 / 4.6 / Sonnet 4.6 / Kimi K2) |

Same skeleton, different fan-out. The figure at the top of this README shows it side by side.

**Measured lift, same files, same model:**

**Layer 1 โ€” Prompt.** A bare *"find security vulnerabilities"* prompt yields 2 unique findings. Replacing it with a **CWE-aware prompt** that spells out an expert's mental checklist โ€” *where to look* (sinks), *what counts as safe* (sanitizers / allowlists), *what counts as evidence* (a PoC that runs) โ€” gets the model to **2 โ†’ 5** unique findings. Same model, same files; the prompt now carries the security knowledge the model otherwise has to guess at.

**Layer 2 โ€” Agent.** Wrapping that CWE-aware prompt in a tool-using agent loop and **fanning it out by category** (auth ยท SSRF ยท deserialization ยท path-traversal ยท injection โ€” one independent session per file ร— per CWE), plus a **separate Reviewer stage** that re-runs the PoC: **5 โ†’ 11**. The Reviewer matters because letting the same hunter session both find *and* judge produces over-confident calls; a fresh session that only sees grouped findings, re-runs the PoC, and assigns CVSS is allowed to drop a group entirely.

**Layer 3 โ€” Model.** With layers 1 and 2 fixed, swap the underlying model (Opus 4.7 / 4.6 / Sonnet 4.6 / Kimi K2). Spread is real but smaller than the prompt-and-agent lift, and not monotone with cost โ€” see [experiment](#experiment-which-model--scaffold-actually-works).

This is the entire reason the project is worth open-sourcing: **the lift lives in the layers above the model**, which means defenders without preview access can build the same tool. Attackers without preview access can too โ€” that's the real point.

---

## How it differs from SAST

| | Traditional SAST | This scanner |
|---|---|---|
| What it sees | Source code patterns | Source code **and** runtime behavior |
| What it returns | Every match (FP flood) | **Only what it could trigger in a sandbox** |
| Adding a new bug class | Update the rule engine | Drop a new `*.md` prompt into `config/prompts/hunters/` |
| Cross-file logic / intent gaps | Hard | Native โ€” the agent greps and follows calls |

The trade-off is cost (an LLM call per file ร— per category). The repo includes a 6-config experiment showing where that cost is and isn't worth it.

---

## Pipeline


  


```
Arch  โ†’  Filter  โ†’  Rank  โ†’  Hunter ร— N (parallel)  โ†’  Cluster  โ†’  Review  โ†’  Report
```

1. **Arch** โ€” one LLM call: language, framework, build system.
2. **Filter** โ€” drop tests, vendored, generated code (no LLM).
3. **Rank** โ€” score every source file 1โ€“5 for security relevance.
4. **Hunter** โ€” *one independent session per (top-N file ร— category)*. Reads, greps, writes a PoC into `/workspace`, executes it in a network-isolated Docker container. `network: none`, `/code` read-only, `/workspace` tmpfs.
5. **Cluster** โ€” group near-duplicate findings within a file.
6. **Review** โ€” verdict + CVSS + writeup.
7. **Report** โ€” JSON + Markdown.

Each Hunter is a fresh session. They don't share history; the diversity of independent runs is the point.

---

## The core: specialized hunters, per file

The interesting part of this repo is what happens *inside step 4*. For every top-ranked file we don't run **one** hunter โ€” we run **M** hunters in parallel, each looking through one specialty's lens.


  


**Why specialize.** A single generic prompt biases the model toward whatever it has seen most. Splitting the search by category โ€” auth, SSRF, deserialization, path-traversal, injection โ€” forces independent passes through the same code and surfaces bugs that a generalist run misses. In our experiment, **bare prompt โ†’ full specialists** gave **~5ร— more unique bugs on the same files with the same model.**

**Why a separate Reviewer.** Hunters are tuned to find. Letting the same session also decide "is this real?" produces over-confident calls. The Reviewer is a fresh session that only sees the grouped findings, re-runs the PoCs, and assigns CVSS โ€” and is allowed to drop a group entirely if the PoC doesn't reproduce.

**Why per-file.** Hunters all reading the whole tree at once would race for the same easy wins. Bounding each session to one file produces deterministic coverage and lets us run files in parallel.

The three layers map directly onto the Mythos building blocks (Ranker ยท Hunters ยท Reviewer) โ€” that mapping is the entire reason this works on a public model.

---

## Quick start

> **Requirements:** Python 3.11+, Docker, AWS credentials with Bedrock access (Opus 4.7 enabled in `us-west-2`).

```bash
# 1. install
git clone https://github.com//vulnhunt-agent.git
cd vulnhunt-agent
pip install -e .

# 2. credentials
export AWS_REGION=us-west-2
aws sso login          # or set AWS_PROFILE / AWS_ACCESS_KEY_ID

# 3. run the UI
streamlit run src/vulnhunt_agent/app.py
```

Then in the sidebar: pick a repo (git URL or local path) โ†’ click **Save** โ†’ click **Run**.

**Troubleshooting** โ€” if Bedrock returns `AccessDeniedException`, enable model access for `claude-opus-4-7` in the Bedrock console (Region: `us-west-2`); the model id lives in [config/settings.toml](config/settings.toml).


  


> Drop a screenshot at `assets/img/ui_screenshot.png` (suggested: full window, sidebar + main panel mid-run).

---

## Configuration

Everything an operator needs to change lives under [config/](config/).

**[config/settings.toml](config/settings.toml)** โ€” model catalog and defaults. Swap the hunter / reviewer / tools model independently from the sidebar:

```toml
[default_model]
model_id = "global.anthropic.claude-opus-4-7"

[[models]]
label = "Opus 4.7"
model_id = "global.anthropic.claude-opus-4-7"
input_per_m = 15.0
output_per_m = 75.0

[[models]]
label = "Kimi K2 Thinking"
model_id = "moonshot.kimi-k2-thinking"
input_per_m = 0.60
output_per_m = 2.50
supports_caching = false
```

**[config/prompts/hunters/](config/prompts/hunters/)** โ€” one Markdown file per category. The defaults shipped:

```
config/prompts/hunters/
  python_auth.md            python_path_traversal.md
  python_ssrf.md            python_injection.md
  python_deserialization.md python_general.md
  c_buffer_overflow.md      js_nosqli.md
  c_use_after_free.md       js_proto_pollution.md
  c_integer_issues.md       js_ssrf.md
  custom/                   โ† your private specialists go here
```

---

## Customize: add your own specialist

The interesting part for an internal team isn't the bundled CWE list โ€” it's that **your security and platform knowledge can be expressed as one more specialist**. A "specialist" here is just a Markdown file with a few hundred words describing *what to look for*, *what counts as safe*, and *what counts as evidence*. Drop the file, run again, and you get an extra hunter session per top-ranked file with no code change.


  


**Where to put it.** Anything in [config/prompts/hunters/custom/](config/prompts/hunters/custom/) is gitignored, so you can keep your org-internal rules out of source control. A file in the parent directory ships as a default for everyone.

**The shape of a good specialist.** Every default in [config/prompts/hunters/](config/prompts/hunters/) follows the same structure โ€” copy one and adapt:

```markdown
---
name: no_retry
title: Outbound HTTP without our retry middleware
description: Enforce that every outbound HTTP call goes through http_client.with_retry().
language: python
---

You are a security auditor whose **starting hint** is missing-retry-middleware
in this codebase. This is your entry point โ€” NOT a constraint. If you find
ANY exploitable bug while exploring, report it regardless of category.

# Starting points
- `requests.*`, `httpx.*`, `urllib.request.*`, `aiohttp.*` calls.
- Anything bypassing the project's `http_client.with_retry(...)` wrapper.
- Sync paths in async code that re-implement the request inline.

# What counts as safe
- Calls wrapped in `http_client.with_retry(...)`.
- Calls that go through the shared `RetryClient` session object.

# Evidence required
- Run a PoC: stand up a flaky local server (returns 500 on first call,
  200 on the second), call the target, and show no second attempt.
- Report only if the PoC reproduces in the sandbox.
```

**Why it works (and why it's not just "another lint rule").**

- **Starting hint, not a constraint.** Every default specialist tells the model the topic *biases* the search but doesn't restrict it โ€” that's how cross-category bugs still surface (e.g. an SSRF specialist finding a `dict` subclass `__contains__` bypass while exploring an allow-list check).
- **Sinks ยท safe patterns ยท evidence.** The same three slots as a code reviewer's mental checklist. The model needs all three; missing the *evidence* slot is the single most common reason a custom specialist drowns in false positives.
- **PoC required.** The Reviewer stage will drop a finding whose PoC doesn't reproduce in the sandbox. That bound on the false-positive rate is what makes it safe to keep adding specialists โ€” more lenses, not more noise.
- **No code change.** The orchestrator discovers `*.md` files at run time. New specialist = new file. Removing one = delete the file.

---

## Experiment: which model & scaffold actually works

**Setup.** One open-source target (`langchain-core`, ~1,700 Python files). After filtering tests / vendored / generated code we ranked the remaining ~180 source files for security relevance and took the **top 5**. Every cfg below is run against those same 5 files. Each hunter session has a hard cap of 100 tool-use iterations; reviewer is always Opus 4.7 (fixed across all cfgs so we measure the *hunter* layer). **Unique** = same root cause counted once.

| cfg | hunter model | scaffold | unique findings | crit + high | total $ |
|---|---|---|---:|---:|---:|
| 1 | Opus 4.7 | bare prompt | 2 | 1 | $6 |
| 2 | Opus 4.7 | + scaffold | 5 | 0 | $15 |
| **3** | **Opus 4.7** | **full (5 specialists)** | **11** | **6** | **$63.5** |
| 4 | Opus 4.6 | full | 10 | 1 | $127 |
| 5 | Kimi K2 Thinking | full | 5 | 0 | **$7** |
| 6 | Sonnet 4.6 | full | 9 | 6 | $39 |

Takeaways:

- **Scaffold lift is real.** Same model, same files: bare โ†’ full โ‰ˆ **5ร— more unique bugs**.
- **Opus 4.6 ran 1,082 hunter iterations** and still produced fewer high/criticals than 4.7's 412. Iteration count is not skill.
- **Kimi K2** is the cheapest by an order of magnitude but found zero high/critical findings โ€” useful as a fast first pass, not as a final verdict.
- Of the **16 unique bug categories** found across all six runs, **9 were caught by only one config** โ€” single-model ensembles miss roughly half.

---

## Inspiration & credit

The scaffold is directly inspired by **Anthropic's "Project Mythos" preview** โ€” original write-up:  (2026-04-07). The building blocks reproduced here are file-level parallel hunters, an LLM reviewer for dedup and verdicts, and sandbox-verified PoCs. The novelty here is that the same skeleton works on a publicly accessible frontier model โ€” meaning defenders without preview access can build the same tool. *Attackers can too; that is the actual point.*

---

## Project layout

```
src/vulnhunt_agent/
  agents/        hunter, reviewer, clusterer, queue
  pipeline/      arch โ†’ filter โ†’ rank โ†’ hunt โ†’ finalize
  core/          llm, settings, run_store, cvss, events
  ui/            streamlit (sidebar, steps, result_cards, cost)
  sandbox/       Docker executor
  repo/          git/local source resolver
config/
  settings.toml          # models + defaults
  prompts/hunters/*.md   # one specialist per file
```

Code style (see [CLAUDE.md](CLAUDE.md)): ~150 LOC per file, ~30 LOC per function, no `Protocol`/`ABC`/`Generic` until there are two implementations, no defensive `try/except` for cases that can't happen.

---

## Externally validated finding

A scan with this pipeline produced a finding that was reported upstream, accepted by the project maintainer, and published as a public advisory:

- **[GHSA-pjwx-r37v-7724](https://github.com/langchain-ai/langchain/security/advisories/GHSA-pjwx-r37v-7724)** โ€” `langchain-core` (CWE-502, CVSS 7.5).

Linked here as evidence that the scaffold produces findings that survive third-party review โ€” not as the headline. Other findings on other targets are still under disclosure and are intentionally not listed.

---

## License

[Apache-2.0](LICENSE)