Share
## https://sploitus.com/exploit?id=F29A449A-F0E1-5770-81FD-182516C34E23
# PoCXGen Agent

An LLM-orchestrated multi-agent pipeline for automated vulnerability reproduction. Given a bug description and a target codebase, PoCAgent explores the source, analyses the root cause, writes a PoC that exploits a running UAT server over the network, and independently verifies it.

## Topology

The UAT server is an **independent entity** contacted by URL only β€” the pipeline never starts, stops, or restarts it. Each run provisions an analysis network with an OOB callback collector; when GitNexus is enabled, the sandbox also joins a shared bridge network.

```
  UAT Server (operator-managed β€” any source)
  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
  β”‚  pocxgen-agent uat start   β†’  {name}-uat           β”‚
  β”‚  docker run / compose up         β†’  any container        β”‚
  β”‚  remote staging                  β†’  https://...          β”‚
  β”‚  Exposed at: http://host.docker.internal:{port}          β”‚
  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
         β”‚  plain HTTP β€” any URL works
         β–Ό
  β”Œβ”€β”€ poc-net-- (analysis-internal) ──────────────────────┐
  β”‚                                                                       β”‚
  β”‚   β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”  SHARED (optional)        β”‚
  β”‚   β”‚ -sandbox--      β”‚  pocxgen-gitnexus/        β”‚
  β”‚   β”‚   pocxgen-sandbox:latest             │◄──────────────────────    β”‚
  β”‚   β”‚   /workspace ← source bind-mount     β”‚  gitnexus-bridge-{port}   β”‚
  β”‚   β”‚   UAT_BASE_URL =       β”‚  (ENABLE_GITNEXUS=True)   β”‚
  β”‚   β”‚   UAT_OOB_BASE_URL = host:port       β”‚                           β”‚
  β”‚   β”‚   UAT_BROWSER_WS = ws://browser:8080 β”‚ ← only when uses_browser  β”‚
  β”‚   β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜                           β”‚
  β”‚                                                                       β”‚
  β”‚   oob-collector--                                     β”‚
  β”‚   ports: 9999β†’{host_port}   (published β€” UAT can POST callbacks)     β”‚
  β”‚                                                                       β”‚
  β”‚   browser--          (only when uses_browser: true)   β”‚
  β”‚   pocxgen-browser:latest  β€” Playwright/Chromium remote server        β”‚
  β”‚   PoCs connect via BrowserClient.from_env() for DOM-based testing    β”‚
  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
```

The agent runs inside the **sandbox** container; it contacts the UAT only through
`uat_bridge.UATClient` (scope-enforced HTTP). The OOB collector is published to a
random host port so any UAT β€” local container, compose stack, or remote server β€”
can POST SSRF blind-test callbacks to it. When the UAT spec sets `uses_browser: true`,
a `pocxgen-browser` sidecar (Playwright/Chromium) is also provisioned on the same
network; PoCs reach it via `BrowserClient.from_env()` for DOM-based and client-side
vulnerability testing (DOM-XSS, etc.).

See [docs/operator_guide.md](docs/operator_guide.md) for end-to-end onboarding,
[docs/remote_uat.md](docs/remote_uat.md) for the UAT connection model,
[docs/gitnexus.md](docs/gitnexus.md) for the GitNexus knowledge graph,
and [docs/metrics.md](docs/metrics.md) for per-run cost / token / container
accounting and the live monitoring panel.

## Quick start

Choose a UAT server and follow its dedicated setup guide in `docs/uat-servers/`:

| UAT server | Stack | Port | Test cases | Guide |
|---|---|---|---|---|
| `go-test-bench` | Go | 8080 | β€” | [docs/uat-servers/go-test-bench.md](docs/uat-servers/go-test-bench.md) |
| `benchmarkjava` | Maven / Tomcat (Java) | 8443 | ~3000 | [docs/uat-servers/benchmarkjava.md](docs/uat-servers/benchmarkjava.md) |
| `benchmarkpython` | Flask (Python) | 8443 | ~330 | [docs/uat-servers/benchmarkpython.md](docs/uat-servers/benchmarkpython.md) |
| `brokencrystal` | NestJS / Compose | 3000 | β€” | [docs/uat-servers/brokencrystal.md](docs/uat-servers/brokencrystal.md) |

Each guide covers: dependency installation, environment setup, source population, image build,
pre-flight checks, vulnerability extraction, optional GitNexus knowledge graph pre-computation,
and running the agent.

**Prerequisites (all UATs):** Python 3.11+, Docker Desktop/Engine, [`uv`](https://docs.astral.sh/uv/getting-started/installation/).

## How it works

An **LLM orchestrator agent** coordinates four specialist sub-agents by calling them as tools β€” alongside direct investigative tools it can use itself without spawning a sub-agent:

```
AgentOrchestrator (LLM ReAct loop)
  β”œβ”€ execute_bash / read_file / grep_search / glob_search  ← direct tools (lightweight)
  β”œβ”€ run_explorer    β†’ Explorer sub-agent (maps codebase via /workspace)
  β”œβ”€ run_planner     β†’ Planner sub-agent (synthesises plan + Key Facts)
  β”œβ”€ run_developer   β†’ PocDeveloper sub-agent (writes /artifacts/poc.py against the UAT)
  β”œβ”€ run_verificator β†’ Verificator sub-agent (re-runs against the restarted UAT + writes report)
  └─ verify_result / think / finish_orchestration            ← meta tools
```

`project-information.md` (generated by `make scout`) is bind-mounted into the sandbox container at `{work_dir}/project-information.md` β€” each sub-agent reads it as its first step to discover the project's build system, test commands, and directory layout.

Each sub-agent is a ReAct loop (Reasoning + Acting) that calls tools and thinks before acting. Sub-agents emit structured signals in their reports: `BUILD_FAILED`, `POC_FAILED`, `SUCCESS`, `EXHAUSTED` (Developer) and `VERIFIED`, `VERIFICATION_FAILED` (Verificator). The orchestrator LLM reads those signals and decides adaptively whether to re-plan, re-explore, or finish.

The pipeline is **UAT-agnostic**. The vulnerable server image, its source tree, scope policy, and start command are all declared per UAT in `uat_servers//uat.json`. The agent always runs in the same unified `pocxgen-sandbox:latest` image; the per-UAT source tree is bind-mounted at `/workspace` at run time.

## Tools

### Main tool 

PoCAgent provides 9 tools to its agents, composed per-agent based on their role:

| Tool | Description |
|---|---|
| `execute_bash` | Run shell commands in the sandbox container |
| `str_replace_editor` | View, create, and edit files |
| `execute_ipython_cell` | Run Python code with pre-loaded helper functions |
| `read_file` | Read file content with offset/limit (preferred for large files) |
| `write_file` | Write files with correct newline handling (preferred for artifacts) |
| `glob_search` | Find files by glob pattern, sorted by modification time |
| `grep_search` | Regex content search with context lines and file filtering |
| `think` | Log reasoning without side effects |
| `finish` | Signal task completion |

Generated PoCs additionally use `from uat_bridge import UATClient` to reach the UAT over HTTP with built-in scope/rate/destructive-block enforcement and a per-run audit log. For DOM-based vulnerabilities, `from uat_bridge import BrowserClient` drives a remote Playwright/Chromium instance (provisioned automatically when `uses_browser: true` in `uat.json`).

### Callgraph tool (optional)

When `ENABLE_CALLGRAPH_ANALYSIS = True` in `config.py`, Explorer receives a `query_callgraph` tool. The call graph is built **at run startup** inside the sandbox container by Joern or CodeQL β€” no offline setup step required.

| Tool | Description |
|---|---|
| `query_callgraph` | Static call graph traversal β€” find callers/callees of a function or trace the shortest call path between two functions |

See [docs/callgraph.md](docs/callgraph.md) for setup, configuration, and troubleshooting.

### GitNexus graph tools (optional)

When `ENABLE_GITNEXUS = True` in `config.py`, Explorer also receives 4 read-only knowledge graph tools. They call a pre-built graph served by a `gitnexus-bridge` container on the shared `pocxgen-gitnexus` Docker network; the agent's sandbox container is dual-attached to that network and reaches the serve container by docker-DNS name.

| Tool | Description |
|---|---|
| `gitnexus_query` | Hybrid BM25+semantic search β€” find symbols by concept before knowing the exact name |
| `gitnexus_context` | 360-degree symbol view β€” get all callers, callees, and type relationships at once |
| `gitnexus_impact` | Bl                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       ast radius analysis β€” map every user-facing entry point to a vulnerable symbol |
| `gitnexus_cypher` | Raw Cypher queries β€” advanced multi-hop graph traversal with custom filters |

See [docs/gitnexus.md](docs/gitnexus.md) for setup and configuration.

### OSS-Fuzz tools (optional)

When `ENABLE_OSSFUZZ = True` in `config.py`, Explorer and Developer receive 5 in-container tools for dynamic fuzzing-based vulnerability reproduction. The OSS-Fuzz scripts are bind-mounted from `sandbox/_base/ossfuzz/` at run time β€” no offline setup required.

| Tool | Description |
|---|---|
| `ossfuzz_compile` | Compile a fuzz target with sanitizer instrumentation (AddressSanitizer, etc.) |
| `ossfuzz_fuzz` | Run the fuzzer for crash discovery |
| `ossfuzz_reproduce` | Reproduce a known crash from a saved corpus entry |
| `ossfuzz_coverage` | Measure code coverage for the instrumented target |
| `ossfuzz_query_project` | Query the public Fuzz Introspector API for existing OSS-Fuzz coverage data |

See [docs/ossfuzz.md](docs/ossfuzz.md) for setup, configuration, and troubleshooting.

## Repository structure

```
pocxgen-agent/
β”œβ”€β”€ Makefile                         Build, run, dev, and evaluation shortcuts β€” run `make help` for full list
β”œβ”€β”€ compose.dev.yml                  Dev-container compose file (see Development below)
β”œβ”€β”€ sandbox/                         THE one unified analysis image (built once, shared)
β”‚   β”œβ”€β”€ Dockerfile                   Python + Joern + CodeQL + language toolchains + uat_bridge deps
β”‚   β”œβ”€β”€ README.md
β”‚   └── _base/                       callgraph + ossfuzz scripts (bind-mounted at runtime)
β”‚
β”œβ”€β”€ uat_servers/                     One subdir per customer UAT β€” fully self-contained
β”‚   └── /
β”‚       β”œβ”€β”€ uat.json                 image source, start_command, port, healthcheck, env, scope
β”‚       β”œβ”€β”€ Dockerfile.uat           only when image_source.kind == "dockerfile"
β”‚       β”œβ”€β”€ compose.yml              only when image_source.kind == "compose"
β”‚       β”œβ”€β”€ source_code/             customer source tree, bind-mounted at /workspace (gitignored)
β”‚       β”œβ”€β”€ README.md                per-customer notes (sidecars, EULA, source provenance)
β”‚       └── .customer-artifacts/     gitignored: tar files, cert bundles, secrets material
β”‚
β”œβ”€β”€ docker/
β”‚   β”œβ”€β”€ gitnexus-bridge/             Dockerfile + helpers for the gitnexus-bridge image (Node.js + gitnexus npm)
β”‚   β”œβ”€β”€ browser/                     Playwright/Chromium sidecar (built when uses_browser: true in uat.json)
β”‚   β”œβ”€β”€ oob-collector/               OOB callback collector container (Python; published to a random host port)
β”‚   └── redai/                       Dockerfile for the RedAI baseline image
β”‚
β”œβ”€β”€ src/
β”‚   β”œβ”€β”€ scouter/                     Standalone project discovery tool
β”‚   β”‚   └── __main__.py              scout CLI β€” generates uat_servers//project-information.md
β”‚   β”œβ”€β”€ endpoint_scout/              OWASP Noir-based API endpoint extractor
β”‚   β”‚   β”œβ”€β”€ __main__.py              endpoints CLI β€” runs Noir inside sandbox, writes endpoints.json
β”‚   β”‚   β”œβ”€β”€ models.py                Endpoint data models
β”‚   β”‚   └── noir_runner.py           Noir subprocess lifecycle
β”‚   β”œβ”€β”€ gitnexus_bridge/             Optional GitNexus knowledge graph integration
β”‚   β”‚   β”œβ”€β”€ __main__.py              gitnexus-bridge CLI (analyze, serve, status)
β”‚   β”‚   β”œβ”€β”€ server.py                serve container lifecycle (joins pocxgen-gitnexus network)
β”‚   β”‚   β”œβ”€β”€ client.py                MCP HTTP client (GITNEXUS_BASE_URL env-driven)
β”‚   β”‚   β”œβ”€β”€ extractor.py             Symbol extraction helpers
β”‚   β”‚   β”œβ”€β”€ tool_executors.py        Tool call executors for graph queries
β”‚   β”‚   └── tools.py                 Tool definitions for the bridge
β”‚   β”œβ”€β”€ redai/                       RedAI baseline agent (TypeScript/Bun)
β”‚   └── pocxgen_agent/               Python package
β”‚       β”œβ”€β”€ __main__.py              CLI dispatcher: run / scout / endpoints / sandbox / uat / extract / benchmark / check
β”‚       β”œβ”€β”€ instance_runner.py       Single-instance pipeline: resolve URL, OOB + network, analysis, teardown
β”‚       β”œβ”€β”€ benchmarks/              OWASP Benchmark scoring (owasp_benchmark_scorer.py)
β”‚       β”œβ”€β”€ metrics/                 Per-run metrics layer (batch, docker_sampler, estimator, llm_hook, watchdog …)
β”‚       β”œβ”€β”€ uat_bridge/              HTTP + browser clients PoCs use to reach the UAT
β”‚       β”‚   β”œβ”€β”€ client.py            UATClient β€” scope/rate/audit-enforced HTTP
β”‚       β”‚   β”œβ”€β”€ browser.py           BrowserClient β€” Playwright remote server for DOM-based PoCs
β”‚       β”‚   β”œβ”€β”€ oob.py               OOB callback poller
β”‚       β”‚   β”œβ”€β”€ scope.py             ScopeMatcher
β”‚       β”‚   β”œβ”€β”€ auth.py / credentials.py / cookies.py / audit.py
β”‚       └── core/
β”‚           β”œβ”€β”€ agents/              Orchestrator + 5 sub-agents (explorer/planner/developer/verificator/project_info)
β”‚           β”œβ”€β”€ prompts/             Jinja2 system-prompt templates + per-bug-type fragments
β”‚           β”œβ”€β”€ orchestrator_tools/  Modular orchestrator tool implementations (direct, subagents, output, context)
β”‚           β”œβ”€β”€ config.py            Centralised configuration (AgentConfig)
β”‚           β”œβ”€β”€ sandbox_environment.py  Network + OOB + browser sidecar lifecycle
β”‚           β”œβ”€β”€ uat_lifecycle.py     UAT container detection and host-port resolution
β”‚           β”œβ”€β”€ docker_session.py    Persistent bash inside the sandbox container; bind-mounts source
β”‚           β”œβ”€β”€ image_resolver.py    Pull / load / tag / digest-verify a UAT image per kind
β”‚           β”œβ”€β”€ key_facts.py         Key Facts accumulator (shared state across sub-agents)
β”‚           β”œβ”€β”€ orchestrator_state.py  Orchestrator run state dataclass
β”‚           β”œβ”€β”€ session.py           Conversation session manager
β”‚           β”œβ”€β”€ skills.py            Optional skill injection (endpoint scout, etc.)
β”‚           β”œβ”€β”€ tool_registry.py / tools.py / hooks.py / provider.py / compaction.py / api_keys.py / …
β”‚
β”œβ”€β”€ scripts/                         Evaluation and utility scripts
β”‚   β”œβ”€β”€ evaluate-poc.py              Score a single PoC output directory
β”‚   β”œβ”€β”€ evaluate-all-pocs.sh         Batch evaluation over an output directory
β”‚   β”œβ”€β”€ overview.py                  Print a summary table of a batch run
β”‚   β”œβ”€β”€ chain_vulns.py               Chain multiple VULN JSONs into a single run
β”‚   └── regen-replay.py              Replay and regenerate a prior run from saved messages
β”‚
β”œβ”€β”€ input/extracted_vuls/            Vulnerability instance JSON files (one folder per UAT, VULN-*.json)
β”œβ”€β”€ docs/
β”‚   β”œβ”€β”€ operator_guide.md            Adding a new customer UAT (the 7-step workflow)
β”‚   β”œβ”€β”€ remote_uat.md                UAT integration model + safety rails
β”‚   β”œβ”€β”€ gitnexus.md                  GitNexus setup + architecture
β”‚   β”œβ”€β”€ ossfuzz.md                   OSS-Fuzz dynamic analysis integration
β”‚   β”œβ”€β”€ templates/uat.json           Copy-paste starter for new UATs
β”‚   └── …
β”œβ”€β”€ .env.example                     Environment variable template β€” copy to .env and fill in
β”œβ”€β”€ pocxgen-agent-config.json        Optional JSON overlay over config.py constants (loaded via --config)
β”œβ”€β”€ pyproject.toml                   Python project metadata + dependencies
└── CONTRIBUTING.md
```

## CLI reference

After `uv sync` and activating `.venv`, `pocxgen-agent` is available as a command:

| Subcommand | Purpose |
|---|---|
| `pocxgen-agent sandbox build` | Build `pocxgen-sandbox:latest` + `pocxgen-oob-collector:latest` + `pocxgen-browser:latest` |
| `pocxgen-agent sandbox info` | Show the unified image's tag, build date, size |
| `pocxgen-agent uat list` | List every `uat_servers//` with image + source + scope status |
| `pocxgen-agent uat resolve ` | Pull / load / build / verify a UAT's image per `image_source.kind` |
| `pocxgen-agent uat start ` | Start the UAT container and print its URL |
| `pocxgen-agent uat stop ` | Stop the UAT container |
| `pocxgen-agent uat container-url ` | Print the host-accessible URL for a running UAT container (exits non-zero if not running) |
| `pocxgen-agent uat network-name ` | Print the docker network name for a running UAT instance |
| `pocxgen-agent scout --uat-server  [--report PATH]` | Generate `uat_servers//project-information.md`; optionally extract vulns first via `--report` |
| `pocxgen-agent endpoints --uat-server ` | Extract API endpoints from the UAT source tree using OWASP Noir; writes `endpoints.json` to `uat_servers//` |
| `pocxgen-agent run --uat-url  --uat-server  --instances-dir …` | Run the agent β€” `--uat-url` is required; `--uat-server` loads spec (scope, auth, source path) |
| `pocxgen-agent extract --report … --output-dir …` | Extract vulnerability instances from a SAST Markdown report (standalone) |
| `pocxgen-agent benchmark score --uat-server  --run-dir ` | Score a completed run against labelled benchmark expected results (OWASP Benchmark family) |
| `pocxgen-agent check` | Verify Docker daemon, images, LLM config, UAT servers |

The root `Makefile` provides shortcuts for all common workflows. Run `make help` to list all targets (includes `build`, `check`, `uat-start`, `uat-stop`, `run`, `endpoints`, `dev-*`, `redai-*`).

> **Models are set in `config.py`**, not via a per-agent CLI flag. Edit the `MODEL_*` constants at the top of `src/pocxgen_agent/core/config.py`, or pass an overlay JSON file via `--config PATH` (see [Configuration](#configuration)).

> **Root `Dockerfile`**: for CI/production deployments where you want a self-contained image without installing Python locally. Day-to-day development just needs `uv sync`.

## Adding a new customer UAT

The full operator walk-through is in [docs/operator_guide.md](docs/operator_guide.md). Short version:

```bash
# 1. Create the folder and populate source
mkdir -p uat_servers//source_code
# … git clone / tar -xf the customer's source into source_code/

# 2. Describe the UAT
cp docs/templates/uat.json uat_servers//uat.json
# Edit: image_source, port, healthcheck, env, scope_allowlist, …

# 3. (Private registry only) Add credentials to .env
#    matching uat.json's image_source.pull_secret_env

# 4. Build the UAT image
pocxgen-agent uat resolve 

# 5. Extract vulns + generate project-information.md
pocxgen-agent scout --uat-server  --report input/sast_reports/report.md

# 6. Start the UAT, then run (all targets read UAT_SERVER + UAT_URL from .env)
#    Every UAT publishes to the port in UAT_HOST_PORT (default 9090); flip http→https in .env for TLS UATs.
make uat-start
make run INSTANCES_DIR=input/extracted_vuls//
make uat-stop

# Or target any URL directly (staging server, existing container, …)
# make run UAT_URL=https://staging.example.com INSTANCES_DIR=input/extracted_vuls//
```

`image_source.kind` chooses how the UAT image is obtained:

| `kind` | Use when | Required fields |
|---|---|---|
| `dockerfile` | We build the UAT image from a `Dockerfile.uat` (e.g. the go-test-bench pilot). | (none beyond `Dockerfile.uat` next to `uat.json`) |
| `registry` | Customer publishes the UAT to a Docker registry. | `image` (digest-pin recommended), optional `pull_secret_env` |
| `tar` | Customer hands over a `docker save`-ed tar (air-gapped). | `path` to the tar, relative to the UAT directory |
| `local` | The image is already loaded on the operator's host. | `image` |
| `compose` | UAT is multi-container (app + DB + cache). Builds from local `source_code/` by default (`docker compose build`). Use `--pull` to fetch the pre-built registry image instead. | `compose_file` |

Customer UATs **must** declare a non-empty `scope_allowlist` in `uat.json` β€” the runner enforces this at validation time so generated PoCs cannot accidentally hit out-of-scope endpoints.

## Development container

A pre-configured dev container is provided for contributors who prefer not to install Python or `uv` locally. It mounts the repo into a container with all dependencies pre-installed.

```bash
# First-time: build the dev image
make dev-build

# Start an interactive shell
make dev-up          # or: make dev-shell

# Inside the container β€” build sandbox + gitnexus images once
make dev-init

# Teardown (removes container + named venv volume)
make dev-down
```

The compose file is `compose.dev.yml`. Docker socket is passed through so the dev container can build and manage sibling containers (sandbox, UAT, OOB collector) on the host.

## Configuration

PoCAgent's defaults live as `UPPER_CASE` constants in [`src/pocxgen_agent/core/config.py`](src/pocxgen_agent/core/config.py). Edit that file to change models, iteration limits, the unified sandbox image tag, hooks, etc.

For per-run overrides without editing source, pass a JSON file via the top-level `--config PATH` flag. Any field present in the file overrides the corresponding constant; missing fields keep their default value:

```bash
pocxgen-agent --config pocxgen-agent-config.json run \
  --instances-dir input/extracted_vuls/go-test-bench \
  --uat-server go-test-bench
```

Example overlay file:

```json
{
  "model_orchestrator": "gemini/gemini-2.5-flash",
  "model_explorer": "gemini/gemini-2.5-flash",
  "model_planner": "gemini/gemini-2.5-pro",
  "model_developer": "openrouter/qwen/qwen3-235b-a22b",
  "model_verificator": "gemini/gemini-2.5-flash",
  "model_project_info": "gemini/gemini-2.5-flash",
  "model_condenser": "gemini/gemini-2.0-flash-lite",
  "thinking_budget": 8000,
  "max_orchestrator_iterations": 25,
  "max_invocations_per_agent": {"explorer": 2, "planner": 3, "developer": 2, "verificator": 1, "verify_result": 6},
  "max_total_tokens": 2000000,
  "max_iterations": {
    "explorer": 50,
    "planner": 25,
    "poc_developer": 60,
    "verificator": 30,
    "project_info": 25
  }
}
```

Resolution order (later layers override earlier): defaults from `config.py` β†’ JSON file passed to `--config` β†’ CLI flags on the `run` subcommand. See [`docs/configuration.md`](docs/configuration.md) for the full parameter reference.

## Supported models

Any [LiteLLM-compatible](https://docs.litellm.ai/docs/providers) model can be used for the main agents. The memory condenser uses the same model as the main agent by default. Set `condense_model` in your config to use a cheaper model (e.g. `gemini/gemini-2.0-flash-lite`) for condensation.

| Prefix | Key required |
|---|---|
| `gemini/` | `GEMINI_API_KEY` |
| `vertex_ai/` | `GEMINI_API_KEY` |
| `openrouter/` | `OPENROUTER_API_KEY` |

## Cost tracking

PoCAgent tracks token usage per agent and estimates costs. Token usage is reported with the full 5-field schema matching the Claude Agent SDK:

| Field | Meaning |
|---|---|
| `input_tokens` | Non-cached prompt tokens |
| `output_tokens` | Generated completion tokens |
| `cache_creation_input_tokens` | Expensive one-time prompt-cache writes |
| `cache_read_input_tokens` | Cheap cache hits (~10% of input cost) |
| `cached_tokens` | Alias: `cache_creation + cache_read` (backcompat) |

After a run, cost breakdowns are included in `metadata.json` and printed to the log:

```
## Token Usage & Cost Report
- explorer: 60,000 in / 8,000 out (cache: 5k creation + 40k read) β€” $0.0135
- planner: 30,000 in / 4,000 out (cache: 0 creation + 20k read) β€” $0.0068
- poc_developer: 160,000 in / 18,000 out (cache: 0 creation + 80k read) β€” $0.0360
**Total**: 250,000 in / 30,000 out β€” **$0.0555**
```

The metrics layer additionally writes a richer per-run summary alongside
`metadata.json`:

- `output/_/_/metrics.json` β€” totals, per-agent rollup,
  container peak CPU / MEM / NET, phase timings, watchdog events
- `output/_/_/metrics.jsonl` β€” time-series stream
- `output/_/batch_metrics.json` β€” per-batch aggregates (avg / p50 /
  p95 / total) and the projected-batch-total used by the live panel

A live Rich panel shows running cost, in-flight projection, container
resources, and watchdog warnings while the run is in progress. See
[docs/metrics.md](docs/metrics.md) for the full reference.

## Testing

```bash
# Run all tests
python -m pytest tests/ -v

# Run specific test file
python -m pytest tests/test_hooks.py -v
```

## Environment variables

| Variable | Required | Purpose |
|---|---|---|
| `GEMINI_API_KEY` | If using `gemini/` or `vertex_ai/` | Main model API key for Gemini / Vertex AI |
| `OPENROUTER_API_KEY` | If using `openrouter/` | Main model API key |
| `UAT_SERVER` | No (default: `go-test-bench`) | Default UAT server for `make` targets |
| `UAT_HOST_PORT` | No (default: `9090`) | Host port published by `pocxgen-agent uat start` and `brokencrystal/compose.yml`. Override if 9090 is taken. Must match UAT_URL port. |
| `GITNEXUS_PORT` | No (default: `4747`) | Host port published by `gitnexus-bridge serve`. Override if 4747 is taken. Changing requires re-`serve` (docker-DNS name embeds port). |
| `` | If a UAT uses a private registry | Per-UAT credential, named by `image_source.pull_secret_env` in `uat.json` |
| `HF_TOKEN` | No | HuggingFace token (if needed by a benchmark) |
| `DEBUG_TRACE` | No | Set to any value to enable debug logging |
| `POCXGEN_METRICS` | No (default: on) | Set to `0` to disable the per-run + per-batch metrics layer entirely. |
| `POCXGEN_NO_RICH` | No | Set to `1` to disable the Rich live console panel (metrics still record to disk). |

For the full list of `POCXGEN_*` runtime toggles (including GC tunables and
`STRICT_THINKING`), see [docs/configuration.md](docs/configuration.md).

The pipeline injects `UAT_*` env vars (base URL, scope, rate limit, audit log path, …) into the sandbox container automatically; you don't set these from the host. When GitNexus is enabled, `GITNEXUS_BASE_URL` is also injected. When the UAT spec sets `uses_browser: true`, `UAT_BROWSER_WS` (WebSocket URL to the Playwright remote server) is injected for use by `BrowserClient.from_env()`.

> Other configuration knobs (models, iteration caps, token budget, etc.) are set in
> [`config.py`](src/pocxgen_agent/core/config.py) or via the `--config PATH` overlay file.
> There are no `POCXGEN_AGENT_*` environment-variable overrides.

## RedAI Baseline

RedAI is an HTTP-agent-based validation tool that serves as a **baseline** for comparing against pocxgen_agent on the same VULN inputs. It runs in Docker, accepts the same 7-field VULN JSON format, and writes output to the same `output/` folder.

```bash
# 1. Build the RedAI image once
make redai-build

# 2. Run on a VULN file β€” UAT server starts automatically
make redai-run VULN=VULN-004 UAT_SERVER=go-test-bench
# β†’ output/redai_go-test-bench_/redai_VULN-004/metadata.json

# Stop the UAT server when done
make uat-stop UAT_SERVER=go-test-bench
```

Output `metadata.json` uses the same `success` / `eval_result` / `orchestrator_message` fields as pocxgen_agent, so existing benchmark tooling works unchanged.

See [docs/redai-baseline.md](docs/redai-baseline.md) for setup, full CLI reference, VULN→Finding field mapping, and comparison commands.

## Further reading

- [docs/redai-baseline.md](docs/redai-baseline.md) β€” RedAI container baseline (setup + usage)
- [docs/operator_guide.md](docs/operator_guide.md) β€” Add a new customer UAT (7-step workflow)
- [docs/remote_uat.md](docs/remote_uat.md) β€” UAT integration model + safety rails
- [docs/gitnexus.md](docs/gitnexus.md) β€” GitNexus knowledge graph (analyze/serve, shared network)
- [docs/callgraph.md](docs/callgraph.md) β€” Joern / CodeQL callgraph tool
- [docs/ossfuzz.md](docs/ossfuzz.md) β€” OSS-Fuzz dynamic analysis integration (sanitizers, fuzzing, coverage)
- [docs/orchestrator.md](docs/orchestrator.md) β€” `AgentOrchestrator` reference
- [docs/memory-management.md](docs/memory-management.md) β€” Context condensation, 5-field token tracking
- [docs/tools.md](docs/tools.md) β€” Complete tool reference
- [docs/prompts.md](docs/prompts.md) β€” Prompt engineering guide
- [docs/configuration.md](docs/configuration.md) β€” Full configuration reference
- [docs/providers.md](docs/providers.md) β€” Provider reference: supported models, thinking dispatch
- [docs/hooks.md](docs/hooks.md) β€” Hook system guide
- [docs/benchmarks.md](docs/benchmarks.md) β€” Benchmark integration guide
- [CONTRIBUTING.md](CONTRIBUTING.md) β€” Development setup and contribution guide