RedCell

Tool-using LLM agents in fintech can be talked into leaking data or calling tools they shouldn't. I wanted to measure that, not guess.

• A LangGraph supervisor that sends attack agents (prompt injection, policy evasion) against a fintech-style agent whose tools sit behind a FastMCP server.
• A deterministic verifier that confirms each breach from a server-recorded trace: leaked canaries, unauthorised tool calls, PII and schema checks. No LLM decides what counts as a breach.
• Five layers of defences, a frozen corpus of 42 attacks, and a CI gate that fails if a blocked attack gets through again.
• A results dashboard in Next.js 15, built entirely from React Server Components.

An LLM judge can be fooled by the same obfuscation the attacks use, so verdicts come from deterministic checks instead. Measured against the verifier, an LLM judge agreed only moderately (Cohen's kappa about 0.68).

Attack success fell from 59.5% to 0% on the frozen replay, and to 31% under adaptive attack. Structural controls held; probabilistic filters fell to encoding tricks.