AI evaluation & red-teaming

A demo is not evidence that a system works

An agent that answers well in a review meeting has been tested on the cases somebody thought to try. The cases that matter are the ones nobody thought to try, on the day the input is malformed, the contract has been amended, or a value falls just outside the range everyone assumed.

Evaluation is the discipline of finding those cases on purpose, before the system finds them for you in front of a lender or an auditor.

How we test

We build an evaluation set from the work the agent actually has to do, not from generic benchmarks. Real documents, real edge cases, the values that sit at the boundary of a rule, and the inputs that are simply wrong.

Then we run it adversarially: prompt injection through the documents an agent is asked to read, instructions hidden in content it treats as data, values crafted to pass a validation rule while being false, and tool calls the agent should have refused.

Automated verification runs the same set on every change, so a fix to one behaviour cannot quietly regress another. A result that cannot be reproduced is not a result.

What we check beyond accuracy

Whether confidence means anything — an agent that is wrong and certain is more dangerous than one that is wrong and routes to review.

Whether permissions hold when the model is persuaded otherwise. Scoped, fail-closed credentials are a claim until somebody tries to break them.

Whether the evidence chain survives failure. When a step goes wrong, the record should show what the agent saw and which rule it applied, not just that something happened.

What you get

Findings with the input that produced them, ranked by what they would cost if they reached production, each with a remedy rather than an observation.

The evaluation set itself, so your team can keep running it after we leave. This is the same work as an audit of an agent fleet somebody else built, applied earlier — before the system is live rather than after.