Practice 02 / AI safety / For organizations adopting or building AI
Test your AI before you trust it.
AI is going into your business faster than anyone is checking it. Agents that send email, change records and spend money. Models that answer customers. We put them under pressure first, so the surprise happens in our lab and not in the news.
- Who it’s for
- Executives deciding how far to let AI into the business. Teams that build, fine-tune or ship AI models and agents. The boards, investors and regulators who have to trust the answer.
Sound familiar?
- “We've approved AI across the company and nobody has seriously tried to make it misbehave”
- “An AI agent can act on its own in our systems and no one has checked what happens when it's tricked”
- “Every release is tested by hand, differently each time. Nobody can say whether the new version is safer than the old one”
- “A customer, regulator or investor asked how we know our AI is safe. The honest answer was a shrug”
How it goes wrong
Model failures aren't evenly spread. A system that behaves on a million ordinary requests fails badly on the one request someone crafted, and that is the one that ends up in the news. Averages hide the tail. We build the tests that find it, and we run them again every time something changes.
What we do
Before AI is everywhere in your business, we test it. You find out what it does under pressure, not what the vendor promised.
- 01
Find what's there
Every AI tool, model and agent your business relies on, who uses it, what data goes in, and what it's allowed to do.
- 02
Put it under pressure
Red-teaming and simulations: jailbreaks, hidden-instruction attacks, data leaks, and what an agent does when it's given tools, money or access and then lied to.
- 03
Build the harness
Repeatable, automated evaluations that run on every version, so safety is measured the same way each time and you can see the trend.
- 04
Set the gate
Pass/fail criteria agreed before anything ships, a written record of the results, and a plan for the day a test fails.
What you get
- A list of your AI systems and what each is allowed to do
- A red-team report with every finding reproduced
- An evaluation harness your team keeps running
- Release criteria your leaders sign, and a safety summary you can show regulators and investors
Standards and tools
We draw on the published methods of the AI labs and standards bodies, and adapt them to your systems and your users.
- NIST AI RMF
- Measuring and managing AI risk
- MITRE ATLAS
- Known attacks on AI systems
- OWASP LLM & Agentic Top 10
- The most common ways AI apps and agents get attacked
- ISO 42001
- Managing AI responsibly, end to end
- EU AI Act
- What high-risk AI will have to prove in Europe
The rest of the practice
- 01
Risk advisory
For boards, executives and governments
Where you really stand, what it could cost, and what to do next. Written for the people who decide.
- 03
Research
Behind both practices
We watch the ground under the other two: what's moving, who's moving it, and what it means for you. Most of it isn't published. Some is.
Start here
Rolling out AI, or shipping it? Let's test it first.
Tell us what the AI does, who it touches, and when it goes live. We'll tell you plainly what we'd test first.
We’ll reply in writing: what we’d look at first, and a quote.
