We give AI a simple job: try to break Didit

At Didit, AI red teaming means using artificial intelligence (AI) agents to challenge our identity verification flows in a staging environment. We study the attempts and their outcomes to find weaknesses, help detect bots performing verification and understand where real users struggle.

We give the agents access to our source code and ask them to find a way through the checks in staging. They try forged documents, injected camera feeds and automated journeys through real workflows. We watch what happens, investigate the results and use what we learn to improve the system.

We also use this work to help detect bots performing verification and understand user behaviour. Seeing how automation moves through a flow helps us ask better questions about both fraud and the experience of the person on the other side of the screen.

Key takeaways

Test the whole journey. Forged documents, artificial camera feeds and automated actions challenge different parts of a verification flow.

Investigate what happened. An agent claiming a bypass needs to be checked against reproducible evidence.

Learn from behaviour. Timing and retries can add context about automation, while also revealing where legitimate users need help.

Make the findings useful. The work continues through investigation, fixes and checks on the effect on real people.

The responsibility behind the screen

TechCrunch reported a suspected breach at a verification provider.

For us, the responsibility starts with what people entrust to a verification company. A passport scan carries your name, photograph and other identifying information. A selfie is a record of your face. You can reset a password after it leaks. You cannot reset your face.

That responsibility reaches beyond deciding whether a document looks genuine or a person is present. It includes the services processing the request, the permissions around the data and the people who operate the system. A strong check at the front of a journey cannot compensate for a weakness elsewhere.

This is why we want to challenge our assumptions while we still have the chance to act on what we find.

What we actually ask the agents to do

A normal test starts with an expected result. An adversarial test starts with a goal the system is supposed to prevent.

For identity verification, that might mean getting a forged document accepted or trying to make a recorded face pass as a person who is present. It might mean probing how the system responds when an automated client moves through a flow in an unexpected way.

The agent can inspect the code, interact with the workflow and adjust its next attempt based on the response. That gives us a useful way to explore assumptions a fixed set of test cases may miss.

In the accompanying footage, agents upload specimen documents, supply artificial camera feeds and work through verification steps in staging. The camera pulls back from one screen to a wall of those recordings so you can see the range of activity.

For background on manipulated media, see our guide to deepfake attacks and detection.

The wall is a visualisation of the work. Each outcome still needs to be checked against the underlying evidence. An agent saying it found a bypass does not make it one.

Our approach includes frontier models and open models with fewer restrictions on the tasks they will attempt. We want useful coverage of adversarial behaviour, including attempts that a model might otherwise refuse to explore. The model's willingness to try an attack says nothing about whether that attack succeeded. That is an engineering question we have to verify.

How the work helps us understand bots

A document and a selfie tell us part of the story. The way a verification happens can add context.

We use these exercises to help detect bots performing verification and to learn more about botting behaviour. Running automation ourselves gives us examples we can study, compare and challenge as the automation changes.

We can examine where an automated attempt pauses, what it repeats after an error and whether it follows the same sequence each time. We can also consider whether that behaviour fits the device and session context. These comparisons help make an investigation more specific.

These are questions to investigate, not rules that establish fraud on their own. A quick user may simply know what to do. A person who retries several times may have poor lighting or an unreliable connection. Someone using assistive technology may interact differently from the pattern a designer expected.

Useful bot detection has to account for that variation. Timing, repetition and device context need to be considered alongside the verification evidence. Treating one unusual action as proof of abuse would make the experience worse for real people.

The same work teaches us where people struggle

Studying behaviour also helps us ask better product questions. A retry can point to an attack, but it can also point to an instruction that was hard to understand. An abandoned camera step might mean the user could not grant permission. A failed capture might reflect glare, a damaged document or a device that needs a different approach.

Automated testing lets us explore how the flow responds to these situations. Understanding what real people experience requires looking at user behaviour too. One cannot substitute for the other.

Together, they help us separate a control that is doing useful work from friction that deserves investigation. The aim is to make abuse harder while helping legitimate users complete verification with clear instructions and a sensible path to recover when something goes wrong.

A finding has to change something

The value of an exercise is what happens after the attempt.

A suspected weakness needs a reproducible case. An engineer needs to understand which control was involved and whether the evidence supports the claim. If a fix is needed, the original attempt needs to be run again against the change.

We also need to check what that change does to legitimate users. A control that rejects more attacks by rejecting more people has created another problem to solve.

That is the discipline we want this work to support: attempt, observe, investigate, improve and test again. The animation shows the attempts. The work behind them is what matters.

We can be hacked too

Anyone can be hacked, including us. Continuous testing does not remove that possibility. It gives us more opportunities to discover a weakness and do something about it.

We build infrastructure for identity and fraud because we believe people should be able to prove who they are online with confidence. Our vision of humanizing the internet in the age of AI depends on earning that confidence in practice.

That means understanding how attackers behave, paying attention to where real users struggle and being honest about what our evidence can tell us.

A call to technical leaders across our industry

Cyber risk is systemic in identity verification. We depend on connected services, integrations and people, and the trust at stake reaches beyond any one provider. A failure at one company can harm individuals and weaken confidence across the industry.

If you are a chief technology officer, an engineering leader or a security leader in this industry, we encourage you to take that responsibility very seriously. Give security work clear owners, time and budget. Challenge the verification flow, the systems that handle identity data, the access people have and the response your team would make to an incident. Revisit those assumptions as the technology and the attacks change.

Last week's reporting should be a wake up call to prepare. We should use it to strengthen our own defences, share useful defensive lessons responsibly and act on verified weaknesses. Waiting until our own company is affected leaves that work too late.

We include ourselves in that responsibility. We can be hacked too, and we need to keep earning the trust people place in us.

Our industry needs to step up. People are trusting us with their identity. We owe them the work.

Frequently asked questions

What does AI red teaming involve at Didit?

We give AI agents access to our source code and a staging environment, then ask them to challenge verification checks. The examples in this article include forged specimen documents, artificial camera feeds and automated journeys. Engineers still need to verify the outcome of each suspected weakness.

How does this help detect bots performing verification?

Running automation ourselves creates examples of bot behaviour to study. It helps us examine sequences, retries and context alongside verification evidence. A single behavioural signal does not establish that a user is a bot or that fraud has occurred.

Does the animation prove that Didit stops every attack?

No. The animation is a composite of staging recordings with repeated screens. It shows the testing process, not a detection benchmark or a promise that every attack will fail.

Can this work improve the experience for real users?

Yes, it can help us investigate where the verification flow becomes difficult. We still need to consider real user behaviour alongside automated testing, so we can distinguish useful security controls from problems such as unclear instructions or difficult camera access.

If you are building a verification journey, explore Didit's User Verification and read the liveness documentation. You can also see the pricing or start free.