Webhook Idempotency: Building Reliable Identity Verification Workflows

Webhook idempotency ensures that processing a webhook multiple times, whether due to retries or network glitches, produces the same result as processing it once, preventing unintended side effects like duplicate identity checks or inconsistent user states.

Why Webhook Idempotency Matters in Identity Verification

Identity verification processes, by their nature, involve critical data and often trigger subsequent actions like account activation, risk scoring, or transaction approvals. In such sensitive workflows, the consequences of duplicate processing can range from minor inefficiencies to significant financial losses or compliance breaches. Imagine a scenario where a user_verified webhook is sent twice due to a transient network error on the receiving end, leading to two separate account activations or, worse, two identical identity checks being initiated and paid for.

This is where webhook idempotency becomes indispensable. By designing your webhook handlers to be idempotent, you guarantee that even if a webhook is received and processed multiple times, the underlying system state changes only once, as intended.

The Core Concept of Idempotency

In mathematics and computer science, an operation is idempotent if applying it multiple times produces the same result as applying it once. For webhooks, this means:

• No duplicate side effects: A payment is processed only once, a user's status is updated only once, an identity check is initiated only once.
• Consistent state: The system's state remains consistent, even if messages are re-delivered.
• Resilience to failures: Your system can tolerate network issues, timeouts, and retries without corrupting data or performing redundant actions.

Implementing Webhook Idempotency

The most common approach to implement webhook idempotency involves using a unique identifier, often called an idempotency key, for each incoming webhook.

1. The Idempotency Key

When a webhook is sent, the sender (e.g., Didit) includes a unique identifier in the request headers or body. This could be a Webhook-Id or X-Didit-Request-Id. This key should be unique for every attempt to deliver a specific webhook event.

2. Storing and Checking the Key

Upon receiving a webhook, your handler should perform the following steps:

1. Extract the idempotency key: Retrieve the unique identifier from the incoming request.
2. Check a persistent store: Query a database (e.g., Redis, PostgreSQL, DynamoDB) to see if this idempotency key has been processed before. This store should be highly available and fast.
3. Conditional processing:

• If the key is found (meaning the webhook has been processed before), immediately return a success response (e.g., HTTP 200 OK) without re-executing the core logic. You might return the result of the previous successful processing if applicable.
• If the key is not found, proceed to process the webhook's payload. As part of this processing, store the idempotency key in your persistent store, marking it as processed. This step must be atomic with the core logic or handled carefully to prevent race conditions.

Example Idempotency Logic (Pseudocode):

def webhook_handler(request):
    idempotency_key = request.headers.get('X-Didit-Request-Id')
    if not idempotency_key:
        return HttpResponseBadRequest('Missing X-Didit-Request-Id header')

    # Check if this key has been processed
    if is_key_processed(idempotency_key):
        # Optionally, retrieve and return the previous result
        return HttpResponse(status=200, content='Already processed')

    try:
        # Process the webhook payload (e.g., update user status, trigger KYC (Know Your Customer))
        process_identity_event(request.json)
        
        # Mark the key as processed *after* successful processing
        mark_key_as_processed(idempotency_key)
        return HttpResponse(status=200, content='Processed successfully')
    except Exception as e:
        # Handle errors, potentially log and retry later
        return HttpResponseServerError(f'Error processing webhook: {e}')

Considerations for Idempotency Key Storage:

• Expiration: Idempotency keys don't need to live forever. After a certain period (e.g., 24 hours to a few days, depending on your retry policies), you can safely expire them.
• Atomicity: The act of checking the key and storing it (or marking it as in-progress) should ideally be atomic to prevent race conditions where two concurrent requests for the same key might both proceed to process the core logic.
• Distributed Systems: In a distributed environment, ensuring that all instances of your webhook handler share the same idempotency store is critical.

Webhooks in Didit's Infrastructure for Identity and Fraud

Didit's infrastructure relies heavily on webhooks to communicate the results of identity verification (User Verification / KYC, Business Verification / KYB (Know Your Business)) and fraud checks (Transaction Monitoring, Wallet Screening / KYT (Know Your Transaction)) back to your systems. For instance, when a User Verification check completes, Didit dispatches a webhook to your configured endpoint, informing you of the outcome (approved, rejected, pending).

Given the critical nature of these events – determining whether a user can onboard, a business can transact, or a payment is safe – ensuring that your system processes these updates reliably and only once is paramount. Implementing webhook idempotency on your end means that even if a Didit webhook is re-delivered due to network congestion or an intermittent issue on your server, your application will correctly interpret it as a single event, preventing duplicate actions like:

• Accidentally activating a user's account twice.
• Triggering redundant internal notifications or workflows.
• Incurring unnecessary costs by re-initiating a check if your system mistakenly thought the first attempt failed.

By leveraging the idempotency keys provided in Didit's webhook headers, you can build truly resilient and trustworthy identity verification workflows that maintain data integrity and optimize resource usage.

Key Takeaways

• Webhook idempotency ensures that repeated processing of a webhook has the same effect as processing it once.
• It is critical for reliable identity verification workflows to prevent duplicate actions and maintain data consistency.
• Idempotency keys (unique identifiers provided by the sender) are fundamental to implementing idempotency.
• Your webhook handler should check and store these keys in a persistent, shared store before processing the core logic.
• Implementing idempotency safeguards against network issues, retries, and system failures without corrupting data.
• Didit's webhooks include idempotency keys to facilitate reliable integration with your systems.

Frequently Asked Questions

Q: What happens if I don't implement webhook idempotency?

A: Without idempotency, your system might process the same webhook multiple times, leading to duplicate actions, inconsistent data, and potential errors, especially during network issues or retries.

Q: Can I use the webhook payload as an idempotency key?

A: While technically possible (e.g., hashing the payload), it's generally better to rely on a dedicated, unique idempotency key provided by the webhook sender. This ensures consistency even if minor, non-essential parts of the payload change or if the payload is too large.

Q: How long should I store idempotency keys?

A: The storage duration depends on your webhook retry policies. A common practice is to store them for 24 to 72 hours, covering most retry windows. After this period, you can safely expire old keys.

Q: Does Didit handle idempotency on its side when sending webhooks?

A: Didit ensures that each event has a unique identifier, and our systems are designed to retry webhook deliveries. It is your responsibility, as the receiver, to implement idempotency in your handler to correctly manage these retries and prevent duplicate processing on your end.

Building reliable systems requires careful attention to potential failure modes. By embracing webhook idempotency, you can ensure that your identity verification and fraud prevention workflows are reliable and resilient. Didit provides the infrastructure for identity and fraud, offering one API with 1,000+ data sources and an open marketplace of modules. Our public pay-per-use pricing, with no minimums, includes 500 free checks every month, and a full identity verification starts from $0.30. Integrate in 5 minutes and build with confidence.

Get started with Didit

Didit is infrastructure for identity and fraud — one API, public pay-per-use pricing, and 500 free verifications every month. Add User Verification to your flow and integrate in 5 minutes.

• User Verification — see how it works and what it costs.
• Read the documentation — API reference and integration guide.
• Start free — 500 verifications every month, no credit card required.