Mastering Didit API Error Handling & Retries in Go

Understand Didit's Rate LimitsImplement client-side throttling and exponential backoff for 429 responses, utilizing X-RateLimit-Limit, X-RateLimit-Remaining, and X-RateLimit-Reset headers to prevent service disruption.

Differentiate Error TypesDistinguish between transient (e.g., network issues, service unavailability) and permanent errors (e.g., invalid API keys, malformed requests) to apply appropriate retry logic or fail fast.

Implement Robust Retry MechanismsEmploy structured retry loops with exponential backoff, jitter, and maximum retry attempts to handle transient failures gracefully, improving the reliability of critical operations like session creation.

Didit's Developer-First Approach Simplifies IntegrationDidit provides clear API documentation, specific rate-limit headers, and a modular architecture, enabling Go developers to build resilient identity verification flows with ease and confidence.

Integrating third-party APIs is a cornerstone of modern application development, and identity verification is no exception. When working with critical services like Didit's identity platform, ensuring your integration is resilient to network fluctuations, service hiccups, and rate limits is paramount. This guide delves into advanced error handling and retry strategies specifically tailored for Didit API integrations using Go, helping you build robust and reliable systems.

Understanding Didit's API Error Landscape

Before implementing any retry logic, it's crucial to understand the types of errors you might encounter. Didit's API, like many well-designed services, communicates various states through HTTP status codes. While standard 2xx codes indicate success, you'll primarily focus on 4xx (client errors) and 5xx (server errors), and especially 429 (Too Many Requests).

Didit's Rate Limiting and What It Means for You

Didit enforces rate limiting to maintain API stability. This is a critical aspect to manage in your integration. For example, GET endpoints typically have a global limit of 300 requests per minute per application, with specific endpoints like session-decision (100 rpm) or session-v2-create (600 rpm) having their own, potentially stricter, limits. When you hit a rate limit, Didit responds with a 429 Too Many Requests status code and includes helpful headers:

• X-RateLimit-Limit: The maximum number of requests allowed in the current window.
• X-RateLimit-Remaining: The number of requests remaining in the current window.
• X-RateLimit-Reset: The time (in epoch seconds) when the current rate limit window resets.
• Retry-After: (Often included) The number of seconds to wait before making another request.

Your Go client should actively monitor these headers. When X-RateLimit-Remaining drops below a certain threshold (e.g., 15% of X-RateLimit-Limit), you should proactively throttle your requests. Ignoring these can lead to sustained 429 errors, impacting your application's ability to create verification sessions or retrieve results from products like Didit's ID Verification or AML Screening.

Implementing Robust Retry Strategies with Exponential Backoff in Go

For transient errors (e.g., network timeouts, temporary service unavailability, or 429s), retries are essential. However, naive retries can exacerbate problems. The gold standard is exponential backoff with jitter.

Exponential Backoff with Jitter

Exponential backoff means increasing the wait time between retries exponentially. Jitter (adding a small random delay) prevents a 'thundering herd' problem where many clients retry simultaneously after an outage, overwhelming the service again. Here's a conceptual Go example:

package main

import (
	"fmt"
	"io/ioutil"
	"net/http"
	"time"
	"math/rand"
)

func callDiditAPIWithRetry(url string, maxRetries int) ([]byte, error) {
	client := &http.Client{Timeout: 10 * time.Second}
	rand.Seed(time.Now().UnixNano())

	for i := 0; i <= maxRetries; i++ {
		resp, err := client.Get(url)
		if err != nil {
			// Handle network errors (e.g., connection refused, timeout)
			fmt.Printf("Attempt %d: Network error: %v\n", i+1, err)
			if i == maxRetries {
				return nil, fmt.Errorf("failed after %d retries: %w", maxRetries, err)
			}
			sleepTime := time.Duration(1<<i)*time.Second + time.Duration(rand.Intn(1000))*time.Millisecond // Exponential backoff + jitter
			fmt.Printf("Retrying in %v...\n", sleepTime)
			time.Sleep(sleepTime)
			continue
		}
		defer resp.Body.Close()

		switch resp.StatusCode {
		case http.StatusOK:
			fmt.Printf("Attempt %d: Success!\n", i+1)
			return ioutil.ReadAll(resp.Body)
		case http.StatusTooManyRequests:
			// Respect Retry-After header if present
			retryAfter := resp.Header.Get("Retry-After")
			if retryAfter != "" {
				if sleepSeconds, err := time.ParseDuration(retryAfter + "s"); err == nil {
					fmt.Printf("Attempt %d: Rate limited. Retrying after %v.\n", i+1, sleepSeconds)
					time.Sleep(sleepSeconds)
					continue
				}
			}
			// Fallback to exponential backoff if Retry-After is missing or invalid
			fmt.Printf("Attempt %d: Rate limited (429).\n", i+1)
			if i == maxRetries {
				return nil, fmt.Errorf("rate limited after %d retries", maxRetries)
			}
			sleepTime := time.Duration(1<<i)*time.Second + time.Duration(rand.Intn(1000))*time.Millisecond
			fmt.Printf("Retrying in %v...\n", sleepTime)
			time.Sleep(sleepTime)
			continue
		case http.StatusInternalServerError, http.StatusBadGateway, http.StatusServiceUnavailable, http.StatusGatewayTimeout:
			// Server-side errors that might be transient
			fmt.Printf("Attempt %d: Server error %d. Retrying.\n", i+1, resp.StatusCode)
			if i == maxRetries {
				return nil, fmt.Errorf("server error %d after %d retries", resp.StatusCode, maxRetries)
			}
			sleepTime := time.Duration(1<<i)*time.Second + time.Duration(rand.Intn(1000))*time.Millisecond
			fmt.Printf("Retrying in %v...\n", sleepTime)
			time.Sleep(sleepTime)
			continue
		default:
			// Non-retryable errors (4xx client errors, etc.)
			body, _ := ioutil.ReadAll(resp.Body)
			return nil, fmt.Errorf("non-retryable API error: %d %s, response: %s", resp.StatusCode, resp.Status, string(body))
		}
	}
	return nil, fmt.Errorf("unexpected control flow") // Should not be reached
}

func main() {
	// Example usage: Replace with actual Didit API endpoint
	// For a real integration, you'd be POSTing to /v3/session/ or similar
	// and handling the verification_url.
	apiURL := "https://verification.didit.me/health"
	data, err := callDiditAPIWithRetry(apiURL, 5)
	if err != nil {
		fmt.Println("Failed to call API:", err)
	} else {
		fmt.Println("API Response:", string(data))
	}
}

This snippet demonstrates how to handle network errors, 429s (respecting Retry-After), and common 5xx errors with exponential backoff and jitter. It also shows failing fast for non-retryable 4xx errors, which are typically due to incorrect input and won't resolve on retry. This is crucial for operations like creating a verification session using Didit's modular architecture.

Implementing a Circuit Breaker Pattern

While retries help with transient issues, continuously retrying a failing service can overload it further or waste resources if the service is truly down. This is where the circuit breaker pattern comes in. A circuit breaker monitors for failures and, if they reach a certain threshold, "opens" the circuit, preventing further requests for a set period. After the period, it allows a few test requests to see if the service has recovered.

In Go, you can use libraries like sony/gobreaker to implement this pattern:

package main

import (
	"errors"
	"fmt"
	"io/ioutil"
	"net/http"
	"time"

	"github.com/sony/gobreaker"
)

var cb *gobreaker.CircuitBreaker

func init() {
	st := gobreaker.Settings{
		Name:        "DiditAPICircuitBreaker",
		MaxRequests: 3, // Allow 3 requests in half-open state
		Interval:    5 * time.Second, // Period to collect data for trip decisions
		Timeout:     10 * time.Second, // Open circuit for 10 seconds
		ReadyToTrip: func(counts gobreaker.Counts) bool {
			return counts.ConsecutiveFailures > 5 // Trip after 5 consecutive failures
		},
		OnStateChange: func(name string, from gobreaker.State, to gobreaker.State) {
			fmt.Printf("Circuit Breaker '%s' changed from %s to %s\n", name, from, to)
		},
	}
	cb = gobreaker.NewCircuitBreaker(st)
}

func callDiditAPIWithCircuitBreaker(url string) ([]byte, error) {
	body, err := cb.Execute(func() (interface{}, error) {
		resp, err := http.Get(url)
		if err != nil {
			return nil, err // Network error
		}
		defer resp.Body.Close()

		if resp.StatusCode != http.StatusOK {
			// Only count 5xx and 429 as failures for circuit breaker
			if resp.StatusCode == http.StatusTooManyRequests || resp.StatusCode >= http.StatusInternalServerError {
				return nil, fmt.Errorf("API returned status %d", resp.StatusCode)
			}
			// For other 4xx errors, we might not want to trip the breaker, but still return an error
			bodyBytes, _ := ioutil.ReadAll(resp.Body)
			return nil, fmt.Errorf("non-retryable API error: %d %s, response: %s", resp.StatusCode, resp.Status, string(bodyBytes))
		}

		return ioutil.ReadAll(resp.Body)
	})

	if err != nil {
		if errors.Is(err, gobreaker.ErrOpenState) {
			return nil, fmt.Errorf("circuit breaker is open: %w", err)
		}
		return nil, err
	}

	return body.([]byte), nil
}

func main() {
	// Example usage
	apiURL := "https://verification.didit.me/health"
	for i := 0; i < 10; i++ {
		data, err := callDiditAPIWithCircuitBreaker(apiURL)
		if err != nil {
			fmt.Printf("Call %d failed: %v\n", i+1, err)
		} else {
			fmt.Printf("Call %d successful: %s\n", i+1, string(data))
		}
		time.Sleep(1 * time.Second)
	}
}

Combining circuit breakers with exponential backoff ensures that your application remains responsive and doesn't overwhelm a struggling external service. This is particularly important when dealing with high-volume identity verification requests, such as those involving Didit's ID Verification or Passive & Active Liveness checks.

Handling Webhook Failures and Asynchronous Results

Didit's API often provides results asynchronously via webhooks, for example, after a user completes an ID verification flow or when an AML Screening check is finished. Your webhook endpoint must be robust. If your endpoint fails to process a webhook (e.g., returns a 5xx status code), Didit will retry delivering the webhook. It's crucial to:

• Acknowledge Receipt Immediately: Return a 2xx status code as quickly as possible to signal successful receipt, even if processing takes longer.
• Process Asynchronously: Hand off the webhook payload to a background worker or message queue (e.g., Kafka, RabbitMQ) for processing. This prevents your webhook endpoint from timing out Didit's retries.
• Idempotency: Ensure your webhook processing logic is idempotent. If Didit retries and delivers the same webhook multiple times, your system should process it only once to avoid duplicate actions or data inconsistencies.
• Verify Signatures: Always verify the webhook signature using your Webhook Secret Key from the Didit Console to ensure the request genuinely originated from Didit and hasn't been tampered with.

How Didit Helps

Didit is designed with developer experience and reliability in mind, providing features that inherently simplify advanced error handling and retry strategies for your Go integrations. Our developer-first approach means clear, comprehensive API documentation, an instant sandbox, and clean APIs that make integration straightforward.

• Predictable Rate Limiting: Didit clearly defines global and endpoint-specific rate limits, along with standard HTTP headers (X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, Retry-After) to guide your client-side throttling and backoff logic. This transparency empowers you to build optimized retry mechanisms for services like ID Verification and AML Screening.
• Modular Architecture: Our modular identity primitives mean you can build workflows that are resilient by design. If one specific check (e.g., Proof of Address) encounters a transient issue, your overall workflow can be configured to adapt or retry specific components without impacting unrelated verification steps.
• AI-Native Reliability: Didit's AI-native backend is built for scale and resilience, minimizing the server-side errors you'll encounter. This allows you to focus your error handling on network and client-side issues, rather than constant service outages.
• Free Core KYC & Flexible Pricing: Get started with Didit's free tier for Core KYC, allowing you to thoroughly test your error handling and retry strategies in a production-like environment without upfront costs. Our pay-per-successful-check model further aligns our success with yours.

Ready to Get Started?

Ready to see Didit in action? Get a free demo today.

Start verifying identities for free with Didit's free tier.