Retry Strategies and Resilience

Why Retry? Transient failures (network issues, rate limits, temporary overload) are common. Proper retry logic improves reliability.

Retry Patterns

Simple Retry

def simple_retry(func, max_retries=3):
    for attempt in range(max_retries):
        try:
            return func()
        except Exception as e:
            if attempt == max_retries - 1:
                raise
            time.sleep(1)

Exponential Backoff

def exponential_backoff(func, max_retries=5, base_delay=1):
    for attempt in range(max_retries):
        try:
            return func()
        except Exception as e:
            if attempt == max_retries - 1:
                raise
            delay = base_delay * (2 ** attempt)
            time.sleep(delay)

With Jitter Prevent thundering herd:

import random

def backoff_with_jitter(func, max_retries=5, base_delay=1):
    for attempt in range(max_retries):
        try:
            return func()
        except Exception as e:
            if attempt == max_retries - 1:
                raise
            delay = base_delay * (2 ** attempt)
            jitter = random.uniform(0, delay * 0.1)
            time.sleep(delay + jitter)

Tenacity Library

from tenacity import retry, stop_after_attempt, wait_exponential

@retry(
    stop=stop_after_attempt(5),
    wait=wait_exponential(multiplier=1, min=1, max=60)
)
def call_llm(prompt):
    return openai.chat.completions.create(
        model="gpt-4",
        messages=[{"role": "user", "content": prompt}]
    )

Retry Only Retriable Errors

from tenacity import retry, retry_if_exception_type

@retry(
    retry=retry_if_exception_type((RateLimitError, TimeoutError)),
    stop=stop_after_attempt(3)
)
def call_api():
    return api.request()

Backoff Strategies

Best Practices

• Always include maximum retry limit
• Add jitter to prevent thundering herd
• Log retry attempts for debugging
• Use circuit breakers for sustained failures
• Only retry retriable errors
• Consider operation idempotency