Idempotency

Definition

Idempotency is a mathematical and computer science concept that means performing an operation multiple times has the same effect as performing it once. In the context of REST APIs, an idempotent HTTP method can be called repeatedly with the same parameters, and the result will be identical after the first successful request - no additional side effects occur on subsequent calls.

Think of idempotency like a light switch. Flipping the switch to “on” multiple times doesn’t make the light “more on” - it stays on after the first flip. Similarly, sending the same DELETE request five times doesn’t delete the resource five times - it’s deleted once, and subsequent requests simply confirm it’s already gone.

This property is crucial for building reliable distributed systems. Networks are unreliable - requests can fail, timeout, or get duplicated. With idempotent operations, clients can safely retry failed requests without worrying about unintended consequences. In REST, GET, PUT, DELETE, HEAD, OPTIONS, and TRACE are idempotent by specification. POST and PATCH are generally NOT idempotent (though PATCH can be designed to be idempotent in some cases).

Example

Payment Processing: When Stripe processes a payment, they use idempotency keys. If your server crashes after sending POST /v1/charges but before receiving the response, you can retry with the same idempotency key. Stripe recognizes the duplicate request and returns the original charge instead of creating a new one. This prevents double-charging customers.

File Upload: Uploading a file with PUT /files/photo.jpg is idempotent. If the network times out, you can safely retry the PUT request. The first successful upload stores the file; subsequent identical PUTs just overwrite it with the same data - the result is the same.

Resource Deletion: DELETE /api/users/123 is idempotent. The first request deletes user 123 and returns 204 No Content. The second request finds the user already gone and returns 404 Not Found or 204 No Content (both are valid). The server state is the same - user 123 doesn’t exist.

Updating with PUT: PUT /api/products/456 { "name": "Widget", "price": 9.99 } is idempotent. Sending this request 10 times results in the product having name “Widget” and price 9.99 - the same state as after the first request.

Counter Increment (NOT Idempotent): POST /api/counter/increment is NOT idempotent. Each request increases the counter by 1. Sending it 5 times makes the counter 5, not 1. This is why POST is generally not idempotent - it creates or modifies state in ways that accumulate.

Bank Transfer (NOT Idempotent): POST /api/transfers { "from": "acct1", "to": "acct2", "amount": 100 } is NOT idempotent. Retrying this request would transfer $100 again. Banks solve this with idempotency tokens or transaction IDs to make transfers safely retryable.

Analogy

The Light Switch: Flipping a light switch to “on” is idempotent. Whether you flip it once or ten times, the light is on. The state after the first flip is the same as after subsequent flips. Flipping it to “off” is also idempotent. But pressing a doorbell is NOT idempotent - each press rings the bell again.

Setting a Thermostat: Setting your thermostat to 72°F is idempotent. Doing it once or 100 times results in the same temperature setting. But pressing “increase temperature by 1 degree” is NOT idempotent - each press adds another degree.

Entering a ZIP Code: When buying something online, entering your ZIP code “10001” multiple times doesn’t create multiple addresses or change the outcome - it’s idempotent. But clicking “Add to Cart” is NOT idempotent - each click adds another item.

Replacing a File: Saving a document with “Save As” to the same filename is idempotent. The file is replaced with the same content. But using “Insert Duplicate Page” is NOT idempotent - each click duplicates the page again.

Parking Brake: Engaging your car’s parking brake is idempotent. Pulling it once or five times results in the same state - the brake is engaged. But honking the horn is NOT idempotent - each press honks again.

Code Example

// ===== IDEMPOTENT HTTP METHODS =====

// GET - Idempotent (read-only, no side effects)
// Multiple calls return the same data, no state changes
fetch('/api/users/123', { method: 'GET' });
fetch('/api/users/123', { method: 'GET' }); // Same result
fetch('/api/users/123', { method: 'GET' }); // Same result

// PUT - Idempotent (replaces resource)
// Multiple identical PUTs result in same final state
await fetch('/api/products/456', {
  method: 'PUT',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    name: 'Widget',
    price: 9.99,
    stock: 100
  })
});
// Retry with same data - resource ends up in identical state
await fetch('/api/products/456', {
  method: 'PUT',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    name: 'Widget',
    price: 9.99,
    stock: 100
  })
}); // Product has same values - idempotent

// DELETE - Idempotent (removes resource)
await fetch('/api/users/123', { method: 'DELETE' }); // 204 No Content
await fetch('/api/users/123', { method: 'DELETE' }); // 404 Not Found (already gone)
// Result is the same: user 123 doesn't exist

// ===== NON-IDEMPOTENT METHOD: POST =====
// Each POST creates a new resource
const response1 = await fetch('/api/orders', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({ product_id: 456, quantity: 1 })
});
// Returns: { "id": 789, "product_id": 456, "quantity": 1 }

const response2 = await fetch('/api/orders', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({ product_id: 456, quantity: 1 })
});
// Returns: { "id": 790, "product_id": 456, "quantity": 1 }
// Different ID! NOT idempotent - created two orders

// ===== MAKING POST IDEMPOTENT WITH IDEMPOTENCY KEYS =====
// Stripe-style idempotency key pattern
const idempotencyKey = crypto.randomUUID();

async function createChargeWithRetry() {
  const maxRetries = 3;
  for (let i = 0; i < maxRetries; i++) {
    try {
      const response = await fetch('/api/charges', {
        method: 'POST',
        headers: {
          'Content-Type': 'application/json',
          'Idempotency-Key': idempotencyKey // Same key on retries
        },
        body: JSON.stringify({
          amount: 5000,
          currency: 'usd',
          customer: 'cus_123'
        })
      });

      if (response.ok) {
        return await response.json();
      }
    } catch (error) {
      if (i === maxRetries - 1) throw error;
      // Retry with same idempotency key
      await new Promise(resolve => setTimeout(resolve, 1000 * Math.pow(2, i)));
    }
  }
}

// Server-side idempotency key implementation
const idempotencyCache = new Map();

app.post('/api/charges', async (req, res) => {
  const idempotencyKey = req.headers['idempotency-key'];

  if (!idempotencyKey) {
    return res.status(400).json({ error: 'Idempotency-Key header required' });
  }

  // Check if we've already processed this request
  if (idempotencyCache.has(idempotencyKey)) {
    const cachedResponse = idempotencyCache.get(idempotencyKey);
    return res.status(200).json(cachedResponse);
  }

  try {
    // Process the charge
    const charge = await processCharge(req.body);

    // Cache the response for future retries
    idempotencyCache.set(idempotencyKey, charge);

    // Set cache expiration (e.g., 24 hours)
    setTimeout(() => idempotencyCache.delete(idempotencyKey), 86400000);

    res.status(201).json(charge);
  } catch (error) {
    res.status(500).json({ error: 'Payment processing failed' });
  }
});

// ===== PATCH CAN BE IDEMPOTENT (if designed carefully) =====
// Idempotent PATCH: Sets absolute values
await fetch('/api/users/123', {
  method: 'PATCH',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    email: '[email protected]' // Absolute value
  })
}); // User email becomes '[email protected]'

await fetch('/api/users/123', {
  method: 'PATCH',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    email: '[email protected]'
  })
}); // User email is still '[email protected]' - IDEMPOTENT

// Non-Idempotent PATCH: Incremental changes
await fetch('/api/products/456', {
  method: 'PATCH',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    stock: { $inc: -1 } // Decrement by 1
  })
}); // Stock decreases by 1

await fetch('/api/products/456', {
  method: 'PATCH',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    stock: { $inc: -1 }
  })
}); // Stock decreases by 1 AGAIN - NOT idempotent

Diagram

graph TB
    subgraph "Idempotent Methods"
        GET[GET /api/users/123]
        PUT[PUT /api/users/123]
        DELETE[DELETE /api/users/123]

        GET -->|Request 1| R1[Returns user data]
        GET -->|Request 2| R2[Returns same data]
        GET -->|Request 3| R3[Returns same data]

        PUT -->|Request 1| P1[Updates user]
        PUT -->|Request 2| P2[User in same state]
        PUT -->|Request 3| P3[User in same state]

        DELETE -->|Request 1| D1[Deletes user, 204]
        DELETE -->|Request 2| D2[Already gone, 404]
        DELETE -->|Request 3| D3[Already gone, 404]
    end

    subgraph "Non-Idempotent Method: POST"
        POST[POST /api/orders]

        POST -->|Request 1| O1[Creates order #789]
        POST -->|Request 2| O2[Creates order #790]
        POST -->|Request 3| O3[Creates order #791]
    end

    subgraph "POST with Idempotency Key"
        POSTI[POST /api/charges
Idempotency-Key: abc123]

        POSTI -->|Request 1| C1[Creates charge #1000]
        POSTI -->|Request 2
Same Key| C2[Returns charge #1000
from cache]
        POSTI -->|Request 3
Same Key| C3[Returns charge #1000
from cache]
    end

    style GET fill:#90EE90
    style PUT fill:#90EE90
    style DELETE fill:#90EE90
    style POST fill:#FFB6C1
    style POSTI fill:#FFD700

Security Notes

Best Practices

1. Design PUT and DELETE to be idempotent: Follow HTTP semantics strictly
2. Implement idempotency keys for POST: Use headers like Idempotency-Key for create operations
3. Use UUIDs for idempotency keys: Client-generated, cryptographically random identifiers
4. Cache idempotency responses: Store results for 24-72 hours to handle retries
5. Return same status code on retries: 201 on first request, 200 or 201 on subsequent requests with same key
6. Set expiration on cached responses: Clean up old idempotency mappings to save storage
7. Document retry behavior: Clearly state which endpoints are idempotent in API docs
8. Use ETags for conditional updates: Prevent overwriting newer data with stale updates
9. Implement retry logic with exponential backoff: Client-side retry strategies for transient failures
10. Test retry scenarios: Validate that retries don’t cause duplicate resources or state corruption

Common Mistakes

Treating POST as idempotent: Retrying POST requests without idempotency keys can create duplicate resources.

Using non-idempotent PATCH: Incremental updates like {"quantity": {"$inc": 1}} are not safe to retry.

Not caching idempotency responses: Forces reprocessing of duplicate requests, wasting resources.

Short cache expiration: Expiring idempotency keys too quickly (e.g., 5 minutes) prevents safe retries for slow clients.

Ignoring DELETE 404 as error: A 404 on DELETE retry doesn’t mean failure - the resource is gone as intended.

Client-generated IDs without validation: Trusting client IDs for idempotency without server-side uniqueness checks.

Reusing idempotency keys: Using the same key for different operations or users causes collisions.

No retry logic: Clients giving up after first failure instead of retrying idempotent operations.

Assuming GET is always safe: GET requests that trigger side effects (like /api/users/123/send-email) violate idempotency.

Not handling timeouts: Timeout doesn’t mean failure - retry with same idempotency key to check result.

Standards & RFCs

Related Terms