API Idempotency: Retry Requests Without Creating Duplicates

Suppose your application asks an API to create a report export. The server creates the job, commits the database transaction and sends a response. The connection drops before your application receives it.

Your application sees a timeout. The server has already done the work.

A retry can create a second job unless the API recognises it as another attempt at the same operation. Idempotency makes those retries safe within an API's documented scope and retention period. For an operation such as creating a job, that usually means keeping one operation key across attempts and having the server record the key and result reliably.

This guide explains how to design that contract, demonstrates it with runnable Python and SQLite code, and shows where the guarantee ends when a worker calls another service.

A timeout leaves the outcome uncertain

A timeout tells the caller that it stopped waiting. It does not establish whether the server received the request, started processing it or committed its changes. Amazon's engineering guidance on timeouts and retries makes this distinction explicit: a failed call can still have produced side effects.

For the report example, there are two very different possibilities:

• The request never reached the service, so no export job exists.
• The service committed the job, but the caller never received its identifier.

The same error screen can represent either state. A retry mechanism needs to handle both without asking the user to guess.

Disabling a button while a request runs can reduce repeated clicks. It does not solve a connection failure, a client restart or a retry made by another part of the application. The server needs enough information to recognise the original operation.

Idempotency concerns the intended effect

HTTP Semantics, RFC 9110 defines idempotency around the effect of repeating a request. PUT, DELETE and the safe methods are idempotent by their defined semantics. POST has no blanket guarantee.

The response itself can change. For example, deleting a resource and then deleting it again can leave it absent both times, even if the second request reports that it was already absent. Idempotency also permits incidental activity such as recording each attempt in a log.

For a POST endpoint that creates an export, define the intended effect precisely: one export-job record for one requested export. That does not yet promise one generated file, one notification email or one successful downstream upload. Each of those actions introduces another place where retries need attention.

Give each intended operation its own key

An idempotency key identifies what the caller is trying to do. Generate it when a new operation begins, before the first attempt, and retain it with that operation's request data.

Consider two identical export requests:

{"report":"usage","format":"csv"}

They might be retries of one export. They might also be two exports deliberately requested at different times. A hash of the request body cannot distinguish those intentions. Amazon's account of making retries safe explains why caller-supplied request identifiers are useful for that reason.

For this example, use the following rule:

A random UUID is a practical identifier. Avoid embedding email addresses, access tokens or other sensitive information in it. Stripe's idempotent-request documentation also recommends random identifiers and warns against sensitive data in keys.

If an SDK creates a key automatically, check how long that key survives. A retry inside one SDK call and a new call after your process restarts are different situations. When the application must resume an operation, save its identity before sending it.

Scope the key to the authenticated caller

A useful application identity is:

(authenticated tenant, operation version, idempotency key)

In the example below, the operation version is create-export:v1. The tenant comes from the caller's authenticated context. The key is supplied for that operation.

This is a design choice for the example, rather than a universal HTTP format. It lets two tenants use the same key without sharing a result, and prevents an identifier for one operation from colliding with another operation's identifier.

Authorise every request, including a retry that will return a saved result. Possessing a key must not grant access to another person's job. Apply limits to key length and retained data, and protect stored response bodies as application data.

Use a fingerprint to detect changed input

Keep a fingerprint of the validated request alongside the key. Its job is to detect a programming mistake such as retrying an existing key with different export options.

Define which values determine the operation before calculating that fingerprint. Include every value that changes what the operation does; account for relevant defaults and operation versions. Exclude transport details that have no bearing on the requested work.

The example accepts exactly two fields with a small set of string values. Sorting those JSON keys gives it a stable representation. That is sufficient for this deliberately narrow schema. It is not a general solution for comparing arbitrary JSON, floating-point values or differently normalised strings.

Read the API's actual retry contract

As checked on 28 October 2026, the IETF's Idempotency-Key proposal, draft-08 , was published on 22 October. It remains a work in progress, not an RFC.

The proposal uses a Structured Fields string, illustrated by a quoted header value:

Idempotency-Key: "7dd42e48-3073-43ec-80f5-a90a36db0e59"

It proposes 400 for a required missing key, 422 for a key reused with different input, and 409 when the original request is still running. It also says clients cannot assume an arbitrary server will honour the header. Follow the endpoint's documentation for syntax, error handling and retention.

The proposal's completed-retry wording concerns a response consistent with the user's intent. It does not require every implementation to return an identical saved response. That distinction already matters in real APIs:

These policies establish different expectations for the caller. A response cache is a valid design for our local job example; it is not the definition of idempotency.

Save the local change and its result together

The dangerous implementation has a gap between doing the work and recording that it happened:

create the job
commit the job
save the idempotency result

If the process dies after the job commits but before the result is saved, a retry can find no record of the key and create another job. Reversing the order introduces a different failure: a recorded key with no completed job.

Amazon's retry design discussion calls for the operation's mutations and identifier record to form an atomic action. For changes held in one database, that means putting the business row and retained result in one transaction.

There is also a concurrency problem. Two requests can both check for a missing key before either writes it. The database must arbitrate who gets to perform the operation. A check in application code is insufficient unless the surrounding transaction and locking rules make it safe.

The failure in the following sequence happens after the database commits. A retry asks the API to recover the result already stored for key K.

*The first reply is lost, but the committed job and its retained result remain together. Retrying with K returns that job without creating another.*

In this example, the job and the retained response are written in one database transaction. The diagram follows that local operation; it does not cover an external export worker or another service.

Both records are present after commit. Before commit, neither is a completed operation. That clear boundary is what the demonstration exercises.

A runnable Python and SQLite example

This example creates a local database row representing a queued export. It does not generate the report or contact another service.

Use Python 3.12 or newer, with its standard-library SQLite module. The code uses the autocommit argument introduced in Python 3.12 and controls transactions through explicit SQL. With autocommit=True, the Python connection's commit() and rollback() methods have no effect; the explicit COMMIT and ROLLBACK statements below are intentional. See the Python sqlite3 transaction-control reference .

The function expects an already authenticated and authorised tenant identifier and a decoded application key. It includes neither authentication nor an HTTP header parser. The optional _checkpoint callback is only a hook for the failure checks described afterwards.

Save this as idempotency_demo.py:

"""Local transaction example; Python 3.12+, standard library only.

The caller must authenticate and authorise tenant_id before calling create_export.
This creates a database job row, not an exported file or an external side effect.
"""
import hashlib
import json
import re
import sqlite3
import tempfile
import uuid
from contextlib import closing
from pathlib import Path

OPERATION = "create-export:v1"


class KeyReuseError(ValueError):
    pass


def connect(path):
    # Explicit SQL controls transactions; commit()/rollback() are not used.
    return sqlite3.connect(path, timeout=5.0, autocommit=True)


def initialise(path):
    with closing(connect(path)) as db:
        db.executescript("""
            CREATE TABLE IF NOT EXISTS export_jobs (
                id TEXT PRIMARY KEY,
                tenant_id TEXT NOT NULL,
                report TEXT NOT NULL,
                format TEXT NOT NULL
            );
            CREATE TABLE IF NOT EXISTS request_results (
                tenant_id TEXT NOT NULL,
                operation TEXT NOT NULL,
                idem_key TEXT NOT NULL,
                fingerprint TEXT NOT NULL,
                status INTEGER NOT NULL,
                body TEXT NOT NULL,
                PRIMARY KEY (tenant_id, operation, idem_key)
            );
        """)


def create_export(path, tenant_id, key, payload, *, _checkpoint=None):
    """Return (status, JSON body); _checkpoint is only for failure injection."""
    if not isinstance(tenant_id, str) or not re.fullmatch(r"[A-Za-z0-9_-]{1,64}", tenant_id):
        raise ValueError("Invalid authenticated tenant identifier")
    if not isinstance(key, str) or not re.fullmatch(r"[A-Za-z0-9_-]{16,128}", key):
        raise ValueError("Key must be 16-128 ASCII letters, digits, '_' or '-'")
    if (not isinstance(payload, dict) or set(payload) != {"report", "format"}
            or payload["report"] not in ("usage", "billing")
            or payload["format"] not in ("csv", "json")):
        raise ValueError("Expected report usage/billing and format csv/json")
    canonical = json.dumps(payload, sort_keys=True, separators=(",", ":"))
    fingerprint = hashlib.sha256(canonical.encode("utf-8")).hexdigest()
    scope = (tenant_id, OPERATION, key)
    with closing(connect(path)) as db:
        try:
            db.execute("BEGIN IMMEDIATE")
            row = db.execute("""
                SELECT fingerprint, status, body FROM request_results
                WHERE tenant_id=? AND operation=? AND idem_key=?
            """, scope).fetchone()
            if row is not None:
                if row[0] != fingerprint:
                    raise KeyReuseError("This key already identifies a different request")
                db.execute("COMMIT")
                return row[1], row[2]
            job_id = str(uuid.uuid4())
            db.execute("INSERT INTO export_jobs VALUES (?, ?, ?, ?)",
                       (job_id, tenant_id, payload["report"], payload["format"]))
            if _checkpoint:
                _checkpoint("after_job")
            body = json.dumps({"job_id": job_id, "state": "queued"}, sort_keys=True)
            db.execute("INSERT INTO request_results VALUES (?, ?, ?, ?, ?, ?)",
                       (*scope, fingerprint, 201, body))
            if _checkpoint:
                _checkpoint("before_commit")
            db.execute("COMMIT")
            if _checkpoint:
                _checkpoint("after_commit")
            return 201, body
        except BaseException:
            if db.in_transaction:
                db.execute("ROLLBACK")
            raise


if __name__ == "__main__":
    with tempfile.TemporaryDirectory(prefix="idempotency-demo-") as directory:
        path = Path(directory) / "demo.sqlite3"
        initialise(path)
        key = str(uuid.uuid4())
        request = {"report": "usage", "format": "csv"}
        first = create_export(path, "tenant-a", key, request)
        second = create_export(path, "tenant-a", key, request)
        print("Same status and response body:", first == second)
        with closing(connect(path)) as db:
            print("Export jobs:", db.execute("SELECT COUNT(*) FROM export_jobs").fetchone()[0])

Run it with:

python3 idempotency_demo.py

Use the command that launches your Python 3.12 or newer installation if it has a different name.

The runner creates a temporary database, calls the operation twice with one key, and removes the temporary directory when it finishes. Its output is:

Same status and response body: True
Export jobs: 1

Why the transaction holds

BEGIN IMMEDIATE starts a SQLite write transaction before the lookup. SQLite permits only one writer at a time, so another writer cannot pass its own lookup and insert concurrently. The next caller can inspect the committed result after the first transaction finishes. The composite primary key also enforces the chosen identity. These behaviours follow SQLite's transaction rules and isolation model .

The first call inserts both the job and its successful response before committing. A matching retry returns the saved pair. A changed payload raises KeyReuseError before any additional job is created. If an exception occurs inside an active transaction, the function rolls it back.

This simple approach serialises unrelated writes too. The connection waits up to its configured five-second lock timeout and can then raise sqlite3.OperationalError. That timeout is a demonstration setting, not a recommended API deadline. The function does not implement an HTTP 409 policy.

A service using another database needs that database's own concurrency design. Copying the initial SELECT into a different transaction model does not carry over SQLite's single-writer guarantee.

What was actually verified

The example was run on Linux with Python 3.12.14 and SQLite 3.53.1. Nine local checks passed. They covered these outcomes:

The termination checks used os._exit(71) in child processes, bypassing Python's normal cleanup. These observations establish what happened in this local demonstration. They are not tests of hardware power loss, filesystem corruption, distributed failover or production throughput.

The example deliberately omits expiry, quotas, worker execution and HTTP response headers. A replayed "state": "queued" is the saved creation response; a separate job-status endpoint would be needed to report later progress.

An external service creates another failure boundary

Now extend the example so that a worker uploads the report or sends a completion email. The SQLite transaction cannot make that remote action atomic with its local status update.

Suppose an email service accepts the message, then the worker dies before marking the job as notified. A replacement worker sees unfinished work. Sending again may produce a second email; skipping it may lose a notification in the case where the first send never succeeded.

A transactional outbox helps by storing the business change and a message-to-send in one database transaction. A relay later delivers committed messages. This preserves the intent to deliver, but the relay can still deliver a message more than once.

Carry a stable identifier to the next system when its API supports idempotency. Otherwise, design a way to query or reconcile the remote outcome. Record which downstream operation belongs to which local job. An outbox alone cannot turn an uncertain remote response into proof that no side effect occurred.

This is also why “the API request is idempotent” and “the entire workflow executes exactly once” are different claims. Name the operation and the system boundary your guarantee actually covers.

Expiry belongs in the client design

A retained key cannot recognise an operation after its record has been removed. Stripe explicitly documents that reusing a pruned API v1 key can start a new request .

For your own API, choose retention alongside the ways clients resume work. Consider an application reconnecting after being offline, a delayed queue message, or a human retrying from an operations console. These are different retry paths, and their allowed windows should be explicit.

If retries can outlive the stored result, give clients another way to resolve the original operation. A durable business identifier can help: for example, a unique export-request ID that remains associated with the job even after a short-lived response record expires.

Do not silently recycle a key for a new operation. Equally, do not claim permanent duplicate prevention from a record with a short expiry. Document what happens when the supported replay window has ended.

Retry with a budget, then resolve uncertainty

An idempotent operation still consumes resources when requests reach the service. Amazon recommends bounded retries with backoff and jitter to avoid making an overloaded service worse. Check whether the SDK already retries before adding another loop around it.

Use a maximum elapsed time and attempt budget, preserve the same operation identity, and honour the endpoint's documented retry guidance. Once the budget is exhausted, the outcome may remain unknown. The application needs a status check or reconciliation path.

For a concrete example, Stripe's API v1 error guidance treats a 500 response as an indeterminate outcome and warns against switching to a new key before resolving it. A new key can mean permission to start another operation.

Before adding automatic retries to a state-changing endpoint, test the point where the response disappears after commit. Then test simultaneous attempts and a process restart. The useful result is straightforward: the caller can recover the original operation's identity and outcome without creating another one.

Sources and further reading

Sources were checked on 28 October 2026. The IETF document below is explicitly a work in progress.

• RFC 9110, section 9.2.2: Idempotent Methods .
• IETF: The Idempotency-Key HTTP Header Field, draft-08 .
• Stripe: Idempotent requests .
• Stripe: API v2 idempotency .
• Stripe: Advanced error handling .
• Amazon Builders' Library: Making retries safe with idempotent APIs .
• Amazon Builders' Library: Timeouts, retries, and backoff with jitter .
• AWS Prescriptive Guidance: Transactional outbox pattern .
• SQLite: Transaction control and isolation .
• Python 3.12: sqlite3 reference .