--- name: retry-safety-review description: Decide which operations are actually safe to retry, by separating failures that definitely did nothing from failures where the work may already have happened. Use before adding a retry wrapper, configuring queue redelivery, or debugging duplicate charges, duplicate emails, or double-applied writes. --- # Retry Safety Review ## Install Save this file as `~/.claude/skills/retry-safety-review/SKILL.md`, or `.claude/skills/retry-safety-review/SKILL.md` to scope it to one repo. Claude Code auto-discovers it. Invoke with `/retry-safety-review` or by asking "is it safe to retry this?". ## Why this exists Retries are added during an outage, by someone who wants the errors to stop. The wrapper goes around everything, the errors do stop, and a category of much worse bug is created silently: work that ran twice. The trap is that the two cases look identical from the caller's side. A connection refused before the request left your process did nothing. A timeout after the request arrived may have committed, charged, or sent. Both surface as "the call failed". Retrying the first is free. Retrying the second duplicates. **The question is never "did it fail?" It is "do I know whether it ran?"** ## Step 1: Find every retry that already exists ``` rg -n "retry|retries|backoff|attempt|maxAttempts|reconnect" -i \ --type-add 'code:*.{ts,tsx,js,jsx,py,rb,go,java}' -t code rg -n "p-retry|async-retry|tenacity|backoff|resilience4j|Polly" -t code ``` Also find the retries you do not control: - Message queue redelivery (SQS, Pub/Sub, Kafka, BullMQ, Sidekiq). - Webhook senders. Stripe, GitHub, and Shopify all retry on non-2xx. - Load balancers and service meshes configured to retry idempotent methods. - Browser or client SDK retries. - A human clicking a button twice. Every one of those is a retry your code must survive, whether or not you wrote a retry loop. ## Step 2: Classify each retried operation For every operation inside a retry, put it in exactly one bucket: **SAFE (naturally idempotent).** Running it twice produces the same end state as running it once. Pure reads. `SET x = 5`. `DELETE WHERE id = 1`. Writes keyed by a unique constraint that would reject the duplicate. **SAFE ONLY WITH A KEY.** Not naturally idempotent, but made safe by an idempotency key, a unique constraint, or a conditional write. Creating a charge with `Idempotency-Key`. `INSERT ... ON CONFLICT DO NOTHING`. Check the key is actually derived from the WORK, not generated per attempt: a key created inside the retry loop is a new key every time and protects nothing. **UNSAFE.** Running it twice does damage and nothing prevents it. Relative updates (`balance = balance + 10`). Sending email, SMS, or a push notification. Appending to a ledger. Calling a third-party API that creates a resource without an idempotency key. Anything that increments a counter. Be specific about which line makes it unsafe. ## Step 3: Classify each failure mode This is the step people skip. For each error your retry catches, decide: **DEFINITELY DID NOT RUN.** Connection refused, DNS failure, TLS handshake failure, request rejected before dispatch, client-side validation error. The work provably never started. Safe to retry anything. **MAY HAVE RUN.** Timeout, connection reset mid-flight, 5xx from a server that already received the body, a database driver error raised after the statement was sent, any error where the response was lost rather than never produced. The second category is the whole point. A retry policy that treats "timed out" the same as "connection refused" will double-apply the first time a slow write succeeds after the client gave up. Go read the error codes your retry actually matches, and sort them into these two buckets by hand. If a code is ambiguous, it belongs in MAY HAVE RUN. Ambiguity is not a reason to retry; it is the reason not to. ## Step 4: Cross the two lists Build the grid. Only one cell is a defect: | | DEFINITELY DID NOT RUN | MAY HAVE RUN | | --- | --- | --- | | **SAFE** | retry | retry | | **SAFE WITH KEY** | retry | retry, if the key is stable across attempts | | **UNSAFE** | retry | **DEFECT: this duplicates** | For every UNSAFE operation retried on a MAY HAVE RUN error, you have found a real bug. Describe the exact interleaving that duplicates, with the concrete sequence of events, not a general worry. ## Step 5: Check what the retry does to the rest of the system Even correct retries cause damage at scale: - **No backoff.** Immediate retries against a struggling service finish it off. - **No jitter.** Every client retries at the same instant and the recovery attempt becomes a second outage. - **No cap.** Retries stacked at several layers multiply: 3 client x 3 gateway x 3 service is 27 attempts from one user action. - **Retrying inside a held transaction or lock.** Multiplies the time the lock is held by the number of attempts. - **Retry budget vs caller timeout.** If the caller gives up after 5s and the retry policy needs 12s, the retries only ever burn capacity for a response nobody will read. Compute the real worst-case latency and attempt count, and compare it to the timeout of whatever is calling. ## Step 6: Report ``` [SEVERITY] retried on Where: file:line Bucket: SAFE / SAFE WITH KEY / UNSAFE On failure: DEFINITELY DID NOT RUN / MAY HAVE RUN Duplicates: Fix: ``` Severity: - **CRITICAL** an UNSAFE operation is retried on an ambiguous error today. - **HIGH** a stacked or uncapped retry that can amplify an outage, or an idempotency key regenerated per attempt. - **MEDIUM** missing jitter or backoff, or a retry budget longer than the caller's timeout. End with the grid from Step 4 filled in for this codebase. If every retry is correctly scoped, show the grid as the evidence, and name the error codes you deliberately excluded. An excluded ambiguous code is a design decision worth recording, and it is exactly what a future reader will otherwise "helpfully" add back. ## Rules - Never widen a retry to cover a new error class without stating which bucket that class is in. - Never claim an operation is idempotent because it "looks like" a write that is. Trace the actual statement. - A retry you did not write still counts. Queue redelivery and webhook senders are retries. - Do not add a retry as the fix for a bug you have not diagnosed. Retrying a deterministic failure just fails more slowly.