2014/10/14 by Peiris, Hiranya V.
#Cosmology and Nongalactic Astrophysics (astro-ph.CO) #FOS: Physical sciences #Instrumentation and Methods for Astrophysics (astro-ph.IM)
paper · doi:10.48550/arxiv.1410.3837
Anomalies drive scientific discovery -- they are associated with the cutting edge of the research frontier, and thus typically exploit data in the low signal-to-noise regime. In astronomy, the prevalence of systematics --- both "known unknowns" and "unknown unknowns" --- combined with increasingly large datasets, the widespread use of ad hoc estimators for anomaly detection, and the "look-elsewhere" effect, can lead to spurious false detections. In this informal note, I argue that anomaly detection leading to discoveries of new physics requires a combination of physical understanding, careful experimental design to avoid confirmation bias, and self-consistent statistical methods. These points are illustrated with several concrete examples from cosmology.