2022/05/27 by Sabina J. Sloman, Sloman, Sabina J., Daniel M. Oppenheimer +5 · 2 citations
Biochemistry, Genetics and Molecular Biology · Chemistry · Decision Sciences · #Analytical Chemistry and Chromatography #FOS: Computer and information sciences #Machine Learning (stat.ML) #Methodology (stat.ME) #Optimal Experimental Design Methods #Receptor Mechanisms and Signaling
paper · pdf · doi:10.48550/arxiv.2205.13698
openalex publication_date 2022/05/27 · openalex created_date 2022/06/13 · openalex updated_date 2026/07/28
Bayesian adaptive experimental design is a form of active learning, which chooses samples to maximize the information they give about uncertain parameters. Prior work has shown that other forms of active learning can suffer from active learning bias, where unrepresentative sampling leads to inconsistent parameter estimates. We show that active learning bias can also afflict Bayesian adaptive experimental design, depending on model misspecification. We analyze the case of estimating a linear model, and show that worse misspecification implies more severe active learning bias. At the same time, model classes incorporating more "noise" - i.e., specifying higher inherent variance in observations - suffer less from active learning bias. Finally, we demonstrate empirically that insights from the linear model can predict the presence and degree of active learning bias in nonlinear contexts, namely in a (simulated) preference learning experiment.