On a Measure of the Information Provided by an Experiment

1956/12/01 by D. V. Lindley · 1,522 citations
Computer Science · Decision Sciences · Mathematics · #Advanced Multi-Objective Optimization Algorithms #Artificial intelligence #Computer science #Data mining #Distribution (mathematics) #Mathematical analysis #Mathematics #Measure (data warehouse) #Object (grammar) #Optimal Experimental Design Methods #Probability distribution #Probability measure #Space (punctuation) #Statistics

paper · pdf · doi:10.1214/aoms/1177728069

published in The Annals of Mathematical Statistics 27(4), 986-1005 (Institute of Mathematical Statistics)

openalex publication_date 1956/12/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/01

Abstract

A measure is introduced of the information provided by an experiment. The measure is derived from the work of Shannon [10] and involves the knowledge prior to performing the experiment, expressed through a prior probability distribution over the parameter space. The measure is used to compare some pairs of experiments without reference to prior distributions; this method of comparison is contrasted with the methods discussed by Blackwell. Finally, the measure is applied to provide a solution to some problems of experimental design, where the object of experimentation is not to reach decisions but rather to gain knowledge about the world.

Cited by

• A stimulus-computable rational model of visual habituation in infants and adults
• Generalization guides human exploration in vast decision spaces
• Taking the Human Out of the Loop: A Review of Bayesian Optimization
• Seeking New Information With Old Questions: Children and Adults Reuse and Recombine Concepts From Prior Questions
• Coherent Infomax as a Computational Goal for Neural Systems
• Specimen design for material parameter identification using topology optimization
• Deep Adaptive Bayesian Screening
• Efficient Bayesian Optimal Experimental Design for Expensive Computational Models over Finite Design Sets
• Expected Free Energy as Belief-Dependent Utility for rho-POMDPs
• Optimal Design for Generalized Progressive Hybrid Censored Data via Constrained, Unconstrained, Compound, and Minimax Optimization
• Can In-Context Learning Support Intrinsic Curiosity?
• Measurement-limited learning of conformational heterogeneity in cryo-electron microscopy
• BED-LLM: Intelligent Information Gathering with LLMs and Bayesian Experimental Design
• Intrinsically-Motivated Humans and Agents in Open-World Exploration
• Distributions of the Kullback–Leibler divergence with applications
• A layered multiple importance sampling scheme for focused optimal Bayesian experimental design
• Icebreaker: Element-wise Active Information Acquisition with Bayesian\n Deep Latent Gaussian Model
• Planning to Be Surprised: Optimal Bayesian Exploration in Dynamic Environments
• JADAI: Jointly Amortizing Adaptive Design and Bayesian Inference
• Active Learning of Model Discrepancy with Bayesian Experimental Design
• Cost-Aware Recovery-Pathway Identification and Bayesian Optimization for Autonomous Materials Discovery
• An Optimal Policy for Learning Controllable Dynamics by Exploration
• Lifting Biomolecular Data Acquisition
• PIS: A Generalized Physical Inversion Solver for Arbitrary Sparse Observations via Set Conditioned Flow Matching
• PySensors: A Python Package for Sparse Sensor Placement
• Identifying Neutron Sources using Recoil and Time-of-Flight Spectroscopy
• Generalised free energy and active inference
• The Sequential Nature of Science: Quantifying Learning from a Sequence of Studies
• Distinguishability and Accessible Information in Quantum Theory
• Variational Bayesian Optimal Experimental Design
• Reinforcement Learning through Active Inference
• Probabilities Are All You Need: A Probability-Only Approach to Uncertainty Estimation in Large Language Models
• Cryptographic Distinguishability Measures for Quantum Mechanical States
• An Approximate Bayesian Approach to Optimal Input Signal Design for System Identification
• Online Bayesian Experimental Design for Partially Observed Dynamical Systems
• Active Learning with Task-Driven Representations for Messy Pools
• Planning to Explore via Self-Supervised World Models
• Information-Based Objective Functions for Active Data Selection
• The Inherent Normativity of Concepts
• Statistics in service of metascience: Measuring replication distance with reproducibility rate
• Calibration of climate model parameterizations using Bayesian experimental design
• Active inference on discrete state-spaces: a synthesis
• Noise Contrastive Priors for Functional Uncertainty
• ShaplEIG: Bayesian Experimental Design for Shapley Value Estimation
• Deep Adaptive Design: Amortizing Sequential Bayesian Experimental Design
• Accelerated Learning on Large Scale Screens using Generative Library Models
• Optimal experimental design via Bayesian optimization: active causal structure learning for Gaussian process networks
• Active Learning: Problem Settings and Recent Developments
• Shoot First, Ask Questions Later? Building Rational Agents that Explore and Act Like People
• ASBI: Leveraging Informative Real-World Data for Active Black-Box Simulator Tuning
• A Geometric Approach to Optimal Experimental Design
• Information Directed Reward Learning for Reinforcement Learning
• Statistics in Service of Metascience: Measuring Replication Distance with Reproducibility Rate
• Bayesian Decision Making around Experts
• Generating Meaning: Active Inference and the Scope and Limits of Passive AI
• Incorporating Expert Knowledge into Bayesian Causal Discovery of Mixtures of Directed Acyclic Graphs
• Hybrid-MST: A Hybrid Active Sampling Strategy for Pairwise Preference Aggregation
• Approximation of differential entropy in Bayesian optimal experimental design
• BALLAST: Bayesian Active Learning with Look-ahead Amendment for Sea-drifter Trajectories under Spatio-Temporal Vector Fields
• ActiveCQ: Active Estimation of Causal Quantities
• Composing trajectories for rapid inference of navigational goals
• Optimal Stopping for Sequential Bayesian Experimental Design
• Active Testing: Sample-Efficient Model Evaluation
• LLMs for Bayesian Optimization in Scientific Domains: Are We There Yet?
• A decision-theoretic framework for uncertainty quantification in epidemiological modelling
• Sequential Bayesian optimal experimental design via approximate dynamic programming
• Generating meaning: active inference and the scope and limits of passive AI
• Learning the relationship between operating condition and flame response from acoustic data
• On Mutual Information Neural Estimation for Localization
• Action and Perception as Divergence Minimization
• When three experiments are better than two: Avoiding intractable correlated aleatoric uncertainty by leveraging a novel bias--variance tradeoff
• Convex Cost of Information via Statistical Divergence
• Reducing Sensing Time through Offline Experimental Design for Nuclear Spin Detection
• Quantifying How Much Has Been Learned from a Research Study
• Surrogate-based Bayesian calibration methods for climate models: a comparison of traditional and non-traditional approaches
• Bayesian experimental design for the active nitridation of graphite by atomic nitrogen
• Prediction-Oriented Subsampling from Data Streams
• Hamiltonian parameter inference from resonant inelastic x-ray scattering with active learning
• Noisy Adaptive Group Testing using Bayesian Sequential Experimental Design
• A Unified Stochastic Gradient Approach to Designing Bayesian-Optimal Experiments
• Active embedding search via noisy paired comparisons
• Sampling theory for spatial field sensing: Application to electro- and magnetoencephalography
• Step-DAD: Semi-Amortized Policy-Based Bayesian Experimental Design
• Modern Bayesian Experimental Design
• Spatial sampling of MEG and EEG revisited: From spatial-frequency spectra to model-informed sampling
• Test Distribution-Aware Active Learning: A Principled Approach Against Distribution Shift and Outliers
• Self‐Directed Learning Favors Local, Rather Than Global, Uncertainty
• The Roles of Low-Noise Stations, Arrays and Ocean-Bottom Seismometers in Monitoring UK Offshore Seismicity associated with Subsurface Storage of Carbon Dioxide
• Evaluating Agents without Rewards
• Resource depletion accelerates rate learning but not composition learning in patch foraging
• Basic Experiment Planning via Information Metrics: the RoboMendel Problem
• Goal-Oriented Sequential Bayesian Experimental Design for Causal Learning
• Learning controllable dynamics through informative exploration
• Bayesian Optimal Design of Experiments For Inferring The Statistical Expectation Of A Black-Box Function
• Lookahead and Hybrid Sample Allocation Procedures for Multiple Attribute Selection Decisions
• GO-PRE: Goal-Oriented Next-Best-View Selection via Predictive Rendering Entropy for Active 3D Reconstruction
• Backward-Compatible Prediction Updates: A Probabilistic Approach
• On Efficient Bayesian Exploration in Model-Based Reinforcement Learning
• Targeted Active Learning for Bayesian Decision-Making
• A Game Theoretic Approach to Quantum Information
• Extending Epistemic Uncertainty Beyond Parameters Would Assist in Designing Reliable LLMs
• Designing ecosystems of intelligence from first principles
• Model-Based Active Exploration
• Representative, Informative, and De-Amplifying: Requirements for Robust Bayesian Active Learning under Model Misspecification
• Knowledge Adaptation as Posterior Correction
• Distributional Training Data Attribution: What do Influence Functions Sample?
• ALINE: Joint Amortization for Bayesian Inference and Active Data Acquisition
• Fusion of heterogeneous data for robust degradation prognostics
• Feedback Coding for Active Learning
• An approach for finding fully Bayesian optimal designs using normal-based approximations to loss functions
• Near-Optimal Bayesian Active Learning with Noisy Observations
• Bayesian Active Learning for Classification and Preference Learning
• Are Large Language Models Reliable AI Scientists? Assessing Reverse-Engineering of Black-Box Systems
• A Practical & Unified Notation for Information-Theoretic Quantities in ML
• VAEM: a Deep Generative Model for Heterogeneous Mixed Type Data
• The Lindley paradox: The loss of resolution in Bayesian inference
• Modeling a Sensor to Improve its Efficacy
• Quantifying Privacy in Nuclear Warhead Authentication Protocols
• Accelerating Bayesian Optimal Experimental Design via Local Radial Basis Functions: Application to Soft Material Characterization
• Vendi Information Gain: An Alternative To Mutual Information For Science And Machine Learning
• Optimal Experimental Design for Mathematical Models of Hematopoiesis
• Guiding Data Collection via Factored Scaling Curves
• Some models are useful, but how do we know which ones? Towards a unified Bayesian model taxonomy
• The Advantage of Cross Entropy over Entropy in Iterative Information Gathering
• On minimum information prior distributions
• ATLAS: Active Theory Learning for Automated Science
• Diffusion Approximations for a Class of Sequential Testing Problems
• Rethinking the Scientific Method: An Introduction to Bayesian Epistemology
• Game theory, maximum entropy, minimum discrepancy and robust Bayesian decision theory
• Reference Analysis
• Ask Early, Ask Late, Ask Right: When Does Clarification Timing Matter for Long-Horizon Agents?
• Bayesian Experimental Design for Model Discrepancy Calibration: An Auto-Differentiable Ensemble Kalman Inversion Approach
• Estimating the Conditional Forecast-Revision Scale in Sequential Models: Local-Smoothing Limits, Matched Models, and Cost--Accuracy Trade-offs
• Maximizing the information gain of a single ion microscope using bayes experimental design
• Maximizing the Information Content of Experiments in Systems Biology
• Estimating Expected Information Gains for Experimental Designs With Application to the Random Fatigue-Limit Model
• Bayesian Genome- and Epigenome-Wide Association Studies with Gene Level Dependence
• GRADIENT-BASED STOCHASTIC OPTIMIZATION METHODS IN BAYESIAN EXPERIMENTAL DESIGN
• On the Heisenberg principle at macroscopic scales: understanding\n classical negative information. Towards a general physical theory of\n information
• Unequal Error Protection Querying Policies for the Noisy 20 Questions Problem
• Maximizing information gain for the characterization of biomolecular circuits
• Applying Information Theory to Design Optimal Filters for Photometric Redshifts
• Quantifying Selective Reporting and the Proteus Phenomenon for Multiple Datasets with Similar Bias
• Intelligent data collection for network discrimination in material flow analysis using Bayesian optimal experimental design
• Gradient-Free Sequential Bayesian Experimental Design via Interacting Particle Systems
• Beyond Relevance: Bayesian Evidence Acquisition for Agentic Whole-Slide Image Reasoning
• Bayesian optimal experimental design with Wasserstein information criteria
• The Interface Between Statistics and Philosophy of Science
• Bayesian experimental design [wikipedia]
• Curiosity driven reinforcement learning for motion planning on humanoids. [europepmc]
• A Bayesian active learning strategy for sequential experimental design in systems biology. [europepmc]
• Bayesian action&perception: representing the world in the brain. [europepmc]
• Wiki surveys: open and quantifiable social data collection. [europepmc]
• How basic-level objects facilitate question-asking in a categorization task. [europepmc]
• A consensus layer V pyramidal neuron can sustain interpulse-interval coding. [europepmc]
• Computational Neuropsychology and Bayesian Inference. [europepmc]
• Cost⁻Benefit Optimization of Structural Health Monitoring Sensor Networks. [europepmc]
• Semantic information, autonomous agency and non-equilibrium statistical physics. [europepmc]
• Dynamic Causal Modelling of Active Vision. [europepmc]
• Generalised free energy and active inference. [europepmc]
• Value of Information: Sensitivity Analysis and Research Design in Bayesian Evidence Synthesis. [europepmc]
• Accelerated knowledge discovery from omics data by optimal experimental design. [europepmc]
• Approximate Entropy and Sample Entropy: A Comprehensive Tutorial. [europepmc]
• Generative Models for Active Vision. [europepmc]
• Second waves, social distancing, and the spread of COVID-19 across the USA. [europepmc]
• Children's exploratory play tracks the discriminability of hypotheses. [europepmc]
• Decision Models and Technology Can Help Psychiatry Develop Biomarkers. [europepmc]
• Resilience and active inference. [europepmc]
• Reward Maximization Through Discrete Active Inference. [europepmc]
• Cognitive effort and active inference. [europepmc]
• Cultivating creativity: predictive brains and the enlightened room problem. [europepmc]
• Forgetting ourselves in flow: an active inference account of flow states and how we experience ourselves within them. [europepmc]
• Detecting hallucinations in large language models using semantic entropy. [europepmc]
• SPM-30 years and beyond. [europepmc]

Related

• Designing Test Information and Test Information in Design
• On Experiments
• Basic Experiment Planning via Information Metrics: the RoboMendel Problem
• Using Information Theory to Measure Psychophysical Performance
• On the Rate of Gain of Information