Rank Analysis of Incomplete Block Designs: I. The Method of Paired Comparisons

1952/12/01 by Ralph Allan Bradley, Ralph A. Bradley, Milton E. Terry · 2,429 citations
Decision Sciences · Mathematics · #Arithmetic #Block (permutation group theory) #Combinatorics #Mathematics #Optimal Experimental Design Methods #Paired comparison #Rank (graph theory) #Statistical Methods in Clinical Trials #Statistics

paper · doi:10.2307/2334029

published in Biometrika 39(3/4), 324 (Oxford University Press)

openalex publication_date 1952/12/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/03

Cited by

• A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions
• Polarization over the priority of political problems
• Worst-case vs Average-case Design for Estimation from Fixed Pairwise Comparisons
• Deep Bayesian Reward Learning from Preferences
• Optimal Paired Comparison Experiments for Second-Order Interactions
• Methods for Ordinal Peer Grading
• Nonparametric Estimation in the Dynamic Bradley-Terry Model
• Deep Image Harmonization
• Non-Parametric Bayesian Inference for Partial Orders with Ties from Rank Data observed with Mallows Noise
• Accelerated Experimental Design for Pairwise Comparisons
• A Theory of Tournament Representations
• A Reduced Basis Decomposition Approach to Efficient Data Collection in Pairwise Comparison Studies
• Pairwise Comparison for Bias Identification and Quantification
• Joint Progression Modeling (JPM): A Probabilistic Framework for Mixed-Pathology Progression
• Constructive Preference Elicitation by Setwise Max-margin Learning
• Reference-aware image harmonization
• Learning Temporal Coherence via Self-Supervision for GAN-based Video Generation
• EnlightenGAN: Deep Light Enhancement without Paired Supervision
• Combinatorial Optimization using Comparison Oracles
• The existence of maximum likelihood estimates in the Bradley-Terry model and its extensions
• Exploring question answering: metric analysis and evaluation framework for enhanced interpretability
• Statistical ranking and combinatorial Hodge theory
• Ties in Paired-Comparison Experiments: A Generalization of the Bradley-Terry Model
• The Bradley-Terry Stochastic Block Model
• Optimal rates for ranking a permuted isotonic matrix in polynomial time
• Learning from Comparisons and Choices
• Scaling Open-Ended Survey Responses Using LLM-Paired Comparisons
• Phase Transitions in Approximate Ranking
• Aligning Large Language Models With Human Feedback: Mathematical foundations and algorithm design [Special Issue on the Mathematics of Deep Learning]
• Preference-based Reinforcement Learning with Finite-Time Guarantees
• Fitness incentives to male fighters undermine fighting performance in intergroup contests
• Default Bayes factors for ANOVA designs
• Deep reinforcement learning from human preferences
• TOPIQ: A Top-Down Approach From Semantics to Distortions for Image Quality Assessment
• Graph Resistance and Learning from Pairwise Comparisons
• Simple, Robust and Optimal Ranking from Pairwise Comparisons
• Learning to Blindly Assess Image Quality in the Laboratory and Wild
• Legislators’ sentiment analysis supervised by legislators
• APRIL: Interactively Learning to Summarise by Combining Active Preference Learning and Reinforcement Learning
• Spectral MLE: Top-K Rank Aggregation from Pairwise Comparisons
• Direct Preference Optimization with Unobserved Preference Heterogeneity: The Necessity of Ternary Preferences
• PCMC-Net: Feature-based Pairwise Choice Markov Chains
• The effect of safety attire on perceptions of cyclist dehumanisation
• Learning Combinatorial Functions from Pairwise Comparisons
• Ranking Recovery from Limited Comparisons using Low-Rank Matrix\n Completion
• Few-shot multi-token DreamBooth with LoRa for style-consistent character generation
• What Makes a Visualization Image Complex?
• On the Role of Difficult Prompts in Self-Play Preference Optimization
• Minimax Rates and Efficient Algorithms for Noisy Sorting
• What can I do here? Leveraging Deep 3D saliency and geometry for fast and scalable multiple affordance detection
• Scaling Open-ended Survey Responses Using LLM-Paired Comparisons
• A theoretical guarantee for SyncRank
• Failure Modes of Maximum Entropy RLHF
• PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-training
• Estimating Robot Strengths with Application to Selection of Alliance Members in FIRST Robotics Competitions
• Beyond Brightening Low-light Images
• A State-Space Perspective on Modelling and Inference for Online Skill Rating
• Comparative judgement for experimental philosophy: A method for assessing ordinary meaning in vehicles in the park cases
• A Ranking Model Motivated by Nonnegative Matrix Factorization with Applications to Tennis Tournaments
• Making Images Real Again: A Comprehensive Survey on Deep Image Composition
• The Eval4NLP Shared Task on Explainable Quality Estimation: Overview and Results
• Learning to Caricature via Semantic Shape Transform
• Understanding and Pushing the Limits of the Elo Rating Algorithm
• Data augmentation and image understanding
• Preference-based Interactive Multi-Document Summarisation
• Efficient Online Scalar Annotation with Bounded Support
• Clustering and Inference From Pairwise Comparisons
• Interactive Text Ranking with Bayesian Optimisation: A Case Study on Community QA and Summarisation
• Improved Evaluation and Generation Of Grid Layouts Using Distance Preservation Quality and Linear Assignment Sorting
• Beyond Correctness: Harmonizing Process and Outcome Rewards through RL Training
• Vevo2: A Unified and Controllable Framework for Speech and Singing Voice Generation
• EZ-Sort: Efficient Pairwise Comparison via Zero-Shot CLIP-Based Pre-Ordering and Human-in-the-Loop Sorting
• VERIRL: Boosting the LLM-based Verilog Code Generation via Reinforcement Learning
• Neural Image Beauty Predictor Based on Bradley-Terry Model
• Rank Centrality: Ranking from Pair-wise Comparisons
• Breaking the 1/√(n) Barrier: Faster Rates for Permutation-based Models in Polynomial Time
• As you like it: Localization via paired comparisons
• Strategy for Boosting Pair Comparison and Improving Quality Assessment Accuracy
• Cold-Start Active Preference Learning in Socio-Economic Domains
• SGPO: Self-Generated Preference Optimization based on Self-Improver
• A Roadmap for Robust End-to-End Alignment
• Spectral Ranking using Seriation
• Active embedding search via noisy paired comparisons
• One-vs-Each Approximation to Softmax for Scalable Estimation of Probabilities
• Gaussian Process Priors for Dynamic Paired Comparison Modelling
• A continuous rating method for preferential voting
• Deep Image Harmonization by Bridging the Reality Gap
• Ranking Creative Language Characteristics in Small Data Scenarios
• Food and Food-Odor Preferences in Dogs: A Pilot Study
• EnlightenGAN: Deep Light Enhancement Without Paired Supervision
• Active Ranking from Pairwise Comparisons and when Parametric Assumptions Don't Help
• Hybrid Machine Learning Forecasts for the UEFA EURO 2020
• Hybrid Machine Learning Forecasts for the FIFA Women's World Cup 2019
• SAGE: A Visual Language Model for Anomaly Detection via Fact Enhancement and Entropy-aware Alignment
• Protest Activity Detection and Perceived Violence Estimation from Social Media Images
• Deep Research Comparator: A Platform For Fine-grained Human Annotations of Deep Research Agents
• SAVOIAS: A Diverse, Multi-Category Visual Complexity Dataset
• Tied Pools and Drawn Games
• LEHA-CVQAD: Dataset To Enable Generalized Video Quality Assessment of Compression Artifacts
• On the Sample Complexity of Rank Regression from Pairwise Comparisons
• Oneshot Differentially Private Top-k Selection
• Reward-rational (implicit) choice: A unifying formalism for reward learning
• Requirements Elicitation Follow-Up Question Generation
• Who's Sorry Now: User Preferences Among Rote, Empathic, and Explanatory Apologies from LLM Chatbots
• Improving Consistency in Vehicle Trajectory Prediction Through Preference Optimization
• Reasoning or Not? A Comprehensive Evaluation of Reasoning LLMs for Dialogue Summarization
• A Normal Approximation Method for Statistics in Knockouts
• Capturing Variation and Uncertainty in Human Judgment
• Less but Better: Generalization Enhancement of Ordinal Embedding via Distributional Margin
• JointRank: Rank Large Set with Single Pass
• Aggregation of pairwise comparisons with reduction of biases
• Preference Completion: Large-scale Collaborative Ranking from Pairwise Comparisons
• Reward learning from human preferences and demonstrations in Atari
• CrowdGrader: Crowdsourcing the Evaluation of Homework Assignments
• A Multiresolution Analysis Framework for the Statistical Analysis of Incomplete Rankings
• Context Matters: Learning Generalizable Rewards via Calibrated Features
• SENIOR: Efficient Query Selection and Preference-Guided Exploration in Preference-based Reinforcement Learning
• Reward Model Interpretability via Optimal and Pessimal Tokens
• Benchmarking Single Image Dehazing and Beyond
• Adversarial Top-K Ranking
• Scalable and Efficient Comparison-based Search without Features
• Modelling Competitive Sports: Bradley-Terry-Élő Models for Supervised and On-Line Learning of Paired Competition Outcomes
• Rating competitors in games with strength-dependent tie probabilities
• OneSug: The Unified End-to-End Generative Framework for E-commerce Query Suggestion
• Dynamic Ranking with the BTL Model: A Nearest Neighbor based Rank Centrality Method
• Learning to Automate Chart Layout Configurations Using Crowdsourced Paired Comparison
• EOMM: An Engagement Optimized Matchmaking Framework
• Status hierarchy and group cooperation: A generalized model
• Davidson-Luce model for multi-item choice with ties
• Robust Consensus in Ranking Data Analysis: Definitions, Properties and Computational Issues
• Simple estimation of hierarchical positions and uncertainty in networks of asymmetric interactions
• DumbleDR: Predicting User Preferences of Dimensionality Reduction Projection Quality
• BargainNet: Background-Guided Domain Translation for Image Harmonization
• DeblurGAN-v2: Deblurring (Orders-of-Magnitude) Faster and Better
• Tournament of Prompts: Evolving LLM Instructions Through Structured Debates and Elo Ratings
• "Are you sure?": Preliminary Insights from Scaling Product Comparisons to Multiple Shops
• Spectral Survival Analysis
• Ludometrics: Luck, and How to Measure It
• NTIRE 2025 Challenge on UGC Video Enhancement: Methods and Results
• The "No Justice in the Universe" phenomenon: why honesty of effort may not be rewarded in tournaments
• A Nearly Instance Optimal Algorithm for Top-k Ranking under the Multinomial Logit Model
• NTIRE 2025 Challenge on Video Quality Enhancement for Video Conferencing: Datasets, Methods and Results
• Do Large Language Models (Really) Need Statistical Foundations?
• Skill Preferences: Learning to Extract and Execute Robotic Skills from Human Feedback
• Ranking Participants in Tournaments by means of Rating Functions
• Maximum entropy models for generation of expressive music
• Learning an arbitrary mixture of two multinomial logits
• Accelerated MM Algorithms for Ranking Scores Inference from Comparison Data
• Deep Networks for Image Super-Resolution with Sparse Prior
• Support vector comparison machines
• Evaluating Visual Properties via Robust HodgeRank
• Fast Text-to-Audio Generation with Adversarial Post-Training
• Fundamental Limits of Testing the Independence of Irrelevant Alternatives in Discrete Choice
• Configuration Space Metrics
• Learning Guarantee of Reward Modeling Using Deep Neural Networks
• TREND: Tri-teaching for Robust Preference-based Reinforcement Learning with Demonstrations
• Abductive Reasoning as Self-Supervision for Common Sense Question Answering
• Tournesol: A quest for a large, secure and trustworthy database of reliable human judgments
• On a statistical approach to mate choices in reproduction
• A Hyperbolic Cosine Latent Trait Model For Unfolding Dichotomous Single-Stimulus Responses
• Mathematical methods of reinforcement learning
• Rankings from Paired Comparisons
• Maximum-likelihood paired comparison rankings
• Human-Robot Handshaking: A Review
• Quantifying Response Dependence Between Two Dichotomous Items Using the Rasch Model
• Poisoning Attack Against Estimating From Pairwise Comparisons
• Perceptual evaluation of liquid simulation methods
• PAC Ranking from Pairwise and Listwise Queries: Lower Bounds and Upper Bounds
• SPIE: Semantic and Structural Post-Training of Image Editing Diffusion Models with AI feedback
• FLoRA: Sample-Efficient Preference-based RL via Low-Rank Style Adaptation of Reward Functions
• Network-based ranking in social systems: three challenges
• Assessment of examiner leniency and stringency ('hawk-dove effect') in the MRCP(UK) clinical examination (PACES) using multi-facet Rasch modelling. [europepmc]
• An absolute interval scale of order for point patterns. [europepmc]
• Computing a ranking network with confidence bounds from a graph-based Beta random field. [europepmc]
• Do Humans Really Prefer Semi-open Natural Landscapes? A Cross-Cultural Reappraisal. [europepmc]
• How does the method change what we measure? Comparing virtual reality and text-based surveys for the assessment of moral decisions in traffic dilemmas. [europepmc]
• Rank orders and signed interactions in evolutionary biology. [europepmc]
• Visual Perception of Moisture Is a Pathogen Detection Mechanism of the Behavioral Immune System. [europepmc]
• A Bayesian Random Block Item Response Theory Model for Forced-Choice Formats. [europepmc]
• Perceptual Evaluation of Signal-to-Noise-Ratio-Aware Dynamic Range Compression in Hearing Aids. [europepmc]
• Data synthesis for crop variety evaluation. A review. [europepmc]
• Comparison of Full-Reference Image Quality Models for Optimization of Image Processing Systems. [europepmc]
• Male genital lobe morphology affects the chance to copulate in Drosophila pachea. [europepmc]
• Influence of competition and intraguild predation between two candidate biocontrol parasitoids on their potential impact against Harrisia cactus mealybug, Hypogeococcus sp. (Hemiptera: Pseudococcidae). [europepmc]
• Estimation of final standings in football competitions with a premature ending: the case of COVID-19. [europepmc]
• Human Preferences for Robot Eye Gaze in Human-to-Robot Handovers. [europepmc]
• A study of forecasting tennis matches via the Glicko model. [europepmc]
• Which clinical research questions are the most important? Development and preliminary validation of the Australia & New Zealand Musculoskeletal (ANZMUSC) Clinical Trials Network Research Question Importance Tool (ANZMUSC-RQIT). [europepmc]
• gosset: An R package for analysis and synthesis of ranking data in agricultural experimentation. [europepmc]
• Standardized assessment of vascular reconstruction kernels in photon-counting CT angiographies of the leg using a continuous extracorporeal perfusion model. [europepmc]
• Comparison of ultrahigh and standard resolution photon-counting CT angiography of the femoral arteries in a continuously perfused in vitro model. [europepmc]
• Amplitude Compression for Preventing Rollover at Above-Conversational Speech Levels. [europepmc]
• A novel comparative study of NNAR approach with linear stochastic time series models in predicting tennis player's performance. [europepmc]
• The tricot approach: an agile framework for decentralized on-farm testing supported by citizen science. A retrospective. [europepmc]
• Opportunities and challenges of diffusion models for generative AI. [europepmc]
• Advancing preference testing in humans and animals. [europepmc]