A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting

1997/08/01 by Yoav Freund, Robert E Schapire, Robert E. Schapire · 284 citations
Decision Sciences · Computer Science · #Advanced Bandit Algorithms Research #Machine Learning and Algorithms #Auction Theory and Applications

paper · doi:10.1006/jcss.1997.1504

Cited by

• Wrappers for feature subset selection
• Enhanced aspect-based sentiment analysis models with progressive self-supervised attention learning
• Data imbalance in classification: Experimental evaluation
• Automated Scoring of Figural Tests of Creativity with Computer Vision
• Boosted key-frame selection and correlated pyramidal motion-feature representation for human action recognition
• Presettlement Tree Distributions and Forest Types of Northeast Ohio, <scp>USA</scp>, Mapped With Species Distribution Models
• Experimental review and recent advances in deep learning techniques for solar irradiance forecasting and prediction
• A Framework for Predicting Neurofeedback Treatment Response in ADHD Using EEG Functional Connectivity and Genetic Algorithm-Driven Channel Selection
• The composite absolute penalties family for grouped and hierarchical variable selection
• Least angle regression
• The Nonstochastic Multiarmed Bandit Problem
• Prediction, interpolation and extrapolation of the bonding strength between FRP bars and UHPC with machine learning
• Learning Instance Occlusion for Panoptic Segmentation
• Every Corporation Owns Its Image: Corporate Credit Ratings via Convolutional Neural Networks
• Machine learning, phase stability, and disorder with the Automatic Flow Framework for Materials Discovery
• Large Scale Strongly Supervised Ensemble Metric Learning, with Applications to Face Verification and Retrieval
• Activity Detection from Encrypted Remote Desktop Protocol Traffic
• An Induced Multi-Relational Framework for Answer Selection in Community Question Answer Platforms
• On the Convergence Properties of Optimal AdaBoost
• A combinatorial conjecture from PAC-Bayesian machine learning
• Empirical Models for Multidimensional Regression of Fission Systems
• Rapid Learning with Stochastic Focus of Attention
• Better Boosting with Bandits for Online Learning
• Large Margin Distribution Machine
• Generalized Boosting Algorithms for Convex Optimization
• Stable Feature Selection for Biomarker Discovery
• Learning Shallow Detection Cascades for Wearable Sensor-Based Mobile Health Applications
• A Regularized Method for Selecting Nested Groups of Relevant Genes from Microarray Data
• Online Learning via the Differential Privacy Lens
• MonoStream: A Minimal-Hardware High Accuracy Device-free WLAN Localization System
• Supervised Machine Learning Techniques: An Overview with Applications to Banking
• Fused DNN: A deep neural network fusion approach to fast and robust pedestrian detection
• A Methodological Review of Visual Road Recognition Procedures for Autonomous Driving Applications
• Enhash: A Fast Streaming Algorithm For Concept Drift Detection
• Misclassification cost-sensitive ensemble learning: A unifying framework
• Toward a Classification of Finite Partial-Monitoring Games
• A Polynomial Time Algorithm for Learning Halfspaces with Tsybakov Noise
• Rethinking Generative Mode Coverage: A Pointwise Guaranteed Approach
• Features in Concert: Discriminative Feature Selection meets Unsupervised Clustering
• On the Current State of Research in Explaining Ensemble Performance Using Margins
• The Complexity of Adversarially Robust Proper Learning of Halfspaces with Agnostic Noise
• TBM Penetration Rate Prediction in Himalayan Geology Using Machine Learning (ML) Techniques
• Anisotropic Proximal Gradient
• The Dynamics of AdaBoost Weights Tells You What's Hard to Classify
• A Tale of Three Probabilistic Families: Discriminative, Descriptive and Generative Models
• GOOWE: Geometrically Optimum and Online-Weighted Ensemble Classifier for Evolving Data Streams
• Calibration with Changing Checking Rules and Its Application to Short-Term Trading
• Learning Gaussian Graphical Models via Multiplicative Weights
• Fundamental Limits of Online and Distributed Algorithms for Statistical Learning and Estimation
• How to boost autoencoders?
• Selective Ensembles for Consistent Predictions
• On quantum ensembles of quantum classifiers
• Pre-trained Language Model based Ranking in Baidu Search
• Branching fraction measurement of the decay <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"><mml:msup><mml:mi>B</mml:mi><mml:mo>+</mml:mo></mml:msup><mml:mo stretchy="false">→</mml:mo><mml:mi>ψ</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mn>2</mml:mn><mml:mi>S</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mi>ϕ</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mn>1020</mml:mn><mml:mo stretchy="false">)</mml:mo><mml:msup><mml:mi>K</mml:mi><mml:mo>+</mml:mo></mml:msup></mml:math>
• Object Detection with Deep Learning: A Review
• Complexity Theoretic Limitations on Learning Halfspaces
• A parameter-free hedging algorithm
• Selecting a number of voters for a voting ensemble
• Machine learning based hyperspectral image analysis: A survey
• Predicting building age from urban form at large scale
• Deep learning approaches show promise for predicting childhood malnutrition: A comparative study with traditional machine learning methods using survey data
• Directional convergence and alignment in deep learning
• Ensemble Fine-tuned mBERT for Translation Quality Estimation
• A Bayesian reassessment of nearest-neighbour classification
• Feature Extraction for Machine Learning Based Crackle Detection in Lung Sounds from a Health Survey
• A Comprehensive Investigation of the Role of Imbalanced Learning for Software Defect Prediction
• A Survey on Malware Detection Using Data Mining Techniques
• Verifying Robustness of Gradient Boosted Models
• Local-Aggregate Modeling for Big-Data via Distributed Optimization: Applications to Neuroimaging
• Robust Real-Time Face Detection
• Sample Compression for Real-Valued Learners
• Causal aggregation: estimation and inference of causal effects by\n constraint-based data fusion
• Deep Differential Recurrent Neural Networks
• A Novel Multiple Classifier Generation and Combination Framework Based on Fuzzy Clustering and Individualized Ensemble Construction
• Implementation of machine-learning classification in remote sensing: an applied review
• Cronus: Robust and Heterogeneous Collaborative Learning with Black-Box Knowledge Transfer
• Automatic Clone Recommendation for Refactoring Based on the Present and the Past
• Estimation of Causal Effects of Multiple Treatments in Observational Studies with a Binary Outcome
• Learning Nonlinear Functions Using Regularized Greedy Forest
• Machine learning and the physical sciences
• Quantum Cognitively Motivated Decision Fusion for Video Sentiment Analysis
• Closed-Loop Neural Prostheses with On-Chip Intelligence: A Review and A Low-Latency Machine Learning Model for Brain State Detection
• Chaos of Learning Beyond Zero-sum and Coordination via Game Decompositions
• Computational Ceramicology
• The Implicit Regularization of Ordinary Least Squares Ensembles
• Boosting in the Presence of Massart Noise
• Active Learning with Expert Advice
• A Bayesian Approach for Online Classifier Ensemble
• ENSEMBLE OF SOFTWARE DEFECT PREDICTORS: AN AHP-BASED EVALUATION METHOD
• Bagging is an Optimal PAC Learner
• An Optimal Algorithm for Online Unconstrained Submodular Maximization
• Learning Fast Sample Re-weighting Without Reward Data
• Softplus Regressions and Convex Polytopes
• Quantum Inspired Adaptive Boosting
• MESA: Boost Ensemble Imbalanced Learning with MEta-SAmpler
• Quantifying Explainability in NLP and Analyzing Algorithms for Performance-Explainability Tradeoff
• Boosting with Lexicographic Programming: Addressing Class Imbalance\n without Cost Tuning
• Observation of the Doubly Charmed Baryon <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"> <mml:mrow> <mml:msubsup> <mml:mrow> <mml:mi mathvariant="normal">Ξ</mml:mi> </mml:mrow> <mml:mrow> <mml:mi>c</mml:mi> <mml:mi>c</mml:mi> </mml:mrow> <mml:mrow> <mml:mo>+</mml:mo> </mml:mrow> </mml:msubsup> </mml:mrow> </mml:math> with the LHCb Run 3 Detector
• Functional Frank-Wolfe Boosting for General Loss Functions
• Streaming Active Deep Forest for Evolving Data Stream Classification
• Fast Convolutional Neural Network Training Using Selective Data Sampling: Application to Hemorrhage Detection in Color Fundus Images
• Improved Second-Order Bounds for Prediction with Expert Advice
• How to Beat the Adaptive Multi-Armed Bandit
• A Closer Look at Small-loss Bounds for Bandits with Graph Feedback
• Boosting the Generalized Margin in Cost-Sensitive Multiclass Classification
• Driver distraction detection and recognition using RGB-D sensor
• High-Dimensional Variable Selection and Prediction under Competing Risks with Application to SEER-Medicare Linked Data
• Towards automated high-throughput screening of C. elegans on agar
• DARC: Differentiable ARchitecture Compression
• Estimating Subagging by cross-validation
• A PAC-Bayesian Analysis of Randomized Learning with Application to Stochastic Gradient Descent
• Interpretable Classification Models for Recidivism Prediction
• MLBench: How Good Are Machine Learning Clouds for Binary Classification Tasks on Structured Data?
• Deep Learning-Based Diagnostic Model for Parkinson's Disease Using Handwritten Spiral and Wave Images
• On integral probability metrics, ϕ-divergences and binary classification
• Application of newly developed ensemble machine learning models for daily suspended sediment load prediction and related uncertainty analysis
• Convergence and Margin of Adversarial Training on Separable Data
• On Recent Advances in Supervised Ranking for Metabolite Profiling
• Less Is Better: Unweighted Data Subsampling via Influence Function
• Learning compositional functions via multiplicative weight updates
• A Survey on Metric Learning for Feature Vectors and Structured Data
• Explore the Potential Performance of Vision-and-Language Navigation Model: a Snapshot Ensemble Method
• Evidence for the Rare Decay <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"> <mml:mrow> <mml:msup> <mml:mrow> <mml:mi>B</mml:mi> </mml:mrow> <mml:mrow> <mml:mo>+</mml:mo> </mml:mrow> </mml:msup> <mml:mo stretchy="false">→</mml:mo> <mml:mover accent="true"> <mml:mrow> <mml:mi mathvariant="normal">Λ</mml:mi> </mml:mrow> <mml:mrow> <mml:mo stretchy="false">¯</mml:mo> </mml:mrow> </mml:mover> <mml:mi>p</mml:mi> <mml:msup> <mml:mrow> <mml:mi>μ</mml:mi> </mml:mrow> <mml:mrow> <mml:mo>+</mml:mo> </mml:mrow> </mml:msup> <mml:msup> <mml:mrow> <mml:mi>μ</mml:mi> </mml:mrow> <mml:mrow> <mml:mo>−</mml:mo> </mml:mrow> </mml:msup> </mml:mrow> </mml:math>
• Fast Convergence of Regularized Learning in Games
• A comprehensive review of object detection with deep learning
• Bankruptcy forecasting: An empirical comparison of AdaBoost and neural networks
• Playing with and against Hedge
• Margins are Insufficient for Explaining Gradient Boosting
• PAC-Bayesian aggregation and multi-armed bandits
• Exploiting Smoothness in Statistical Learning, Sequential Prediction, and Stochastic Optimization
• On Upper-Confidence Bound Policies for Non-Stationary Bandit Problems
• Veridical data science
• Multi-modal framework for automatic detection of diagnostically important regions in nonalcoholic fatty liver ultrasonic images
• Adversarial Tracking Control via Strongly Adaptive Online Learning with Memory
• A Preliminary Review of Influential Works in Data-Driven Discovery
• Evaluating Classifiers Without Expert Labels
• Concentration inequalities of the cross-validation estimate for stable predictors
• Rethinking Class-Balanced Methods for Long-Tailed Visual Recognition from a Domain Adaptation Perspective
• Intrinsic traits of woodland caribou Rangifer tarandus caribou calves depredated by black bears Ursus americanus and coyotes Canis latrans
• Prediction by Random-Walk Perturbation
• Green Accelerated Hoeffding Tree
• Intelligence, physics and information -- the tradeoff between accuracy and simplicity in machine learning
• Dynamic Regret of Convex and Smooth Functions
• Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learning
• Learning to Hash for Indexing Big Data - A Survey
• MIC: Mining Interclass Characteristics for Improved Metric Learning
• OpenSpiel: A Framework for Reinforcement Learning in Games
• Angle-Based Cost-Sensitive Multicategory Classification
• Soft Gradient Boosting Machine
• Predicting global warming potential: A self-training and ensemble machine learning approach
• Domain Adaptation for Visual Applications: A Comprehensive Survey
• MRI-Based Brain Tumor Classification Using Ensemble of Deep Features and Machine Learning Classifiers
• Multi-category Angle-based Classifier Refit
• Outcome Indistinguishability
• Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels
• Dynamic Data Selection for Curriculum Learning via Ability Estimation
• Fusion of Multispectral Data Through Illumination-aware Deep Neural Networks for Pedestrian Detection
• Achieving All with No Parameters: Adaptive NormalHedge
• AdaBoost is not an Optimal Weak to Strong Learner
• Adversarial Online Learning with noise
• Learning to Reweight with Deep Interactions
• Data Augmentation for Text Generation Without Any Augmented Data
• Re-scale AdaBoost for Attack Detection in Collaborative Filtering Recommender Systems
• Staged Mixture Modelling and Boosting
• Exploiting Unlabeled Data to Enhance Ensemble Diversity
• MARVIN: An Open Machine Learning Corpus and Environment for Automated Machine Learning Primitive Annotation and Execution
• No-Regret Learning in Unknown Games with Correlated Payoffs
• Sample compression schemes for VC classes
• Adversarial Online Learning with Changing Action Sets: Efficient Algorithms with Approximate Regret Bounds
• Contrastive Representations for Label Noise Require Fine-Tuning
• Small-Group Learning, with Application to Neural Architecture Search
• Online Learning: A Comprehensive Survey
• A Reductions Approach to Fair Classification
• Sparse Variational Inference: Bayesian Coresets from Scratch
• Impossible Tuning Made Possible: A New Expert Algorithm and Its\n Applications
• Fast Pedestrian Detection based on T-CENTRIST in infrared image
• A novel ensemble model for multi-temporal forest vegetation classification: integrating spectral-temporal features and topographic constraints
• Improving Simple Models with Confidence Profiles
• Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict
• A solution to the single-question crowd wisdom problem
• Plug-and-play use of tree-based methods: consequences for clinical prediction modeling
• Posterior concentration and fast convergence rates for generalized\n Bayesian learning
• Metric-Free Individual Fairness in Online Learning
• On Deep Ensemble Learning from a Function Approximation Perspective
• Stratified and Time-aware Sampling based Adaptive Ensemble Learning for Streaming Recommendations
• Classification Uncertainty of Deep Neural Networks Based on Gradient\n Information
• Unconstrained Iris Segmentation using Convolutional Neural Networks
• Contextual Bandits with Continuous Actions: Smoothing, Zooming, and Adapting
• Active Dictionary Learning in Sparse Representation Based Classification
• Novel hybrid machine learning framework for high-fidelity prediction of fly ash-based geopolymer concrete strength
• AAA: Adaptive Aggregation of Arbitrary Online Trackers with Theoretical Performance Guarantee
• Multi-cohort machine learning identifies predictors of cognitive impairment in Parkinson’s disease
• Defensive forecasting for optimal prediction with expert advice
• Ensemble of convolution neural networks on heterogeneous signals for sleep stage scoring
• Interpretable Machine Learning for Cross‐Cohort Prediction of Motor Fluctuations in Parkinson's Disease
• Online Gradient Boosting
• Of Moments and Matching: A Game-Theoretic Framework for Closing the\n Imitation Gap
• Dense Adaptive Cascade Forest: A Self Adaptive Deep Ensemble for Classification Problems
• Assessing the combined performance of supervised learning and spike-in constructs for bias correction in eDNA metabarcoding
• PPSNO: A Feature-Rich SNO Sites Predictor by Stacking Ensemble Strategy from Protein Sequence-Derived Information
• Comparative machine learning and deep learning frameworks for robust carcinogenicity prediction and activity cliff analysis
• Online Learning in Case of Unbounded Losses Using the Follow Perturbed Leader Algorithm
• Second-order Quantile Methods for Experts and Combinatorial Games
• Near-Optimal Statistical Query Hardness of Learning Halfspaces with\n Massart Noise
• Learning Halfspaces and Neural Networks with Random Initialization
• Detection of Furigana Text in Images
• Gradient Boosting Machine: A Survey
• Online Learning of Optimal Bidding Strategy in Repeated Multi-Commodity Auctions
• Game-Theoretic Multiagent Reinforcement Learning
• PA-Cache: Evolving Learning-Based Popularity-Aware Content Caching in Edge Networks
• Cox process functional learning
• Insurance Premium Prediction via Gradient Tree-Boosted Tweedie Compound Poisson Models
• Minimizing the Maximal Loss: How and Why?
• Study of the rare decay J/ψ → μ+μ−μ+μ−
• Observation of the Open-Charm Tetraquark Candidate <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"> <mml:mrow> <mml:msubsup> <mml:mrow> <mml:mi>T</mml:mi> </mml:mrow> <mml:mrow> <mml:mi>c</mml:mi> <mml:mi>s</mml:mi> <mml:mn>0</mml:mn> </mml:mrow> <mml:mrow> <mml:mo>*</mml:mo> </mml:mrow> </mml:msubsup> <mml:mo stretchy="false">(</mml:mo> <mml:mn>2870</mml:mn> <mml:msup> <mml:mrow> <mml:mo stretchy="false">)</mml:mo> </mml:mrow> <mml:mrow> <mml:mn>0</mml:mn> </mml:mrow> </mml:msup> </mml:mrow> </mml:math> in the <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"> <mml:mrow> <mml:msup> <mml:mrow> <mml:mi>B</mml:mi> </mml:mrow> <mml:mrow> <mml:mo>−</mml:mo> </mml:mrow> </mml:msup> <mml:mo stretchy="false">→</mml:mo> <mml:msup> <mml:mrow> <mml:mi>D</mml:mi> </mml:mrow> <mml:mrow> <mml:mo>−</mml:mo> </mml:mrow> </mml:msup> <mml:msup> <mml:mrow> <mml:mi>D</mml:mi> </mml:mrow> <mml:mrow> <mml:mn>0</mml:mn> </mml:mrow> </mml:msup> <mml:msubsup> <mml:mrow> <mml:mi>K</mml:mi> </mml:mrow> <mml:mrow> <mml:mi>S</mml:mi> </mml:mrow> <mml:mrow> <mml:mn>0</mml:mn> </mml:mrow> </mml:msubsup> </mml:mrow> </mml:math> Decay
• One for More: Selecting Generalizable Samples for Generalizable ReID Model
• Gradient Boosted Normalizing Flows
• BoostClean: Automated Error Detection and Repair for Machine Learning
• Gradient descent follows the regularization path for general losses
• Statistically Motivated Second Order Pooling
• Diffusion Fingerprints
• Peak-Piloted Deep Network for Facial Expression Recognition
• Smoothed Analysis of Online and Differentially Private Learning
• Online Learning: A Modern Introduction Using Convex Optimization
• Boosted Zero-Shot Learning with Semantic Correlation Regularization
• Efficient Regret Minimization in Non-Convex Games
• Voting in clustering and finding the number of clusters
• Statistical Challenges with High Dimensionality: Feature Selection in Knowledge Discovery
• Ensemble Pruning via Margin Maximization
• Sparse Solutions to Nonnegative Linear Systems and Applications
• CatBoost for big data: an interdisciplinary review
• Large dimensional analysis of general margin based classification methods
• Near-Optimal SQ Lower Bounds for Agnostically Learning Halfspaces and ReLUs under Gaussian Marginals
• Minimax Regret for Stochastic Shortest Path with Adversarial Costs and Known Transition
• Online Agnostic Boosting via Regret Minimization
• Conditional validity of inductive conformal predictors
• Wisdom of Committees: An Overlooked Approach To Faster and More Accurate Models
• SafeVchat: Detecting Obscene Content and Misbehaving Users in Online Video Chat Services
• The Utility of Abstaining in Binary Classification
• Dynamic First Price Auctions Robust to Heterogeneous Buyers
• Planter: Rapid Prototyping of In-Network Machine Learning Inference
• Dari Barisan ke Pakatan: berubahnya dinamiks Pilihan Raya UmumKuala Lumpur 1955-2013
• Regret Bound by Variation for Online Convex Optimization
• Binary Excess Risk for Smooth Convex Surrogates
• Power to the People: The Role of Humans in Interactive Machine Learning
• A Winnow-Based Approach to Context-Sensitive Spelling Correction
• 3D Keypoint Detection Based on Deep Neural Network with Sparse Autoencoder
• Automatic creation of urban velocity fields from aerial video
• A DEEP analysis of the META-DES framework for dynamic selection of ensemble of classifiers
• Efficient First-Order Contextual Bandits: Prediction, Allocation, and Triangular Discrimination
• An Interpretable Produce Price Forecasting System for Small and Marginal Farmers in India using Collaborative Filtering and Adaptive Nearest Neighbors
• Few-Shot Domain Adaptation with Polymorphic Transformers
• Neural network ensembles: Evaluation of aggregation algorithms
• On the Optimality of Perturbations in Stochastic and Adversarial\n Multi-armed Bandit Problems
• Minimally Supervised Feature Selection for Classification (Master's Thesis, University Politehnica of Bucharest)
• A Drifting-Games Analysis for Online Learning and Applications to\n Boosting
• ConceptVision: A Flexible Scene Classification Framework
• Towards Minimax Online Learning with Unknown Time Horizon
• Combating Fraud in Online Social Networks: Detecting Stealthy Facebook\n Like Farms
• Online Importance Weight Aware Updates
• A Review of Artificial Intelligence Technologies for Early Prediction of Alzheimer's Disease
• Deep Extreme Feature Extraction: New MVA Method for Searching Particles in High Energy Physics
• Private Learning Implies Online Learning: An Efficient Reduction
• From Kernel Machines to Ensemble Learning
• A Sampling-Based Approach to Computing Equilibria in Succinct\n Extensive-Form Games
• Linear discriminant initialization for feed-forward neural networks
• Consistency of weighted majority votes
• Learning to Reweight Examples for Robust Deep Learning
• Adaptive Base Class Boost for Multi-class Classification
• Muddling Labels for Regularization, a novel approach to generalization
• Scale Free Adversarial Multi Armed Bandits
• Don't Wait, Just Weight: Improving Unsupervised Representations by Learning Goal-Driven Instance Weights
• WDR FACE: The First Database for Studying Face Detection in Wide Dynamic Range
• Online Learning of Facility Locations
• Improved Regret Bounds for Tracking Experts with Memory
• Re-evaluating Evaluation
• Boosting as a Product of Experts
• Automatic Detection of Coronavirus Disease (COVID-19) in X-ray and CT\n Images: A Machine Learning-Based Approach
• Bayes Imbalance Impact Index: A Measure of Class Imbalanced Dataset for Classification Problem
• Learning to Sample: an Active Learning Framework
• Learning to Bid Optimally and Efficiently in Adversarial First-price Auctions
• Online Adaptive Principal Component Analysis and Its extensions
• Vote-boosting ensembles
• Efficient Online Portfolio with Logarithmic Regret