On a Method to Measure Supervised Multiclass Model’s Interpretability: Application to Degradation Diagnosis (Short Paper)

2024/01/01 by Gauriat, Charles-Maxime, Pencolé, Yannick, Ribot, Pauline +5 · 48 citations
Computer Science · Engineering · #Bayesian Modeling and Causal Inference #Engineering Diagnostics and Reliability #Explainable Artificial Intelligence (XAI) #Machine Learning and Data Classification

paper · pdf · doi:10.4230/oasics.dx.2024.27

openalex publication_date 2024/01/01 · openalex created_date 2025/01/14 · openalex updated_date 2026/07/29

Abstract

In an industrial maintenance context, degradation diagnosis is the problem of determining the current level of degradation of operating machines based on measurements. With the emergence of Machine Learning techniques, such a problem can now be solved by training a degradation model offline and by using it online. While such models are more and more accurate and performant, they are often black-box and their decisions are therefore not interpretable for human maintenance operators. On the contrary, interpretable ML models are able to provide explanations for the model’s decisions and consequently improves the confidence of the human operator about the maintenance decision based on these models. This paper proposes a new method to quantitatively measure the interpretability of such models that is agnostic (no assumption about the class of models) and that is applied on degradation models. The proposed method requires that the decision maker sets up some high level parameters in order to measure the interpretability of the models and then can decide whether the obtained models are satisfactory or not. The method is formally defined and is fully illustrated on a decision tree degradation model and a model trained with a recent neural network architecture called Multiclass Neural Additive Model.

Citations

• "Why Should I Trust You?": Explaining the Predictions of Any Classifier

Cited by

• Play Fair: Frame Attributions in Video Models
• ExKMC: Expanding Explainable k-Means Clustering
• Global geometry of chemical graph neural network representations in terms of chemical moieties
• Explaining the data or explaining a model? Shapley values that uncover non-linear dependencies
• Hybrid CNN XGBoost intrusion detection approach tuned by modified sine cosine algorithm towards better cloud security
• On the price of explainability for some clustering problems
• Dominant Controls on Preferential Flow and Their Implications for Future Soil Water Fluxes
• Multi-omics driven genome-scale metabolic modeling improves viral vector yield in HEK293
• Developing a crop- wild-reservoir pathogen system to understand pathogen evolution and emergence
• Disentangling Effects of Vegetation Structure and Physiology on Land–Atmosphere Coupling
• On Explainability of Graph Neural Networks via Subgraph Explorations
• Prediction of Diffusion Coefficient Through Machine Learning Based on Transition State Theory Descriptors
• Predicting Dyadic Synchrony: A Theory‐Driven Machine Learning Approach
• EvoMapX: An explainable framework for metaheuristic optimization algorithms
• On Locality of Local Explanation Models
• Predicting sepsis in multi-site, multi-national intensive care cohorts using deep learning
• Facilitating Database Tuning with Hyper-Parameter Optimization: A Comprehensive Experimental Evaluation
• Interpreting Super-Resolution Networks with Local Attribution Maps
• A simple defense against adversarial attacks on heatmap explanations
• Fine-grained Uncertainty Modeling in Neural Networks
• A Multimodal Deep Learning Approach for Soil Moisture Downscaling Using Remote Sensing and Weather Data
• Learning Interpretable Shapelets for Time Series Classification through Adversarial Regularization
• To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-making
• From Verification to Causality-based Explications
• EDDA: Explanation-driven Data Augmentation to Improve Explanation Faithfulness
• Explainable AI meets Healthcare: A Study on Heart Disease Dataset
• Compound soil and atmospheric drought (CSAD) events and CO ₂  fluxes of a mixed deciduous forest: the occurrence, impact, and temporal contribution of main drivers
• Human-Centered Explainable AI (XAI): From Algorithms to User Experiences
• Does the dataset meet your expectations? Explaining sample representation in image data
• Efficient computation and analysis of distributional Shapley values
• The many Shapley values for model explanation
• Explaining Data-Driven Decisions made by AI Systems: The Counterfactual Approach
• Modeling the Asymptotic Behavior of Higher Order Aftershocks with Deep Learning
• AR-Net: A simple Auto-Regressive Neural Network for time-series
• Trustworthy AI: From Principles to Practices
• Investigating the Robustness of Artificial Intelligent Algorithms with Mixture Experiments
• Synthetic Benchmarks for Scientific Research in Explainable Machine Learning
• Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation\n Methods
• Counterfactual Explanations Can Be Manipulated
• Reducing Unintended Bias of ML Models on Tabular and Textual Data
• Invariant Rationalization
• Uncovering the Dark Side of Telegram: Fakes, Clones, Scams, and Conspiracy Movements
• Understanding Heart-Failure Patients EHR Clinical Features via SHAP Interpretation of Tree-Based Machine Learning Model Predictions
• Feature Synergy, Redundancy, and Independence in Global Model Explanations using SHAP Vector Decomposition
• Near-Optimal Explainable k-Means for All Dimensions
• Machine Learning-Based Soft Sensors for Vacuum Distillation Unit
• Online-to-Offline Advertisements as Field Experiments
• A machine-learning-based approach to predict early hallmarks of progressive hearing loss
• Explaining Knowledge Distillation by Quantifying the Knowledge
• Reimagining GNN Explanations with ideas from Tabular Data
• Explaining by Removing: A Unified Framework for Model Explanation
• Competing narratives in AI ethics: a defense of sociotechnical pragmatism
• Explaining heatwaves with machine learning
• Model Distillation for Revenue Optimization: Interpretable Personalized\n Pricing
• NASirt: AutoML based learning with instance-level complexity information
• Interpretable Machine Learning: Moving From Mythos to Diagnostics
• Explaining Youth Driver Licensing Determinants Using XGBoost and SHAP
• Modeling the latent impacts of extreme floods on indoor mold spores in residential buildings: Application of machine learning algorithms
• EEG-based brain age prediction in infants–toddlers: Implications for early detection of neurodevelopmental disorders
• Clusters in Explanation Space: Inferring disease subtypes from model explanations
• A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI
• A Comprehensive Survey of Incentive Mechanism for Federated Learning
• Current state and prospects of artificial intelligence in allergy
• Shapley Interpretation and Activation in Neural Networks
• A multi-component framework for the analysis and design of explainable artificial intelligence
• Interpretable feature subset selection: A Shapley value based approach
• Ontology-based Interpretable Machine Learning for Textual Data
• Machine Learning Accelerates Discovery of High-Performance Corrole Photosensitizers for Optical Imaging Diagnosis and Photodynamic Therapeutics of Nasopharyngeal Carcinoma
• Getting a CLUE: A Method for Explaining Uncertainty Estimates
• Explainability for fair machine learning
• A Human-Grounded Evaluation of SHAP for Alert Processing
• Time-series forecasting with deep learning: a survey
• Causal Interpretability for Machine Learning -- Problems, Methods and Evaluation
• Cracking the code of adaptive immunity: The role of computational tools
• DCI-ES: An Extended Disentanglement Framework with Connections to Identifiability
• Disentangling Influence: Using Disentangled Representations to Audit Model Predictions
• Model-agnostic interpretation by visualization of feature perturbations
• Data-driven composition-only machine learning for high-performance solid-state electrolytes
• Mechanisms and drivers of soil pH assessed by Shapley additive explanation
• A predictive analytics framework for early detection of production halts and quality issues
• Identification of a 10-species microbial signature of inflammatory bowel disease by machine learning and external validation
• Interpreting graph neural networks with Myerson values for cheminformatics approaches
• Insights into heart failure metabolite markers through explainable machine learning
• NormLime: A New Feature Importance Metric for Explaining Deep Neural Networks
• Concept Learners for Few-Shot Learning
• Efficient nonparametric statistical inference on population feature importance using Shapley values
• Explanations can be manipulated and geometry is to blame
• Interpretations are useful: penalizing explanations to align neural networks with prior knowledge
• ‘Entanglement’ – A new dynamic metric to measure team flow
• Algorithm Fairness in AI for Medicine and Healthcare
• A Distributional Framework for Data Valuation
• "How do I fool you?": Manipulating User Trust via Misleading Black Box Explanations
• FastIF: Scalable Influence Functions for Efficient Model Interpretation and Debugging
• Identifying high‐risk firearms dealers: A machine learning study of rapidly diverted firearm sales in California
• Environmentally derived subgroups of preadolescents with family history of substance use exhibit distinct patterns of psychopathology and reward-related behaviors: insights from the ABCD study
• Seeing the unseen: A novel approach to extract latent plant root traits from digital images
• Machine Learning Algorithms to Predict Heavy Episodic Drinking in the United States Using Survey Data
• Unifying local and global model explanations by functional decomposition of low dimensional structures
• Is Genre Enough? A Theory of Genre Signaling as Generative AI Rhetoric
• SbD4Skin by EosCloud: Integrating multi-view molecular representation for predicting skin sensitization, irritation, and acute dermal toxicity
• AI-driven multi-omics modeling of myalgic encephalomyelitis/chronic fatigue syndrome
• Understanding plant phenotypes in crop breeding through explainable AI
• Practical machine learning model selection and interpretation for organophosphorus flame retardancy in Epoxy resin
• Interpretable machine learning approach to predict Hepatitis C virus NS5B inhibitor activity using voting-based LightGBM and SHAP
• DoliClock: a lipid-based aging clock reveals accelerated aging in neurological disorders
• Transformers and genome language models
• Fast and Efficient Root Phenotyping via Pose Estimation
• Expanding the flexibility of base editing for high-throughput genetic screens in bacteria
• Explainable Machine Learning in Human Gait Analysis: A Study on Children With Cerebral Palsy
• Machine learning-based reclassification of germline variants of unknown significance: The RENOVO algorithm
• Genome-wide transcriptome profiling and development of age prediction models in the human brain
• Generative causal explanations of black-box classifiers
• Multi-concept adversarial attacks
• Optimizing wastewater treatment through artificial intelligence: recent advances and future prospects
• Interpretable Anomaly Detection with DIFFI: Depth-based Isolation Forest Feature Importance
• Efficient Explanations With Relevant Sets
• Can We Faithfully Represent Masked States to Compute Shapley Values on a DNN?
• Neural Image Compression and Explanation
• Teaching the Machine to Explain Itself using Domain Knowledge
• ViDi: Descriptive Visual Data Clustering as Radiologist Assistant in COVID-19 Streamline Diagnostic
• How Much Can I Trust You? -- Quantifying Uncertainties in Explaining Neural Networks
• Rationalizing Medical Relation Prediction from Corpus-level Statistics
• Generative Causal Explanations for Graph Neural Networks
• Robust and Stable Black Box Explanations
• Trustworthy Convolutional Neural Networks: A Gradient Penalized-based Approach
• Explanation of machine learning models using shapley additive explanation and application for real data in hospital
• Radiomics feature analysis for survival prediction in multiple myeloma: An automated PET/CT approach
• Causal Forecasting:Generalization Bounds for Autoregressive Models
• Shedding light on underrepresentation and Sampling Bias in machine learning
• A Survey of Deep Learning for Scientific Discovery
• The Eval4NLP Shared Task on Explainable Quality Estimation: Overview and Results
• Sampling Permutations for Shapley Value Estimation
• On Completeness-aware Concept-Based Explanations in Deep Neural Networks
• BMB-LIME: LIME with modeling local nonlinearity and uncertainty in explainability
• TwoWin-SOVA: Two Windows Discrete Cosine Transform and Synthetic Minority Oversampling Technique One-Versus-All Ensemble Classifiers for Imbalanced Hyperspectral Image Explainable Classification
• Prediction of post-stroke urinary tract infection risk in immobile patients using machine learning: an observational cohort study
• TREX: Tree-Ensemble Representer-Point Explanations
• Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Models
• TED: Teaching AI to Explain its Decisions
• Deep learning meets urban design: Assessing streetscape aesthetic and design quality through AI and cluster analysis
• Assessing equity in infrastructure investment distribution among U.S. cities
• Why did the distribution change?
• SurvLIME-Inf: A simplified modification of SurvLIME for explanation of\n machine learning survival models
• Explainable AI and Adoption of Financial Algorithmic Advisors: an\n Experimental Study
• What Did You Think Would Happen? Explaining Agent Behaviour Through Intended Outcomes
• Global Explainability of BERT-Based Evaluation Metrics by Disentangling\n along Linguistic Factors
• GNNExplainer: Generating Explanations for Graph Neural Networks
• A turbulent eddy-viscosity surrogate modeling framework for\n Reynolds-Averaged Navier-Stokes simulations
• Human-grounded Evaluations of Explanation Methods for Text\n Classification
• Towards Out-Of-Distribution Generalization: A Survey
• Shapley Flow: A Graph-based Approach to Interpreting Model Predictions
• MMLSpark: Unifying Machine Learning Ecosystems at Massive Scales
• Extending LIME for Business Process Automation
• Visualizing Color-wise Saliency of Black-Box Image Classification Models
• Unifying Model Explainability and Robustness via Machine-Checkable Concepts
• Do Explanations Reflect Decisions? A Machine-centric Strategy to Quantify the Performance of Explainability Algorithms
• Explainable predictions of different machine learning algorithms used to predict Early Stage diabetes
• Explaining GNN over Evolving Graphs using Information Flow
• Portfolio Performance Attribution via Shapley Value
• Explaining the Explainer: A First Theoretical Analysis of LIME
• Exploring Deep Neural Networks and Transfer Learning for Analyzing\n Emotions in Tweets
• Contextual Local Explanation for Black Box Classifiers
• Beta Shapley: a Unified and Noise-reduced Data Valuation Framework for Machine Learning
• Best of both worlds: local and global explanations with human-understandable concepts
• Generative Counterfactuals for Neural Networks via Attribute-Informed Perturbation
• An interpretable semi-supervised classifier using two different\n strategies for amended self-labeling
• Explain and Improve: LRP-Inference Fine-Tuning for Image Captioning Models
• QNNVerifier: A Tool for Verifying Neural Networks using SMT-Based Model Checking
• CACTUS: Detecting and Resolving Conflicts in Objective Functions
• Understanding Global Feature Contributions With Additive Importance Measures
• Ranking Policy Decisions
• Interpretable CNNs for Object Classification
• Towards Hierarchical Importance Attribution: Explaining Compositional Semantics for Neural Sequence Models
• Clinical Utility of the Automatic Phenotype Annotation in Unstructured Clinical Notes: ICU Use Cases
• Making ML models fairer through explanations: the case of LimeOut
• Latent-CF: A Simple Baseline for Reverse Counterfactual Explanations
• Fairwashing: the risk of rationalization
• Generalized Integrated Gradients: A practical method for explaining diverse ensembles
• Concise Explanations of Neural Networks using Adversarial Training
• Who's responsible? Jointly quantifying the contribution of the learning algorithm and training data
• On Efficiently Explaining Graph-Based Classifiers
• Self-learn to Explain Siamese Networks Robustly
• Valid Explanations for Learning to Rank Models
• Machine Learning approach for Credit Scoring
• SHAQ: Incorporating Shapley Value Theory into Multi-Agent Q-Learning
• ASAC: Active Sensing using Actor-Critic models
• Show or Suppress? Managing Input Uncertainty in Machine Learning Model Explanations
• Efficient Explanations for Knowledge Compilation Languages
• High resolution weakly supervised localization architectures for medical\n images
• Adversarial Attacks and Defenses: An Interpretation Perspective
• Critique of impure reason: Unveiling the reasoning behaviour of medical large language models
• Consensus Multiplicative Weights Update: Learning to Learn using\n Projector-based Game Signatures
• Model-Agnostic Explanations using Minimal Forcing Subsets
• Retrosplenial cortex enables context-dependent goal-directed sensorimotor transformation
• Incorporating Priors with Feature Attribution on Text Classification
• Do not explain without context: addressing the blind spot of model explanations
• Mapping sleep-promoting volatiles in aromatic plants with machine learning: a comprehensive survey of 2300 molecules
• Critical Empirical Study on Black-box Explanations in AI
• Explaining Algorithmic Fairness Through Fairness-Aware Causal Path Decomposition
• On the Importance of Domain-specific Explanations in AI-based Cybersecurity Systems (Technical Report)
• Explaining Groups of Points in Low-Dimensional Representations
• Human-interpretable model explainability on high-dimensional data
• Interpreting Undesirable Pixels for Image Classification on Black-Box Models
• Explainable Deep Learning for Uncovering Actionable Scientific Insights for Materials Discovery and Design
• Scalable Topological Data Analysis and Visualization for Evaluating Data-Driven Models in Scientific Applications
• Towards Explainability of Machine Learning Models in Insurance Pricing
• A Novel Approach for Semiconductor Etching Process with Inductive Biases

Related

• An Explainable Regression Framework for Predicting Remaining Useful Life of Machines
• On The Stability of Interpretable Models
• Dynamic Model Agnostic Reliability Evaluation of Machine-Learning Methods Integrated in Instrumentation & Control Systems
• Probabilistic Diagnostic Tests for Degradation Problems in Supervised Learning
• Assessing the Local Interpretability of Machine Learning Models