Robust ML Auditing using Prior Knowledge

2025/05/07 by Jade Garcia Bourrée, Bourrée, Jade Garcia, Augustin Godinot +15 · 1 voice · 4 citations
Computer Science · #Audit #Audit risk #Control (management) #FOS: Computer and information sciences #Generalization #Machine Learning (cs.LG) #Neural Networks and Applications #Prior probability #Task (project management) #cs.LG

paper · pdf · doi:10.48550/arxiv.2505.04796

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2025/05/07 · arxiv published 2025/05/07 · arxiv updated 2025/05/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Abstract

Among the many technical challenges to enforcing AI regulations, one crucial yet underexplored problem is the risk of audit manipulation. This manipulation occurs when a platform deliberately alters its answers to a regulator to pass an audit without modifying its answers to other users. In this paper, we introduce a novel approach to manipulation-proof auditing by taking into account the auditor's prior knowledge of the task solved by the platform. We first demonstrate that regulators must not rely on public priors (e.g. a public dataset), as platforms could easily fool the auditor in such cases. We then formally establish the conditions under which an auditor can prevent audit manipulations using prior knowledge about the ground truth. Finally, our experiments with two standard datasets illustrate the maximum level of unfairness a platform can hide before being detected as malicious. Our formalization and generalization of manipulation-proof auditing with a prior opens up new research directions for more robust fairness audits.

Citations

• Active Fourier Auditor for Estimating Distributional Properties of ML Models
• From Transparency to Accountability and Back: A Discussion of Access and Evidence in AI Auditing
• Reasons to Doubt the Impact of AI Risk Evaluations
• Trustless Audits without Revealing Data or Models
• FairProof : Confidential and Certifiable Fairness for Neural Networks
• Fairness Auditing with Multi-Agent Collaboration
• AI auditing: The Broken Bus on the Road to AI Accountability
• Auditing Fairness by Betting
• Leveraging Imperfect Sources to Detect Fairwashing in Black-Box Auditing
• Active Fairness Auditing
• XAudit : A Theoretical Look at Auditing with Explanations
• Outsider Oversight: Designing a Third Party Audit Ecosystem for AI Governance
• Attribution-based Explanations that Provide Recourse Cannot be Robust
• Leveling Down in Computer Vision: Pareto Inefficiencies in Fair Deep Classifiers
• Optimal Transport of Classifiers to Fairness
• AI and the Everything in the Whole Wide World Benchmark
• Algorithmic Fairness Verification with Graphical Models
• Retiring Adult: New Datasets for Fair Machine Learning
• Characterizing the risk of fairwashing
• Fairness seen as Global Sensitivity Analysis
• A Statistical Test for Probabilistic Fairness
• Justicia: A Stochastic SAT Approach to Formally Verify Fairness
• Fairwashing Explanations with Off-Manifold Detergent
• Evaluating Fairness Using Permutation Tests
• Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods
• A Survey on Bias and Fairness in Machine Learning
• Inherent Tradeoffs in Learning Fair Representations
• Fairwashing: the risk of rationalization
• Faking Fairness via Stealthily Biased Sampling
• Multiaccuracy: Black-Box Post-Processing for Fairness in Classification
• Equality of Opportunity in Supervised Learning
• Deep Learning Face Attributes in the Wild
• Fool SHAP with Stealthily Biased Sampling
• Gradient-based learning applied to document recognition

Cited by

• Position: Token Taxes Can Mitigate AI's Economic Risks
• Auditing Pay-Per-Token in Large Language Models
• Exposing the Illusion of Fairness: Auditing Vulnerabilities to Distributional Manipulation Attacks
• Fairness Auditing: Lower Bounds on Company Manipulation
• Manipulation-Proof Oblivious Audits against Deceptive Model Providers

Discussions

• 🧵2/6 link: arxiv.org/pdf/2505.04796 An audit is pretty straightforward. 1/ I, the auditor 🕵️ come up with questions to ask your model. 2/ You, the platform 😈 answer my questions. 3/ I look at your [bsky, 0 points, 1 comments]

Related

• Exposing the Illusion of Fairness: Auditing Vulnerabilities to Distributional Manipulation Attacks
• Leveraging Imperfect Sources to Detect Fairwashing in Black-Box Auditing
• Peer-induced Fairness: A Causal Approach for Algorithmic Fairness Auditing
• TRUST: A Decentralized Framework for Auditing Large Language Model Reasoning
• Active Fairness Auditing