Inverting Black-Box Face Recognition Systems via Zero-Order Optimization in Eigenface Space

2025/06/11 by Anton Razzhigaev, Razzhigaev, Anton, Matvey Mikhalchuk +9
Computer Science · #Advanced Neural Network Applications #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Face and Expression Recognition #Face recognition and analysis

paper · pdf · doi:10.48550/arxiv.2506.09777

openalex publication_date 2025/06/11 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Reconstructing facial images from black-box recognition models poses a significant privacy threat. While many methods require access to embeddings, we address the more challenging scenario of model inversion using only similarity scores. This paper introduces DarkerBB, a novel approach that reconstructs color faces by performing zero-order optimization within a PCA-derived eigenface space. Despite this highly limited information, experiments on LFW, AgeDB-30, and CFP-FP benchmarks demonstrate that DarkerBB achieves state-of-the-art verification accuracies in the similarity-only setting, with competitive query efficiency.

Citations

• DiffMI: Breaking Face Recognition Privacy via Diffusion-Driven Training-Free Model Inversion
• Face Reconstruction from Face Embeddings using Adapter to a Face Foundation Model
• A Closer Look at GAN Priors: Exploiting Intermediate Features for Enhanced Model Inversion Attacks
• Label-Only Model Inversion Attacks via Knowledge Transfer
• Controllable Inversion of Black-Box Face Recognition Models via Diffusion
• Plug & Play Attacks: Towards Robust and Flexible Model Inversion Attacks
• Vec2Face: Unveil Human Faces from their Blackbox Features in Face\n Recognition
• The Secret Revealer: Generative Model-Inversion Attacks Against Deep Neural Networks
• A Style-Based Generator Architecture for Generative Adversarial Networks
• Understanding Deep Image Representations by Inverting Them

Related

• Darker than Black-Box: Face Reconstruction from Similarity Queries
• Controllable Inversion of Black-Box Face Recognition Models via Diffusion
• Improved Eigenfeature Regularization for Face Identification
• DiffMI: Breaking Face Recognition Privacy via Diffusion-Driven Training-Free Model Inversion
• Privacy-Preserving Face Recognition in Hybrid Frequency-Color Domain