2025/05/16 by Zeyu Gao, Yuxin Cui, Gao, Zeyu +13 · 1 voice · 3 citations
Computer Science · #Advanced Malware Detection Techniques #Benchmark (surveying) #Correctness #Fidelity #Focus (optics) #Function (biology) #Key (lock) #Reverse engineering #Security and Verification in Computing #Software Testing and Debugging Techniques #Workflow #cs.AI #cs.SE
paper · pdf · doi:10.48550/arxiv.2505.11340
published in arXiv (Cornell University) (Cornell University)
openalex publication_date 2025/05/16 · arxiv published 2025/05/16 · arxiv updated 2025/05/16 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05
Decompilers are fundamental tools for critical security tasks, from vulnerability discovery to malware analysis, yet their evaluation remains fragmented. Existing approaches primarily focus on syntactic correctness through synthetic micro-benchmarks or subjective human ratings, failing to address real-world requirements for semantic fidelity and analyst usability. We present DecompileBench, the first comprehensive framework that enables effective evaluation of decompilers in reverse engineering workflows through three key components: real-world function extraction (comprising 23,400 functions from 130 real-world programs), runtime-aware validation, and automated human-centric assessment using LLM-as-Judge to quantify the effectiveness of decompilers in reverse engineering workflows. Through a systematic comparison between six industrial-strength decompilers and six recent LLM-powered approaches, we demonstrate that LLM-based methods surpass commercial tools in code understandability despite 52.2% lower functionality correctness. These findings highlight the potential of LLM-based approaches to transform human-centric reverse engineering. We open source \hrefhttps://github.com/Jennieett/DecompileBenchDecompileBench to provide a framework to advance research on decompilers and assist security experts in making informed tool selections based on their specific requirements.