Pinjia He

1. GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher
   2023/08/12 by Youliang Yuan, Wenxiang Jiao, Yuan, Youliang +11 · 57 citations
   Computer Science · Medicine · #Topic Modeling #Artificial Intelligence in Healthcare and Education #Natural Language Processing Techniques
2. Loghub: A Large Collection of System Log Datasets for AI-driven Log Analytics
   2020/08/14 by Jieming Zhu, Shilin He, Zhu, Jieming +6 · 22 citations
   Computer Science · #Anomaly Detection Techniques and Applications #Cloud Computing and Resource Management #FOS: Computer and information sciences #Software Engineering (cs.SE) #Software System Performance and Reliability
3. Tools and Benchmarks for Automated Log Parsing
   2018/11/08 by Jieming Zhu, Zhu, Jieming, Shilin He +11 · 11 citations
   Computer Science · #Software System Performance and Reliability #Network Security and Intrusion Detection #Software Reliability and Analysis Research
4. LILAC: Log Parsing using LLMs with Adaptive Parsing Cache
   2023/10/03 by Zhihan Jiang, Jiang, Zhihan, Jinyang Liu +15 · 20 citations
   Computer Science · #FOS: Computer and information sciences #Natural Language Processing Techniques #Software Engineering (cs.SE)
5. Logzip: Extracting Hidden Structures via Iterative Clustering for Log\n Compression
   2019/09/23 by Jinyang Liu, Jieming Zhu, Liu, Jinyang +8 · 3 citations
   Computer Science · Decision Sciences · #Cloud Computing and Resource Management #Databases (cs.DB) #FOS: Computer and information sciences #Scientific Computing and Data Management #Software Engineering (cs.SE) #Software System Performance and Reliability
6. BiasAsker: Measuring the Bias in Conversational AI System
   2023/05/21 by Yuxuan Wan, Wan, Yuxuan, Wenxuan Wang +9 · 4 citations
   Computer Science · #Adversarial Robustness in Machine Learning #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Hate Speech and Cyberbullying Detection
7. Trust, But Verify: A Self-Verification Approach to Reinforcement Learning with Verifiable Rewards
   2025/05/19 by Xiaoyuan Liu, Tian Liang, Liu, Xiaoyuan +15 · 14 citations
   Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Reinforcement Learning in Robotics #Topic Modeling
8. Testing Machine Translation via Referential Transparency
   2020/04/22 by Pinjia He, He, Pinjia, Clara Meister +3 · 2 citations
   Computer Science · #Adversarial Robustness in Machine Learning #Topic Modeling #Natural Language Processing Techniques
9. UTBoost: Rigorous Evaluation of Coding Agents on SWE-Bench
   2025/06/10 by Bin Yu, Yu, Boxi, Pinjia He +4 · 7 citations
   Computer Science · #Computation and Language (cs.CL) #D.0 #FOS: Computer and information sciences #I.2 #Software Engineering (cs.SE) #Software Engineering Research #Software Testing and Debugging Techniques #Topic Modeling
10. AEON: A Method for Automatic Evaluation of NLP Test Cases
    2022/05/13 by Jen-tse Huang, Huang, Jen-tse, Jianping Zhang +9 · 2 citations
    Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Software Engineering (cs.SE) #Software Engineering Research #Topic Modeling
11. Automated Testing and Improvement of Named Entity Recognition Systems
    2023/08/14 by Boxi Yu, Yiyan Hu, Yu, Boxi +7 · 2 citations
    Computer Science · Decision Sciences · #Topic Modeling #Natural Language Processing Techniques #Data Quality and Management
12. Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility
    2025/01/18 by Jialun Cao, Cao, Jialun, Chan, Yuk-Kit +24 · 4 citations
    Medicine · Social Sciences · #Artificial Intelligence (cs.AI) #Artificial Intelligence in Healthcare and Education #Computation and Language (cs.CL) #Computational and Text Analysis Methods #Ethics and Social Impacts of AI #FOS: Computer and information sciences #Software Engineering (cs.SE)
13. AutoLog: A Log Sequence Synthesis Framework for Anomaly Detection
    2023/08/18 by Yintong Huo, Huo, Yintong, Yichen Li +9 · 2 citations
    Computer Science · #Anomaly Detection Techniques and Applications #FOS: Computer and information sciences #Software Engineering (cs.SE) #Software Engineering Research #Software System Performance and Reliability
14. Structure-Invariant Testing for Machine Translation
    2019/07/19 by Pinjia He, He, Pinjia, Clara Meister +3 · 1 citation
    Computer Science · #Software Testing and Debugging Techniques #Software Engineering Research #Topic Modeling
15. Can't See the Forest for the Trees: Benchmarking Multimodal Safety Awareness for Multimodal LLMs
    2025/02/16 by Wenxuan Wang, Wang, Wenxuan, Xiaoyuan Liu +11 · 4 citations
    Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Multimedia (cs.MM) #Natural Language Processing Techniques #Semantic Web and Ontologies #Speech and dialogue systems
16. HeMiRCA: Fine-Grained Root Cause Analysis for Microservices with Heterogeneous Data Sources
    2024/11/21 by Zhouruixing Zhu, Cheryl Lee, Xiaoying Tang +1 · 3 citations
17. Automated Testing of Image Captioning Systems
    2022/06/14 by Boxi Yu, Zhiqing Zhong, Yu, Boxi +9 · 1 citation
    Computer Science · #Multimodal Machine Learning Applications #Advanced Image and Video Retrieval Techniques #Advanced Neural Network Applications
18. Incident-aware Duplicate Ticket Aggregation for Cloud Systems
    2023/02/19 by Jinyang Liu, Liu, Jinyang, Shilin He +23 · 1 citation
    Computer Science · Decision Sciences · #Cloud Computing and Resource Management #Cloud Data Security Solutions #Data Quality and Management #FOS: Computer and information sciences #Software Engineering (cs.SE)
19. LogicAsker: Evaluating and Improving the Logical Reasoning Ability of Large Language Models
    2024/01/01 by Yuxuan Wan, Wenxuan Wang, Wan, Yuxuan +13 · 1 citation
    Computer Science · Medicine · #Artificial Intelligence (cs.AI) #Artificial Intelligence in Healthcare and Education #Computation and Language (cs.CL) #FOS: Computer and information sciences #Logic in Computer Science (cs.LO) #Natural Language Processing Techniques #Software Engineering (cs.SE) #Topic Modeling
20. AL-Bench: A Benchmark for Automatic Logging
    2025/02/05 by Baohai Tan, Tan, Boyin, Junjielong Xu +5 · 2 citations
    Engineering · #Advanced Manufacturing and Logistics Optimization #FOS: Computer and information sciences #Robotic Mechanisms and Dynamics #Software Engineering (cs.SE)
21. Prompting for Automatic Log Template Extraction
    2023/07/19 by Junjielong Xu, Xu, Junjielong, Ruichun Yang +7 · 1 citation
    Computer Science · Decision Sciences · Engineering · #Software System Performance and Reliability #Data Quality and Management #Traffic Prediction and Management Techniques
22. PAIChecker: Uncovering and Checking PR-Issue Misalignment in SWE-Bench-Like Benchmarks
    2026/07/30 by Manyi Wang, Junjielong Xu, Pinjia He
    Computer Science · #cs.AI #cs.SE