testRNN: Coverage-guided Testing on Recurrent Neural Networks

2019/06/20 by Wei Huang, Huang, Wei, Youcheng Sun +6
Computer Science · #Advanced Neural Network Applications #Adversarial Robustness in Machine Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Neural and Evolutionary Computing (cs.NE) #Software Engineering (cs.SE) #Software Testing and Debugging Techniques #cs.LG #cs.NE #cs.SE

paper · pdf · doi:10.48550/arxiv.1906.08557

Summited to ASE 2019 Demonstrations Track

arxiv created 2019/06/20 · openalex publication_date 2019/06/20 · arxiv updated 2019/06/21 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Recurrent neural networks (RNNs) have been widely applied to various sequential tasks such as text processing, video recognition, and molecular property prediction. We introduce the first coverage-guided testing tool, coined testRNN, for the verification and validation of a major class of RNNs, long short-term memory networks (LSTMs). The tool implements a generic mutation-based test case generation method, and it empirically evaluates the robustness of a network using three novel LSTM structural test coverage metrics. Moreover, it is able to help the model designer go through the internal data flow processing of the LSTM layer. The tool is available through: https://github.com/TrustAI/testRNN under the BSD 3-Clause licence.

Related

• RNN-Test: Towards Adversarial Testing for Recurrent Neural Network Systems
• Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) network
• RETURNN: The RWTH Extensible Training framework for Universal Recurrent Neural Networks
• SLIM LSTMs
• Understanding LSTM -- a tutorial into Long Short-Term Memory Recurrent Neural Networks