Guidance for the Verification and Validation of Neural Networks

Guidance for the Verification and Validation of Neural Networks is a supplement to the IEEE Standard for Software Verification and Validation, IEEE Std 1012-1998. Born out of a need by the National Aeronautics and Space Administration's safety- and mission-critical research, this book compiles over five years of applied research and development efforts. It is intended to assist the performance of verification and validation (V&V) activities on adaptive software systems, with emphasis given to neural network systems. The book discusses some of the difficulties with trying to assure adaptive systems in general, presents techniques and advice for the V&V practitioner confronted with such a task, and based on a neural network case study, identifies specific tasking and recommendations for the V&V of neural network systems.

"As the demand for developing and assuring adaptive systems grows, this guidebook will provide practitioners with the insight and practical steps for verifying and validating neural networks. The work of the authors is a great step forward, offering a level of practical experience and advice for the software developers, assurance personnel, and those performing verification and validation of adaptive systems. This guide makes possible the daunting task of assuring this new technology. NASA is proud to sponsor such a realistic approach to what many might think a very futuristic subject. But adaptive systems with neural networks are here today and as the NASA Manager for Software Assurance and Safety, I believe this work by the authors will be a great resource for the systems we are building today and into tomorrow."
-Martha S. Wetherholt, NASA Manager of Software Assurance and Software Safety NASA Headquarters, Office of Safety & Mission Assurance

Preface.

Acknowledgements.

1 Overview.

1.1 Definitions and Conventions.

1.2 Organization of the Book.

2 Areas of Consideration for Adaptive Systems.

2.1 Safety-Critical Adaptive System Example and Experience.

2.2 Hazard Analysis.

2.3 Requirements for Adaptive Systems.

2.4 Rule Extraction.

2.5 Modified Life Cycle for Developing Neural Networks.

2.6 Operational Monitors.

2.7 Testing Considerations.

2.8 Training Set Analysis.

2.9 Stability Analysis

2.10 Configuration Management of Neural Network Training and Design.

2.11 Simulation of Adaptive Systems.

2.12 Neural Network Visualization.

2.13 Adaptive System and Neural Network Selection.

3 Verification and Validation of Neural Networks—Guidance.

3.1 Process: Management.

3.2 Process: Acquisition.

3.3 Process: Supply.

3.4 Process: Development.

3.5 Process: Operation.

3.6 Process: Maintenance.

4 Recent Changes to IEEE Std 1012.

Appendix A: References.

Appendix B: Acronyms.

Appendix C: Definitions.