Title: Potential Methods to Enhance Safety within Neural Network Based Systems

Author(s): Ali Hessami, Graham Sutherland

Publication Event: Proceedings of the Twenty-seventh Safety-Critical Systems Symposium, Bristol, UK

Publication Date: 2019-02-07

Resource URL: https://scsc.uk/r1083.pdf

Abstract:

Artificial Neural Networks (ANNs) are increasingly prevalent in complex systems and are becoming involved in safety-critical decision-making and in real-time executive action where safety is implicated. Examples include autonomous vehicles and automated process control. However, proving that such systems are acceptably safe in such applications is considered impossible in the current state of the art. One of the main problems is showing how such networks arrive at a given conclusion, when there are many alternative “parallel” decisions a network could take, many of which are difficult or impossible to verify because of the richly connected nature of ANNs generally. Such networks contain very high dimensional tensor manipulations and are implemented in tools such as TensorFlow, which are currently not qualified from a safety assurance stand-point. This paper discusses some potential structures, architectures and configurations of ANNs, for example, potential isolation and segregation of such functions to begin to be able to demonstrate the acceptability of safety. The paper begins by discussing the application of systems engineering techniques to such networks to begin to make them tractable. Issues associated with the assurance of such systems are introduced. Further discussion introduces the potential of game theory to describe behaviour of diversely trained competing networks to produce a verifiable result via a search tree, together with some discussion on the possible computational viability of such structures.