Title: Towards a Safety Argument for Autonomous Systems that Use Machine Learning

Author(s): Chris Allsopp, David H Smith, Elizabeth Lennon, James McCloskey, Lee Ramsay, Sam Jenkins

Publication Event: Proceedings of the Twenty-seventh Safety-Critical Systems Symposium, Bristol, UK

Publication Date: 2019-02-07

Resource URL: https://scsc.uk/r1082.pdf

Abstract:

Artificial Intelligence (AI) (including approaches such as Machine Learning (ML)) is making rapid progress in a variety of applications. It is highly likely to be used in safety-related and possibly safety-critical systems. There is a need to consider how to make Safety Arguments for systems that exploit ML techniques; more generally, there is a need to make Safety Arguments for Autonomous Systems that make use of them. This paper presents the interim work undertaken by a consortium led by Frazer-Nash Consultancy in support of the Defence Science and Technology Laboratory (Dstl) to determine the types of Safety Argument that can and cannot be made about Autonomous Systems where nondeterministic and emergent behaviours are present. It includes consideration of the properties of relevant ML techniques, along with wider system-level attributes. The focus is deliberately on generic arguments, rather than the development of an argument (or arguments) for a very specific system (or systems).