Title: AI Safety and Security

Author(s): John McDermid

Publication Event: Publication of Proceedings of the Thirty third Safety-Critical Systems Symposium

Publication Date: 2025-02-01

Resource URL: https://scsc.uk/r3094.pdf

Abstract:

Safety engineering originated in the 1950s, and has responded to many changes in technology, including the progressive use of computers and software to control safety-critical systems and processes. Cyber security originated in the 1960s (with a focus on communications) and, like safety engineering, has evolved to address new challenges and threats. It has long been recognised that exploitation of cyber security vulnerabilities can have consequences for safety, although the two disciplines have not been well integrated. Both now face challenges from the growing use artificial intelligence and machine learning – both severally and jointly – but there are also opportunities. The aim of this paper is to provide a structure for thinking about these challenges and opportunities, especially those where there are shared or overlapping concerns. It is hoped that this will help inform a research agenda for safety and security of systems employing artificial intelligence, and especially machine learning.