Title: Safe AI Systems

Author(s): Mike Parsons

Publication Event: Proceedings of the Thirty Second Safety-Critical Systems Symposium

Publication Date: 2024-02-12

Resource URL: https://scsc.uk/r1898.pdf

Abstract:

The rapid evolution of AI has shocked the world and further step-changes are likely. Not only does this change the picture regarding safety-critical applications utilizing AI: self-driving vehicles, drones and medical image recognition systems, but it has the potential to revolutionize the way we produce critical systems, everything from AI-generated designs to AI-authored safety cases. System safety practices and approaches must adapt to deal with and make best use of, these new AI applications and uses, and ways must be found to assure systems utilizing AI which are meaningful, understandable and trustworthy. Systems are also getting ever more complex and connected, and in some cases, more fragile. We do not have established tools and techniques to analyse and manage such systems: they are too opaque, have too many parts, interfaces and interactions for the old techniques to work. Two new SCSC Working Groups, the Safe AI Working Group (SAIWG) and Safer Complex Systems WG (SCSWG) have been set up to look at the issues and produce new guidance in these areas.