AI in a Safety Workflow

This event is aimed at safety engineers explaining how to incorporate AI-based tools, methods and techniques into a development or maintenance workflow.

This 1-day event has a series of talks by those working with AI, explaining how the most benefit can be gained and appropriate safeguards and oversight put in place.

Talks include:

Hugues Bonnin, AUMOVIO - Automotive safety with AI workflow: ISO standards perspective - some news (and some thoughts) from ISO 26262 3ed, future ISO 8800 and ISO/IEC TS 22440 on the subject of AI based tools evaluation and qualification

Ben Fulford, BMT - Stop Reviewing AI Outputs. Start Engineering Them: How to design an AI workflow fit for real safety work

Sybert Stroeve, Royal Netherlands Aerospace Centre - A holistic assessment framework for the design of AI-based automation

Graham Sutherland, Canetia Analytics, Inc. - AI-Enabled Digital Twin Monitoring for Transit Structures: Continuous Insight for Reliability and System Performance

TBA, TÜV SÜD - Starting the AI Governance Pathway: From Shadow AI to Trusted AI

Speaker Bios

Hugues Bonnin is a Senior Expert in Connected Services Safety at AUMOVIO. He has spent more than 30 years in systems engineering for railways, aerospace and automotive domains. Safety has always been at the heart of his work, whether regarding the technological choices, the design, or the certification activities of critical systems. Since 2017 he has worked at AUMOVIO where he serves as a senior safety expert in the field of connected and AI-based automotive products and services. He is active in ISO standardization working groups ISO TC22/SC32/WG8 (safety of E/E automotive systems) and WG14 (AI safety for automotive), where he combines several leading roles: WG8 data safety topic leader, liaison between WG18 and WG14, WG14 French delegation leader, WG14 AI based tool evaluation and qualification part group.

 

Ben Fulford is on a mission to modernise safety engineering: rebuilding the tools, processes and training it runs on. He has over 15 years in functional safety across defence, maritime, nuclear and aerospace, including safety manager on the Astute-class submarine Platform Management System and Independent Safety Auditor on radar, software and autonomous programmes. He was lead author of the AI-HAZOP paper at SSS '26 and has since built a knowledge-graph HAZOP tool, with a paper due in 2027. He designed BMT's prompt engineering programme and writes Modernising Safety Engineering on Substack. This is his fifth SCSC presentation.

 

Sybert Stroeve received a PhD from Delft University of Technology for research on neural network control of arm movements, worked at the Foundation for Neural Networks of Radboud University, and has been at the Royal Netherlands Aerospace Centre NLR since 2000. As senior scientist at the Safety & Human Performance department he has conducted a wide range of research for safety and human performance modelling and risk assessments for air transport operations, addressing runway incursion risk, airborne collision avoidance systems, detect and avoid systems, safety culture, and AI-based systems.

 

Graham Sutherland is a Chartered Engineer with extensive experience in the safety assurance of complex railway systems, including CBTC on the London Underground, Crossrail's central section, and Singapore's North-East Line. He is currently Chief Innovation Officer at Canetia Analytics Inc., a company supported by the US National Science Foundation and dedicated to improving the safety of critical infrastructure. Its patented AI platform monitors the structural health of bridges and other infrastructure for owners across the US and UK. Graham studied Electrical Engineering at the University of Liverpool, completed a PhD in fusion devices at UKAEA Culham, and later earned an MBA from Warwick Business School. He is no stranger to the SCSC community, having presented at the 2019 Symposium in Bristol on improving the safety of neural-network-based systems, and at the 2020 Symposium in York on integrity assurance for large datasets. Since then, he has focused on putting those ideas into practice.

Talk Abstracts

Hugues Bonnin, AUMOVIO - Automotive safety with AI workflow: ISO standards perspective - some news (and some thoughts) from ISO 26262 3ed, future ISO 8800 and ISO/IEC TS 22440 on the subject of AI based tools evaluation and qualification

Arguably the recent advancements in AI will have a major impact on safety engineering practices in the coming years. On one hand, AI may help process increasingly large amounts of information, while facilitating the completion of various tasks; on the other hand, AI generally offers no guarantee of quality and potentially reinforces certain undesired biases or human complacency. This presentation proposes an overview of the safety activities that AI is likely to impact in the automotive domain. First, we review how the concept of “evaluation and qualification” of tools has been historically introduced in the safety standards. Then, we look at several safety standards regarding AI. Thirdly, we focus on the “AI based tools” case – an ongoing work in ISO standardization. To conclude, we assess this standardization work, highlighting its strengths, weaknesses and blind spots.

Ben Fulford, BMT - Stop Reviewing AI Outputs. Start Engineering Them: How to design an AI workflow fit for real safety work

When we first used a large language model on a HAZOP, it got about 80% of the study right. That was a workable place to be. It saved time, the mistakes were easy to spot, and an engineer checking every line was a sensible safeguard. The tool I have built since is a different story. It models the system as a knowledge graph and produces thousands of lines of accurate HAZOP output in minutes. Nobody can properly review every line of that, and asking them to turns human oversight into theatre. So, the question changes. Not "how does a human check all of this?" but "how do we engineer the workflow around the model so we can trust what comes out of it?" That means deciding what the model can see, what constrains it, what checks every output before a person does, and where the engineer's time is best spent. Drawing on what I learnt building that tool, this talk sets out the layers of engineering around the model, how an AI judge can triage output for the reviewer, and what evidence each layer gives a safety argument.

Sybert Stroeve, Royal Netherlands Aerospace Centre - A holistic assessment framework for the design of AI-based automation

There is a need to ensure that the application of artificial intelligence (AI) in increasingly automated operations is safe, human-centric, and trustworthy. To this end, this presentation provides an innovative holistic assessment framework to support certification-aware design of AI-based sociotechnical systems with a range of levels of automation along multiple design stages from low to high technology and human readiness levels (TRLs/HRLs). The holistic scope considers the following key performance areas (KPAs): safety, resilience, security, Human Factors, accountability, responsibility, liability, efficiency, societal sustainability, and environmental sustainability. The core of the framework is a seven-step cycle that assesses the KPAs for critical scenarios and evaluates the combined performance, including uncertainty and trade-offs. This provides feedback to either adapt the design at the same TRL/HRL or refine it at higher TRLs/HRLs. The framework is enacted by  a toolbox of assessment methods for the KPAs. The framework has been developed in the aviation domain, but it is formulated in a generic manner, enabling application to various AI techniques and operational domains. Its application is illustrated in detail for an air traffic management use case that employs an AI-based system to support air traffic controllers in sequencing aircraft

Graham Sutherland, Canetia Analytics, Inc. - "AI-Enabled Digital Twin Monitoring for Transit Structures: Continuous Insight for Reliability and System Performance"

Bridges, tunnels, and culverts are essential to safe passenger and freight operations, yet their condition between inspections often remains unclear. Reliance on periodic visual inspections and scheduled maintenance can allow hidden deterioration to progress, increasing the risks of disruption, emergency repair, and premature intervention. Canetia is developing a National Science Foundation-supported system that combines low-cost IoT vibration sensing, structure-specific machine learning, and generative-AI digital-twin inference. An AI-assisted interface provides engineers, owners, and planners with natural-language summaries of asset condition and recommended actions. Structure-mounted sensors stream traffic-induced vibration data to cloud-based analytics. Machine learning establishes baselines, filters routine variation, detects degradation, and forecasts service life. Drawing on reports, sensor data, and environmental context, the digital twin supports remote assessment and targeted inspection. The system operates on two Missouri Department of Transportation road bridges and at multiple sites in the UK. Validation and field deployment are underway at the University of California San Diego’s Jacobs School of Engineering. For rail and transit agencies, continuous condition awareness can improve safety and availability while reducing costs. Earlier detection can limit closures, emergency repairs, site visits, and reactive maintenance; defer capital replacement; and guide assurance-focused action. Applications include extending the service life of aging structures; conducting post-event assessments; monitoring high-traffic, limited-access corridors; supporting systemwide investment and remediation planning; and tracking assets during adjacent construction. Supported by US NSF SBIR Phase II Award No. 2423274, this work demonstrates how AI-enabled monitoring can strengthen reliability, inspection, and investment decisions. Canetia acknowledges Parsons Corporation, NVIDIA’s Inception Program, and AWS. The presentation will highlight results, validation, and lessons for scaled deployment. Keywords: structural health monitoring, digital twin, AI, IoT sensors, vibration monitoring, assurance, predictive maintenance, asset management, NSF SBIR.

TBA, TÜV SÜD - Starting the AI Governance Pathway: From Shadow AI to Trusted AI

Artificial Intelligence is already embedded within most organisations, whether they realise it or not. While boardrooms debate AI strategy, employees are using public generative AI tools to increase productivity, software vendors are quietly embedding AI capabilities into existing products, and business processes are increasingly influenced by algorithmic decision-making. The result is a growing landscape of Shadow AI and Hidden AI: AI that is being used without visibility, oversight, accountability, or assurance. For safety-critical and assurance-driven sectors, this presents a significant challenge. Traditional governance approaches were developed for predictable software and deterministic systems. Modern AI systems are dynamic, adaptive, and often opaque. Many organisations therefore face a widening gap between the speed of AI adoption and their ability to understand, govern, and assure its use. This session explores how organisations can begin their AI governance journey without waiting for perfect regulation, complete standards, or mature AI management systems. It examines the practical realities of discovering AI already in use, identifying governance blind spots, understanding differing organisational roles as AI Users, Providers, and Producers, and establishing proportionate controls that create visibility, accountability, and trust.