Safe AI Working Group
AI is becoming increasingly prevalent and influential in safety systems, including physical systems such as autonomous vehicles, medical devices, and industrial control systems and also software or web-based systems such as search engines and facial recognition systems. AI opens opportunities for improving safety and enhancing the assurance of safety systems but also poses new challenges and risks. These challenges include unpredictability, increasing complexity, dependence on good data, bias ethical issues and compliance with social norms. For example, recent incidents in autonomous vehicle trials have highlighted the unpredictable nature of AI decisions in real-world scenarios, underscoring the urgent need for robust safety frameworks. Additionally, the rapid adoption of AI in healthcare, where AI systems are making increasingly complex decisions, has raised significant concerns.
Generative AI opens opportunities for AI-assisted developments, verifications, safety assessments and safety case generation. The wide range of applicability emphasizes the necessity for a deep and comprehensive understanding of AI's unique risks and behaviours.
Why is the SCSC involved?
Due to the nature of the SCSC being cross domain and non-profit making, it provides an opportunity for experts from different domains and academia to come together to network, share best practice and experience and develop guidance where needed.
What will the Working Group do?
The SCSC Safe AI Working Group (SAIWG) will aim to capture cross-domain best practice and guidance on key topics within the design, evaluation, assurance, and approval of safety systems that use or are developed using AI, bringing together emerging standards and key results from the incredible amount of research being conducted into AI safety.
The working group will kick-off at SSS’24. From then the SAIWG will conduct regular meetings, workshops, and publications to share knowledge and experience on various topics related to AI and safety systems, such as co-ordination of safety with other disciplines, evaluation of risk, and mapping of terminology and language.
Joining instructions will be announced at SSS’24.
To produce guidance on methods to integrate AI and safety systems, in a way that reflects the state-of-the-art and the state-of-the-practice. This guidance will help engineers and stakeholders to be comprehensively informed about the safety and assurance of AI-enabled systems with the aim of developing and operating such systems so as to reduce harm
Our long-term vision is to pioneer a future where the integration of AI in safety systems not only enhances operational efficiency and innovation but also upholds the highest standards of safety and ethical responsibility. Through our work, we aim to foster a paradigm shift in how AI is perceived and utilized in critical sectors, contributing to a safer, more reliable, and ethically conscious technological landscape.
Terms of Reference and working group goals will be made available at SSS24.
Initially the WG will focus on establishing an overview of the state of play for the use of AI in safety systems development and assurance. There will be an opportunity to discuss the Focus Areas for SAIWG at SSS’24 but these could include:
- Risk Assessment: Understanding the potential risks and vulnerabilities introduced by AI in safety systems. This could involve developing new risk assessment methodologies tailored for AI.
- Standards and Regulations: Keeping abreast of emerging standards and regulations related to AI and safety systems. The group could contribute to the development of these standards.
- Ethical Considerations: Exploring the ethical implications of using AI in safety systems. This could include issues of transparency, accountability, and fairness.
- Testing and Validation: Developing new approaches for testing and validating AI in safety systems to ensure they behave as expected.
- Interdisciplinary Collaboration: Facilitating collaboration between AI experts, safety engineers, ethicists, and other stakeholders to address the multifaceted challenges posed by AI in safety systems.
© SCSC 2024