Hello Guest, please log in for better site access.
| February 2020 | Safety Systems | Volume 28 Number 1 |
| ❰ previous | contents | next ❱ |
SCSC Senior Leadership Forum: Safety Management Systems
Senior leaders, drawn from a wide range of safety-related domains, met in York in October 2019 to discuss best practices in Safety Management Systems.
This was a one-day event held in the Grand Hotel in York and the forum was led by Silas Hays from THI Safety Management Systems. There were 16 delegates representing sectors such as Aviation, Railway, Healthcare, Energy and Academia. The main topic of the event was to discuss the group’s experiences and views on the effectiveness of Safety Management Systems.
Safety Management System Effectiveness
Silas opened the session with a personal perspective of the Nimrod MR2 aircraft and his involvement in the subsequent investigation of the XV230 accident in 2006 that culminated in the Hadden-Cave report. In his account, he noted the challenges he experienced in the absence of a formally recognised Safety Management System.
Each individual in the group introduced themselves and described their own personal objectives for the day, which were then collated.
The group then shared their individual views and perspectives of how effective Safety Management Systems (SMSs) were in their own organisations and a number of themes arose:
● SMSs can place inordinate responsibility on the individual at the lowest level of the organisation, with Top Management being disconnected and out of touch with the actual organisational risks;
● SMSs can be overly prescriptive, and thus promote a tendency for staff to think only in terms of compliance to process, rather than applying creative thought in managing safety risks;
● SMSs can be well defined but the organisation may not have the expertise or budget to implement the processes effectively;
● SMSs can be hindered by the complexities of organisational relationships with other stakeholders especially around contractual and regulatory boundaries;
● SMSs can be inflexible to change, as organisations are reluctant to deviate from established practices even when there may be good reasons to challenge those processes – analogous to the “Five Monkeys” experiment.
“Five Monkeys find themselves in a room with a ladder leading to a bunch of bananas. Any monkey that tries to climb the ladder to reach the bananas is sent sprawling with a spray of cold water that soaks the entire company, and they all sit miserably wet, cold and hungry. An experimenter replaces one of the monkeys with a new monkey. When this new, naïve, monkey tries to reach the bananas, he is soon pulled back down from the ladder by the others, as they can’t bear to be soaked again. Another wet monkey is replaced, and again, when this new monkey tries to reach the bananas she is duly pulled back by the group, including the monkey that has no experience of being soaked. One by one, the original wet monkeys are replaced and eventually, even though there are no monkeys left that have ever experienced the soaking, none are permitted to climb the ladder again.”
Measuring Effectiveness
The means by which the effectiveness of an SMS could be measured was discussed by the group. A number of methods used in practices were proposed by the group, such as audits, reviews, performance metrics etc. and there are various tools to support the assessment such as the CAA SMS Evaluation Tool.
It was however, acknowledged that measuring more subjective aspects such as “Top Management engagement” was more challenging. One suggested method was to use metrics on attendance of management (planned versus actual) at safety meetings. Safety culture questionnaires are also another method to measure effectiveness as they represent an individual’s perception of how well the SMS is understood and being followed.
The use of a “traffic light” dashboard across the SMS processes was also recommended:
●
To provide an “at a glance” overview of the
status of the entire SMS
●
To highlight those areas (marked in red) that
are underperforming and requiring attention
●
To allow trend analysis for a service or
engagement to see if practices are improving over time.
It was also noted that it is important to ensure that the measurements and records that are being captured are meaningful and can be used in an effective and timely manner. An analogy was given to the Grayrigg derailment in 2007 where, it is argued, that video footage from a measurement train could have been used to detect the faulty points, but there was too much data being captured to make this practicable.
Risk Matrices
The use of risk matrices was identified as a problematic area with a number of issues identified:
● There can be many different and conflicting matrices in play such as those for safety, security and financial aspects and it is challenging to get stakeholder agreement on priorities across all of these;
● There are multiple pathways through hazards so for example, should a “worst credible” assessment be favoured over considerations of those that are “most likely”?
It was suggested that a combined risk matrix covering all areas of the organisation’s risk might be a way of reconciling the first point.
Managing Change
It was agreed that an effective Safety Management System not only has to support the management of current perceived risks, but also to facilitate and manage changes however this might occur. For example, new features and modifications of a supply need just as much safety assurance as the original build, but this assurance may be done by different staff in different roles as a project transitions from say, build to service.
Artificial Intelligence
The session concluded with a discussion on how Safety Management Systems could be used to manage risks in projects that have Artificial Intelligence and Machine Learning aspects. A number of challenges were identified:
● AI systems can adapt in real-time; so how can a safety management system be applied to a system that is changing itself?
● AI removes people from being in direct control so the way humans interact with the technology will need to change, possibly in novel ways;
● In systems of systems context, one system could learn “bad habits” from another;
● Innovation is so rapid at the moment that regulation is struggling to keep pace.
Ideas such as having a “fail fast” approach and treating data as a separate consideration were proposed, but the group agreed that this was a challenging area that would require further discussion.
Conclusion
Despite the diversity of sectors that the group represented, it is clear that many of the problems with establishing an effective Safety Management Systems are common. For example, issues with resourcing, management buy-in, effort prioritisation and apportionment of responsibilities, were all issues echoed throughout the group. There were however, useful examples of best practice that were shared, such as the use of an SMS dashboard for trend analysis, that the group could take back to their respective organisations for consideration.
It is apparent that the discipline is now moving into new challenging and unexplored areas with the advent of Artificial Intelligence and Machine Learning technologies. There is a clear desire for Safety Management Systems to be able to cater for these technologies but the evolutionary pathway to achieving this is uncertain.
Report by Paul Hampton, SCSC Newsletter Editor
| ❰ previous | contents | next ❱ |