To improve efficiency and productivity in the construction industry, governments around the world encourage the development of modular buildings. Meanwhile, worker safety is an important aspect that the construction industry has been concerned with, as there is still much room for improvement. Furthermore, accidents like London’s Grenfell Tower fire in 2017 demonstrate the imperative need to consider construction safety systemically. However, as pointed out in the academic literature and experienced in practice, there is an apparent disconnect between worker safety and building safety. For this reason, this paper sets the ground for a holistic approach to safety by bringing together worker and building safety under the ‘golden thread of information’ as introduced by the British government. The golden thread calls for more integrated safety information management, which could be achieved with the use of Building Information Modelling (BIM) throughout the entire construction and building lifecycle. BIM is regarded as a holistic process of creating and managing information for built assets. Its simulation and visualisation functions can improve safety in all types of construction; this paper concentrates on modular construction.