Ensuring the safety of cyber-physical systems (CPS) against cybersecurity threats is essential in safety-critical sectors. To this end, the systematic derivation of clear and robust safety claims is needed to demonstrate that safety properties are preserved even when the system is under attack or partially compromised. It is also necessary to demonstrate regulatory compliance. In this paper, we present, a framework for the automatic identification of safety-critical attacks targeting CPS and generation of Assurance Case Fragments (ACF). To identify attacks that can compromise safety we use a combination of STPA-Sec, STRIDE, and formal verification. This allows us to determine sequences of attack steps leading to safety violations (i.e. threat scenarios) and the attack paths that enable them on the system architecture. We automatically derive ACFs in Goal Structuring Notation (GSN), which show whether the CPS can operate within an acceptable risk whilst under attack and that security controls in place are adequate. To illustrate the application of our approach we use the example of a railway traffic control system and discuss how the derived ACFs demonstrate system safety as well as the soundness of our integrated approach to safety and security analysis.