Artificial Intelligence (AI) is a key technology in autonomous systems. AI has undoubted benefits, and the capabilities of many systems are very impressive, but AI poses challenges to classical safety processes and principles. This paper surveys the challenges in assessing the safety of AI, and considers historical and more recent work on AI safety assessment. Many of the issues are generic, i.e. do not depend on the application domain, but some aspects such as compliance with standards are domain specific. The paper is generic, so far as is possible, but reflects the requirements and standards of the maritime sector, where it is helpful or necessary to do so. Overall the aim of the paper is both to outline the state of the art in assessing the safety of AI, and to indicate important research directions.