SoK: How Artificial-Intelligence Incidents Can Jeopardize Safety and Security

Richard May, Jacob Krüger, Thomas Leich

Onderzoeksoutput: Hoofdstuk in Boek/Rapport/CongresprocedureConferentiebijdrageAcademicpeer review

2 Downloads (Pure)

Samenvatting

In the past years, a growing number of highly-automated systems has build on Artificial-Intelligence (AI) capabilities, for example, self-driving vehicles or predictive health-state diagnoses. As for any software system, there is a risk that misbehavior occurs (e.g., system failure due to bugs) or that malicious actors aim to misuse the system (e.g., generating attack scripts), which can lead to safety and security incidents. While software safety and security incidents have been studied in the past, we are not aware of research focusing on the specifics of AI incidents. With this paper, we aim to shed light on this gap through a case survey of 240 incidents that we elicited from four datasets comprising safety and security incidents involving AI from 2014 to 2023. Using manual data analyses and automated topic modeling, we derived relevant topics as well as the major issues and contexts in which the incidents occurred. We find that the topic of AI incidents is, not surprisingly, becoming more and more relevant, particularly in the contexts of autonomous driving and process-automation robotics. Regarding security and its intersection with safety, most incidents connect to generative AI (i.e., large-language models, deep fakes) and computer-vision systems (i.e., facial recognition). This emphasizes the importance of security to also ensure safety in the context of AI systems, with our results further revealing a high number of serious consequences (system compromise, human injuries) and major violations of confidentiality, integrity, availability, as well as authorization. We hope to support practitioners and researchers in understanding major safety and security issues to support the development of more secure, safe, and trustworthy AI systems.
Originele taal-2Engels
TitelARES '24
SubtitelProceedings of the 19th International Conference on Availability, Reliability and Security
UitgeverijAssociation for Computing Machinery, Inc
Aantal pagina's12
ISBN van elektronische versie979-8-4007-1718-5
DOI's
StatusGepubliceerd - 30 jul. 2024
Evenement19th International Conference on Availability, Reliability and Security, ARES 2024 - Vienna, Oostenrijk
Duur: 30 jul. 20242 aug. 2024

Congres

Congres19th International Conference on Availability, Reliability and Security, ARES 2024
Land/RegioOostenrijk
StadVienna
Periode30/07/242/08/24

Vingerafdruk

Duik in de onderzoeksthema's van 'SoK: How Artificial-Intelligence Incidents Can Jeopardize Safety and Security'. Samen vormen ze een unieke vingerafdruk.

Citeer dit