Samenvatting
In the past years, a growing number of highly-automated systems has build on Artificial-Intelligence (AI) capabilities, for example, self-driving vehicles or predictive health-state diagnoses. As for any software system, there is a risk that misbehavior occurs (e.g., system failure due to bugs) or that malicious actors aim to misuse the system (e.g., generating attack scripts), which can lead to safety and security incidents. While software safety and security incidents have been studied in the past, we are not aware of research focusing on the specifics of AI incidents. With this paper, we aim to shed light on this gap through a case survey of 240 incidents that we elicited from four datasets comprising safety and security incidents involving AI from 2014 to 2023. Using manual data analyses and automated topic modeling, we derived relevant topics as well as the major issues and contexts in which the incidents occurred. We find that the topic of AI incidents is, not surprisingly, becoming more and more relevant, particularly in the contexts of autonomous driving and process-automation robotics. Regarding security and its intersection with safety, most incidents connect to generative AI (i.e., large-language models, deep fakes) and computer-vision systems (i.e., facial recognition). This emphasizes the importance of security to also ensure safety in the context of AI systems, with our results further revealing a high number of serious consequences (system compromise, human injuries) and major violations of confidentiality, integrity, availability, as well as authorization. We hope to support practitioners and researchers in understanding major safety and security issues to support the development of more secure, safe, and trustworthy AI systems.
Originele taal-2 | Engels |
---|---|
Titel | ARES '24 |
Subtitel | Proceedings of the 19th International Conference on Availability, Reliability and Security |
Uitgeverij | Association for Computing Machinery, Inc |
Aantal pagina's | 12 |
ISBN van elektronische versie | 979-8-4007-1718-5 |
DOI's | |
Status | Gepubliceerd - 30 jul. 2024 |
Evenement | 19th International Conference on Availability, Reliability and Security, ARES 2024 - Vienna, Oostenrijk Duur: 30 jul. 2024 → 2 aug. 2024 |
Congres
Congres | 19th International Conference on Availability, Reliability and Security, ARES 2024 |
---|---|
Land/Regio | Oostenrijk |
Stad | Vienna |
Periode | 30/07/24 → 2/08/24 |