On the Effect of Ruleset Tuning and Data Imbalance on Explainable Network Security Alert Classifications: a Case-Study on DeepCASE

Koen T.W. Teuwen; Sam Baggen; Emmanuele Zambon; Luca Allodi

doi:10.1109/EuroSPW67616.2025.00009

On the Effect of Ruleset Tuning and Data Imbalance on Explainable Network Security Alert Classifications: a Case-Study on DeepCASE

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

42 Downloads (Pure)

Abstract

Automation in Security Operations Centers (SOCs) plays a prominent role in alert classification and incident escalation. However, automated methods must be robust in the presence of imbalanced input data, which can negatively affect performance. Additionally, automated methods should make explainable decisions. In this work, we evaluate the effect of label imbalance on the classification of network intrusion alerts. As our use-case we employ DeepCASE, the state-of-the-art method for automated alert classification. We show that label imbalance impacts both classification performance and correctness of the classification explanations offered by DeepCASE. We conclude tuning the detection rules used in SOCs can significantly reduce imbalance and may benefit the performance and explainability offered by alert post-processing methods such as DeepCASE. Therefore, our findings suggest that traditional methods to improve the quality of input data can benefit automation.

Original language	English
Title of host publication	10th IEEE European Symposium on Security and Privacy Workshops, EuroS&PW 2025
Publisher	Institute of Electrical and Electronics Engineers
Pages	16-29
Number of pages	14
ISBN (Electronic)	979-8-3315-9546-3
DOIs	https://doi.org/10.1109/EuroSPW67616.2025.00009
Publication status	Published - 1 Sept 2025
Event	10th IEEE European Symposium on Security and Privacy Workshops, EUROS&PW 2025 - Venice, Italy Duration: 30 Jun 2025 → 4 Jul 2025

Conference

Conference	10th IEEE European Symposium on Security and Privacy Workshops, EUROS&PW 2025
Abbreviated title	EUROS&PW 2025
Country/Territory	Italy
City	Venice
Period	30/06/25 → 4/07/25

Funding

Parts of this work were derived from a prior M.Sc. graduation project [41]. The authors thank the unnamed SOC for their cooperation and for making data available for analysis, and Thijs van Ede discussing DeepCASE with us. This publication is part of the CATRIN, INTERSECT, and SeReNity projects (with numbers NWA.1215.18.003, NWA.1160.18.301, and CS.010) which are (partly) financed by the Dutch Research Council (NWO). For the purpose of Open Access, a CC-BY 4.0 public copyright license is applied to any Author Accepted Manuscript version arising from this submission.

Funders	Funder number
Nederlandse Organisatie voor Wetenschappelijk Onderzoek	NWA.1215.18.003
Nederlandse Organisatie voor Wetenschappelijk Onderzoek	NWA.1160.18.301
Nederlandse Organisatie voor Wetenschappelijk Onderzoek	CS.010

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 16 Peace, Justice and Strong Institutions

Keywords

Security Operations Center (SOC)
Network Intrusion Detection System (NIDS)
Network Security Alerts
Alert Reduction
Intrusion Detection Ruleset Tuning
Network Intrusion Detection Rules

Access to Document

10.1109/EuroSPW67616.2025.00009

On_the_Effect_of_Ruleset_Tuning_and_Data_ImbalanceFinal author version , 400 KBLicence: CC BY
pdfFinal published version, 411 KBLicence: TAVERNE

https://arxiv.org/abs/2507.01571Licence: CC BY

Cite this

Teuwen, K. T. W., Baggen, S., Zambon, E., & Allodi, L. (2025). On the Effect of Ruleset Tuning and Data Imbalance on Explainable Network Security Alert Classifications: a Case-Study on DeepCASE. In 10th IEEE European Symposium on Security and Privacy Workshops, EuroS&PW 2025 (pp. 16-29). Article 11129503 Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/EuroSPW67616.2025.00009

@inproceedings{4b10b9adce7b4f4b807cf72cedc37637,

title = "On the Effect of Ruleset Tuning and Data Imbalance on Explainable Network Security Alert Classifications: a Case-Study on DeepCASE",

abstract = "Automation in Security Operations Centers (SOCs) plays a prominent role in alert classification and incident escalation. However, automated methods must be robust in the presence of imbalanced input data, which can negatively affect performance. Additionally, automated methods should make explainable decisions. In this work, we evaluate the effect of label imbalance on the classification of network intrusion alerts. As our use-case we employ DeepCASE, the state-of-the-art method for automated alert classification. We show that label imbalance impacts both classification performance and correctness of the classification explanations offered by DeepCASE. We conclude tuning the detection rules used in SOCs can significantly reduce imbalance and may benefit the performance and explainability offered by alert post-processing methods such as DeepCASE. Therefore, our findings suggest that traditional methods to improve the quality of input data can benefit automation.",

keywords = "Security Operations Center (SOC), Network Intrusion Detection System (NIDS), Network Security Alerts, Alert Reduction, Intrusion Detection Ruleset Tuning, Network Intrusion Detection Rules",

author = "Teuwen, \{Koen T.W.\} and Sam Baggen and Emmanuele Zambon and Luca Allodi",

year = "2025",

month = sep,

day = "1",

doi = "10.1109/EuroSPW67616.2025.00009",

language = "English",

pages = "16--29",

booktitle = "10th IEEE European Symposium on Security and Privacy Workshops, EuroS\&PW 2025",

publisher = "Institute of Electrical and Electronics Engineers",

address = "United States",

note = "10th IEEE European Symposium on Security and Privacy Workshops, EUROS\&PW 2025, EUROS\&PW 2025 ; Conference date: 30-06-2025 Through 04-07-2025",

}

Teuwen, KTW , Baggen, S , Zambon, E & Allodi, L 2025, On the Effect of Ruleset Tuning and Data Imbalance on Explainable Network Security Alert Classifications: a Case-Study on DeepCASE. in 10th IEEE European Symposium on Security and Privacy Workshops, EuroS&PW 2025., 11129503, Institute of Electrical and Electronics Engineers, pp. 16-29, 10th IEEE European Symposium on Security and Privacy Workshops, EUROS&PW 2025, Venice, Italy, 30/06/25. https://doi.org/10.1109/EuroSPW67616.2025.00009

On the Effect of Ruleset Tuning and Data Imbalance on Explainable Network Security Alert Classifications: a Case-Study on DeepCASE. / Teuwen, Koen T.W.; Baggen, Sam ; Zambon, Emmanuele et al.
10th IEEE European Symposium on Security and Privacy Workshops, EuroS&PW 2025. Institute of Electrical and Electronics Engineers, 2025. p. 16-29 11129503.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - On the Effect of Ruleset Tuning and Data Imbalance on Explainable Network Security Alert Classifications

T2 - 10th IEEE European Symposium on Security and Privacy Workshops, EUROS&PW 2025

AU - Teuwen, Koen T.W.

AU - Baggen, Sam

AU - Zambon, Emmanuele

AU - Allodi, Luca

PY - 2025/9/1

Y1 - 2025/9/1

N2 - Automation in Security Operations Centers (SOCs) plays a prominent role in alert classification and incident escalation. However, automated methods must be robust in the presence of imbalanced input data, which can negatively affect performance. Additionally, automated methods should make explainable decisions. In this work, we evaluate the effect of label imbalance on the classification of network intrusion alerts. As our use-case we employ DeepCASE, the state-of-the-art method for automated alert classification. We show that label imbalance impacts both classification performance and correctness of the classification explanations offered by DeepCASE. We conclude tuning the detection rules used in SOCs can significantly reduce imbalance and may benefit the performance and explainability offered by alert post-processing methods such as DeepCASE. Therefore, our findings suggest that traditional methods to improve the quality of input data can benefit automation.

AB - Automation in Security Operations Centers (SOCs) plays a prominent role in alert classification and incident escalation. However, automated methods must be robust in the presence of imbalanced input data, which can negatively affect performance. Additionally, automated methods should make explainable decisions. In this work, we evaluate the effect of label imbalance on the classification of network intrusion alerts. As our use-case we employ DeepCASE, the state-of-the-art method for automated alert classification. We show that label imbalance impacts both classification performance and correctness of the classification explanations offered by DeepCASE. We conclude tuning the detection rules used in SOCs can significantly reduce imbalance and may benefit the performance and explainability offered by alert post-processing methods such as DeepCASE. Therefore, our findings suggest that traditional methods to improve the quality of input data can benefit automation.

KW - Security Operations Center (SOC)

KW - Network Intrusion Detection System (NIDS)

KW - Network Security Alerts

KW - Alert Reduction

KW - Intrusion Detection Ruleset Tuning

KW - Network Intrusion Detection Rules

UR - https://www.scopus.com/pages/publications/105016537275

U2 - 10.1109/EuroSPW67616.2025.00009

DO - 10.1109/EuroSPW67616.2025.00009

M3 - Conference contribution

SP - 16

EP - 29

BT - 10th IEEE European Symposium on Security and Privacy Workshops, EuroS&PW 2025

PB - Institute of Electrical and Electronics Engineers

Y2 - 30 June 2025 through 4 July 2025

ER -

On the Effect of Ruleset Tuning and Data Imbalance on Explainable Network Security Alert Classifications: a Case-Study on DeepCASE

Abstract

Conference

Funding

UN SDGs

Keywords

Access to Document

Other files and links

Fingerprint

Cite this