On the Effect of Ruleset Tuning and Data Imbalance on Explainable Network Security Alert Classifications: a Case-Study on DeepCASE

Koen T.W. Teuwen; Sam Baggen; Emmanuele Zambon; Luca Allodi

doi:10.1109/EuroSPW67616.2025.00009

On the Effect of Ruleset Tuning and Data Imbalance on Explainable Network Security Alert Classifications: a Case-Study on DeepCASE

Koen T.W. Teuwen, Sam Baggen, Emmanuele Zambon, Luca Allodi

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

2 Downloads (Pure)

Abstract

Automation in Security Operations Centers (SOCs) plays a prominent role in alert classification and incident escalation. However, automated methods must be robust in the presence of imbalanced input data, which can negatively affect performance. Additionally, automated methods should make explainable decisions. In this work, we evaluate the effect of label imbalance on the classification of network intrusion alerts. As our use-case we employ DeepCASE, the state-of-the-art method for automated alert classification. We show that label imbalance impacts both classification performance and correctness of the classification explanations offered by DeepCASE. We conclude tuning the detection rules used in SOCs can significantly reduce imbalance and may benefit the performance and explainability offered by alert post-processing methods such as DeepCASE. Therefore, our findings suggest that traditional methods to improve the quality of input data can benefit automation.

Original language	English
Title of host publication	10th IEEE European Symposium on Security and Privacy Workshops
Publisher	Institute of Electrical and Electronics Engineers
DOIs	https://doi.org/10.1109/EuroSPW67616.2025.00009
Publication status	Accepted/In press - 2 Jul 2025

Funding

This publication is part of the CATRIN, INTERSECT, and SeReNity projects (with numbers NWA.1215.18.003, NWA.1160.18.301, and CS.010) which are (partly) financed by the Dutch Research Council (NWO).

Funders	Funder number
Nederlandse Organisatie voor Wetenschappelijk Onderzoek	NWA.1215.18.003
Nederlandse Organisatie voor Wetenschappelijk Onderzoek	NWA.1160.18.301
Nederlandse Organisatie voor Wetenschappelijk Onderzoek	CS.010

Keywords

Security Operations Center (SOC)
Network Intrusion Detection System (NIDS)
Network Security Alerts
Alert Reduction
Intrusion Detection Ruleset Tuning
Network Intrusion Detection Rules

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1109/EuroSPW67616.2025.00009

On_the_Effect_of_Ruleset_Tuning_and_Data_ImbalanceFinal author version , 400 KBLicence: CC BY

https://arxiv.org/abs/2507.01571Licence: CC BY

Cite this

Teuwen, K. T. W., Baggen, S., Zambon, E., & Allodi, L. (Accepted/In press). On the Effect of Ruleset Tuning and Data Imbalance on Explainable Network Security Alert Classifications: a Case-Study on DeepCASE. In 10th IEEE European Symposium on Security and Privacy Workshops Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/EuroSPW67616.2025.00009

@inproceedings{4b10b9adce7b4f4b807cf72cedc37637,

title = "On the Effect of Ruleset Tuning and Data Imbalance on Explainable Network Security Alert Classifications: a Case-Study on DeepCASE",

abstract = "Automation in Security Operations Centers (SOCs) plays a prominent role in alert classification and incident escalation. However, automated methods must be robust in the presence of imbalanced input data, which can negatively affect performance. Additionally, automated methods should make explainable decisions. In this work, we evaluate the effect of label imbalance on the classification of network intrusion alerts. As our use-case we employ DeepCASE, the state-of-the-art method for automated alert classification. We show that label imbalance impacts both classification performance and correctness of the classification explanations offered by DeepCASE. We conclude tuning the detection rules used in SOCs can significantly reduce imbalance and may benefit the performance and explainability offered by alert post-processing methods such as DeepCASE. Therefore, our findings suggest that traditional methods to improve the quality of input data can benefit automation.",

keywords = "Security Operations Center (SOC), Network Intrusion Detection System (NIDS), Network Security Alerts, Alert Reduction, Intrusion Detection Ruleset Tuning, Network Intrusion Detection Rules",

author = "Teuwen, \{Koen T.W.\} and Sam Baggen and Emmanuele Zambon and Luca Allodi",

year = "2025",

month = jul,

day = "2",

doi = "10.1109/EuroSPW67616.2025.00009",

language = "English",

booktitle = "10th IEEE European Symposium on Security and Privacy Workshops",

publisher = "Institute of Electrical and Electronics Engineers",

address = "United States",

}

On the Effect of Ruleset Tuning and Data Imbalance on Explainable Network Security Alert Classifications: a Case-Study on DeepCASE. / Teuwen, Koen T.W.; Baggen, Sam ; Zambon, Emmanuele et al.
10th IEEE European Symposium on Security and Privacy Workshops. Institute of Electrical and Electronics Engineers, 2025.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - On the Effect of Ruleset Tuning and Data Imbalance on Explainable Network Security Alert Classifications: a Case-Study on DeepCASE

AU - Teuwen, Koen T.W.

AU - Baggen, Sam

AU - Zambon, Emmanuele

AU - Allodi, Luca

PY - 2025/7/2

Y1 - 2025/7/2

N2 - Automation in Security Operations Centers (SOCs) plays a prominent role in alert classification and incident escalation. However, automated methods must be robust in the presence of imbalanced input data, which can negatively affect performance. Additionally, automated methods should make explainable decisions. In this work, we evaluate the effect of label imbalance on the classification of network intrusion alerts. As our use-case we employ DeepCASE, the state-of-the-art method for automated alert classification. We show that label imbalance impacts both classification performance and correctness of the classification explanations offered by DeepCASE. We conclude tuning the detection rules used in SOCs can significantly reduce imbalance and may benefit the performance and explainability offered by alert post-processing methods such as DeepCASE. Therefore, our findings suggest that traditional methods to improve the quality of input data can benefit automation.

AB - Automation in Security Operations Centers (SOCs) plays a prominent role in alert classification and incident escalation. However, automated methods must be robust in the presence of imbalanced input data, which can negatively affect performance. Additionally, automated methods should make explainable decisions. In this work, we evaluate the effect of label imbalance on the classification of network intrusion alerts. As our use-case we employ DeepCASE, the state-of-the-art method for automated alert classification. We show that label imbalance impacts both classification performance and correctness of the classification explanations offered by DeepCASE. We conclude tuning the detection rules used in SOCs can significantly reduce imbalance and may benefit the performance and explainability offered by alert post-processing methods such as DeepCASE. Therefore, our findings suggest that traditional methods to improve the quality of input data can benefit automation.

KW - Security Operations Center (SOC)

KW - Network Intrusion Detection System (NIDS)

KW - Network Security Alerts

KW - Alert Reduction

KW - Intrusion Detection Ruleset Tuning

KW - Network Intrusion Detection Rules

U2 - 10.1109/EuroSPW67616.2025.00009

DO - 10.1109/EuroSPW67616.2025.00009

M3 - Conference contribution

BT - 10th IEEE European Symposium on Security and Privacy Workshops

PB - Institute of Electrical and Electronics Engineers

ER -

On the Effect of Ruleset Tuning and Data Imbalance on Explainable Network Security Alert Classifications: a Case-Study on DeepCASE

Abstract

Funding

Keywords

UN SDGs

Access to Document

Fingerprint

Cite this