Unsupervised signature extraction from forensic logs

S.M. Thaler; V. Menkovski; M. Petkovic

doi:10.1007/978-3-319-71273-4_25

Unsupervised signature extraction from forensic logs

S.M. Thaler, V. Menkovski, M. Petkovic

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

5 Citations (Scopus)

1 Downloads (Pure)

Abstract

Signature extraction is a key part of forensic log analysis. It involves recognizing patterns in log lines such that log lines that originated from the same line of code are grouped together. A log signature consists of immutable parts and mutable parts. The immutable parts define the signature, and the mutable parts are typically variable parameter values. In practice, the number of log lines and signatures can be quite large, and the task of detecting and aligning immutable parts of the logs to extract the signatures becomes a significant challenge. We propose a novel method based on a neural language model that outperforms the current state-of-the-art on signature extraction. We use an RNN auto-encoder to create an embedding of the log lines. Log lines embedded in such a way can be clustered to extract the signatures in an unsupervised manner.

Original language	English
Title of host publication	Machine Learning and Knowledge Discovery in Databases
Subtitle of host publication	European Conference, ECML PKDD 2017, Skopje, Macedonia, September 18–22, 2017, Proceedings, Part III
Editors	Michelangelo Ceci, Saso Dzeroski, Donato Malerba, Yasemin Altun, Kamalika Das, Jesse Read, Marinka Zitnik, Jerzy Stefanowski, Taneli Mielikäinen
Place of Publication	Dordrecht
Publisher	Springer
Pages	305-316
Number of pages	12
ISBN (Electronic)	978-3-319-71273-4
ISBN (Print)	978-3-319-71272-7
DOIs	https://doi.org/10.1007/978-3-319-71273-4_25
Publication status	Published - 2017
Event	2017 European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD 2017) - Skopje, Macedonia, The Former Yugoslav Republic of Duration: 18 Sept 2017 → 22 Sept 2017 http://ecmlpkdd2017.ijs.si/index.html

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	10536 LNAI
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	2017 European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD 2017)
Abbreviated title	ECML PKDD 2017
Country/Territory	Macedonia, The Former Yugoslav Republic of
City	Skopje
Period	18/09/17 → 22/09/17
Internet address	http://ecmlpkdd2017.ijs.si/index.html

Keywords

Information forensic
Log clustering
Neural language model
RNN auto-encoder
Signature extraction

Access to Document

10.1007/978-3-319-71273-4_25

Cite this

Thaler, S. M., Menkovski, V., & Petkovic, M. (2017). Unsupervised signature extraction from forensic logs. In M. Ceci, S. Dzeroski, D. Malerba, Y. Altun, K. Das, J. Read, M. Zitnik, J. Stefanowski, & T. Mielikäinen (Eds.), Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2017, Skopje, Macedonia, September 18–22, 2017, Proceedings, Part III (pp. 305-316). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10536 LNAI). Springer. https://doi.org/10.1007/978-3-319-71273-4_25

Thaler, S.M. ; Menkovski, V. ; Petkovic, M. / Unsupervised signature extraction from forensic logs. Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2017, Skopje, Macedonia, September 18–22, 2017, Proceedings, Part III. editor / Michelangelo Ceci ; Saso Dzeroski ; Donato Malerba ; Yasemin Altun ; Kamalika Das ; Jesse Read ; Marinka Zitnik ; Jerzy Stefanowski ; Taneli Mielikäinen. Dordrecht : Springer, 2017. pp. 305-316 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{3cc36cad6a224db3bd918d16d16fa71a,

title = "Unsupervised signature extraction from forensic logs",

abstract = "Signature extraction is a key part of forensic log analysis. It involves recognizing patterns in log lines such that log lines that originated from the same line of code are grouped together. A log signature consists of immutable parts and mutable parts. The immutable parts define the signature, and the mutable parts are typically variable parameter values. In practice, the number of log lines and signatures can be quite large, and the task of detecting and aligning immutable parts of the logs to extract the signatures becomes a significant challenge. We propose a novel method based on a neural language model that outperforms the current state-of-the-art on signature extraction. We use an RNN auto-encoder to create an embedding of the log lines. Log lines embedded in such a way can be clustered to extract the signatures in an unsupervised manner.",

keywords = "Information forensic, Log clustering, Neural language model, RNN auto-encoder, Signature extraction",

author = "S.M. Thaler and V. Menkovski and M. Petkovic",

year = "2017",

doi = "10.1007/978-3-319-71273-4_25",

language = "English",

isbn = "978-3-319-71272-7",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer",

pages = "305--316",

editor = "Michelangelo Ceci and Saso Dzeroski and Donato Malerba and Yasemin Altun and Kamalika Das and Jesse Read and Marinka Zitnik and Jerzy Stefanowski and Taneli Mielik{\"a}inen",

booktitle = "Machine Learning and Knowledge Discovery in Databases",

address = "Germany",

note = "2017 European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD 2017), ECML PKDD 2017 ; Conference date: 18-09-2017 Through 22-09-2017",

url = "http://ecmlpkdd2017.ijs.si/index.html",

}

Thaler, SM, Menkovski, V & Petkovic, M 2017, Unsupervised signature extraction from forensic logs. in M Ceci, S Dzeroski, D Malerba, Y Altun, K Das, J Read, M Zitnik, J Stefanowski & T Mielikäinen (eds), Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2017, Skopje, Macedonia, September 18–22, 2017, Proceedings, Part III. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 10536 LNAI, Springer, Dordrecht, pp. 305-316, 2017 European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD 2017), Skopje, Macedonia, The Former Yugoslav Republic of, 18/09/17. https://doi.org/10.1007/978-3-319-71273-4_25

Unsupervised signature extraction from forensic logs. / Thaler, S.M.; Menkovski, V.; Petkovic, M.
Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2017, Skopje, Macedonia, September 18–22, 2017, Proceedings, Part III. ed. / Michelangelo Ceci; Saso Dzeroski; Donato Malerba; Yasemin Altun; Kamalika Das; Jesse Read; Marinka Zitnik; Jerzy Stefanowski; Taneli Mielikäinen. Dordrecht: Springer, 2017. p. 305-316 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10536 LNAI).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Unsupervised signature extraction from forensic logs

AU - Thaler, S.M.

AU - Menkovski, V.

AU - Petkovic, M.

PY - 2017

Y1 - 2017

N2 - Signature extraction is a key part of forensic log analysis. It involves recognizing patterns in log lines such that log lines that originated from the same line of code are grouped together. A log signature consists of immutable parts and mutable parts. The immutable parts define the signature, and the mutable parts are typically variable parameter values. In practice, the number of log lines and signatures can be quite large, and the task of detecting and aligning immutable parts of the logs to extract the signatures becomes a significant challenge. We propose a novel method based on a neural language model that outperforms the current state-of-the-art on signature extraction. We use an RNN auto-encoder to create an embedding of the log lines. Log lines embedded in such a way can be clustered to extract the signatures in an unsupervised manner.

AB - Signature extraction is a key part of forensic log analysis. It involves recognizing patterns in log lines such that log lines that originated from the same line of code are grouped together. A log signature consists of immutable parts and mutable parts. The immutable parts define the signature, and the mutable parts are typically variable parameter values. In practice, the number of log lines and signatures can be quite large, and the task of detecting and aligning immutable parts of the logs to extract the signatures becomes a significant challenge. We propose a novel method based on a neural language model that outperforms the current state-of-the-art on signature extraction. We use an RNN auto-encoder to create an embedding of the log lines. Log lines embedded in such a way can be clustered to extract the signatures in an unsupervised manner.

KW - Information forensic

KW - Log clustering

KW - Neural language model

KW - RNN auto-encoder

KW - Signature extraction

UR - http://www.scopus.com/inward/record.url?scp=85040226525&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-71273-4_25

DO - 10.1007/978-3-319-71273-4_25

M3 - Conference contribution

SN - 978-3-319-71272-7

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 305

EP - 316

BT - Machine Learning and Knowledge Discovery in Databases

A2 - Ceci, Michelangelo

A2 - Dzeroski, Saso

A2 - Malerba, Donato

A2 - Altun, Yasemin

A2 - Das, Kamalika

A2 - Read, Jesse

A2 - Zitnik, Marinka

A2 - Stefanowski, Jerzy

A2 - Mielikäinen, Taneli

PB - Springer

CY - Dordrecht

T2 - 2017 European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD 2017)

Y2 - 18 September 2017 through 22 September 2017

ER -

Thaler SM, Menkovski V , Petkovic M. Unsupervised signature extraction from forensic logs. In Ceci M, Dzeroski S, Malerba D, Altun Y, Das K, Read J, Zitnik M, Stefanowski J, Mielikäinen T, editors, Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2017, Skopje, Macedonia, September 18–22, 2017, Proceedings, Part III. Dordrecht: Springer. 2017. p. 305-316. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-319-71273-4_25

Unsupervised signature extraction from forensic logs

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML/PKDD)

Cite this

Unsupervised signature extraction from forensic logs

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Activities

European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML/PKDD)

Cite this