Towards Interpreting Topic Models with ChatGPT

Emil Rijcken; Floortje Scheepers; Kalliopi Zervanou; Marco Spruit; Pablo Mosteiro; Uzay Kaymak

Towards Interpreting Topic Models with ChatGPT

Emil Rijcken
, Floortje Scheepers
, Kalliopi Zervanou
, Marco Spruit
, Pablo Mosteiro
, Uzay Kaymak

JADS Research

Research output: Contribution to conference › Paper › Academic

2592 Downloads (Pure)

Abstract

Topic modeling has become a popular approach to identify semantic structures in text corpora. Despite its wide applications, interpreting the outputs of topic models remains challenging. This paper presents an initial study regarding a new approach to better understand this output, leveraging the large language model ChatGPT. Our approach is built on a three-stage process where we first use topic modeling to identify the main topics in the corpus. Then, we ask a domain expert to assign themes to these topics and prompt ChatGPT to generate human-readable summaries of the topics. Lastly, we compare the human- and machine-produced interpretations. The domain expert found half of ChatGPT’s descriptions useful. This explorative work demonstrates ChatGPT’s capability to describe topics accurately and provide useful insights if prompted accurately.

Original language	English
Number of pages	7
Publication status	Published - 2023
Event	The 20th World Congress of the International Fuzzy Systems Association - Daegu, Korea, Republic of Duration: 20 Aug 2023 → 24 Aug 2023 Conference number: 20 https://ifsa2023.org/

Conference

Conference	The 20th World Congress of the International Fuzzy Systems Association
Abbreviated title	IFSA
Country/Territory	Korea, Republic of
City	Daegu
Period	20/08/23 → 24/08/23
Internet address	https://ifsa2023.org/

Keywords

Topic Modeling
Large Language Models
ChatGPT
Electronic Health Records
Fuzzy Topic Models
Prompt Engineering

Access to Document

IFSA_InterpretingTopicModelsWithChatGPTFinal author version , 277 KB

Cite this

@conference{6fe80f19eed94ff2b92c510b6e2670e8,

title = "Towards Interpreting Topic Models with ChatGPT",

abstract = "Topic modeling has become a popular approach to identify semantic structures in text corpora. Despite its wide applications, interpreting the outputs of topic models remains challenging. This paper presents an initial study regarding a new approach to better understand this output, leveraging the large language model ChatGPT. Our approach is built on a three-stage process where we first use topic modeling to identify the main topics in the corpus. Then, we ask a domain expert to assign themes to these topics and prompt ChatGPT to generate human-readable summaries of the topics. Lastly, we compare the human- and machine-produced interpretations. The domain expert found half of ChatGPT{\textquoteright}s descriptions useful. This explorative work demonstrates ChatGPT{\textquoteright}s capability to describe topics accurately and provide useful insights if prompted accurately.",

keywords = "Topic Modeling, Large Language Models, ChatGPT, Electronic Health Records, Fuzzy Topic Models, Prompt Engineering",

author = "Emil Rijcken and Floortje Scheepers and Kalliopi Zervanou and Marco Spruit and Pablo Mosteiro and Uzay Kaymak",

year = "2023",

language = "English",

note = "The 20th World Congress of the International Fuzzy Systems Association, IFSA ; Conference date: 20-08-2023 Through 24-08-2023",

url = "https://ifsa2023.org/",

}

TY - CONF

T1 - Towards Interpreting Topic Models with ChatGPT

AU - Rijcken, Emil

AU - Scheepers, Floortje

AU - Zervanou, Kalliopi

AU - Spruit, Marco

AU - Mosteiro, Pablo

AU - Kaymak, Uzay

N1 - Conference code: 20

PY - 2023

Y1 - 2023

N2 - Topic modeling has become a popular approach to identify semantic structures in text corpora. Despite its wide applications, interpreting the outputs of topic models remains challenging. This paper presents an initial study regarding a new approach to better understand this output, leveraging the large language model ChatGPT. Our approach is built on a three-stage process where we first use topic modeling to identify the main topics in the corpus. Then, we ask a domain expert to assign themes to these topics and prompt ChatGPT to generate human-readable summaries of the topics. Lastly, we compare the human- and machine-produced interpretations. The domain expert found half of ChatGPT’s descriptions useful. This explorative work demonstrates ChatGPT’s capability to describe topics accurately and provide useful insights if prompted accurately.

AB - Topic modeling has become a popular approach to identify semantic structures in text corpora. Despite its wide applications, interpreting the outputs of topic models remains challenging. This paper presents an initial study regarding a new approach to better understand this output, leveraging the large language model ChatGPT. Our approach is built on a three-stage process where we first use topic modeling to identify the main topics in the corpus. Then, we ask a domain expert to assign themes to these topics and prompt ChatGPT to generate human-readable summaries of the topics. Lastly, we compare the human- and machine-produced interpretations. The domain expert found half of ChatGPT’s descriptions useful. This explorative work demonstrates ChatGPT’s capability to describe topics accurately and provide useful insights if prompted accurately.

KW - Topic Modeling

KW - Large Language Models

KW - ChatGPT

KW - Electronic Health Records

KW - Fuzzy Topic Models

KW - Prompt Engineering

M3 - Paper

T2 - The 20th World Congress of the International Fuzzy Systems Association

Y2 - 20 August 2023 through 24 August 2023

ER -

Towards Interpreting Topic Models with ChatGPT

Abstract

Conference

Keywords

Access to Document

Fingerprint

Cite this