Keyword-Assisted Topic Models

Abstract

In recent years, fully automated content analysis based on probabilistic topic models has become popular among social scientists because of their scalability. The unsupervised nature of the models makes them suitable for exploring topics in a corpus without prior knowledge. However, researchers find that these models often fail to measure specific concepts of substantive interest by inadvertently creating multiple topics with similar content and combining distinct themes into a single topic. In this paper, we empirically demonstrate that providing a small number of keywords can substantially enhance the measurement performance of topic models. An important advantage of the proposed keyword assisted topic model (keyATM) is that the specification of keywords requires researchers to label topics prior to fitting a model to the data. This contrasts with a widespread practice of post-hoc topic interpretation and adjustments that compromises the objectivity of empirical findings. In our application, we find that keyATM provides more interpretable results, has better document classification performance, and is less sensitive to the number of topics than the standard topic models. Finally, we show that keyATM can also incorporate covariates and model time trends. An open-source software package is available for implementing the proposed methodology.

Paper

Shusei Eshima, Kosuke Imai, and Tomoya Sasaki. 2024. “Keyword-Assisted Topic Models.” American Journal of Political Science 68(2): 730-750. https://doi.org/10.1111/ajps.12779, arXiv version.

Software

keyATM Website

Presentation

  • American Political Science Association, Annual Meeting
    • San Francisco, CA, September 2020
    • Presentation (Slides)
  • Annual Conference of the Society for Political Methodology
    • Virtual conference, July 2020
    • Poster
  • International Methods Colloquium
    • Online colloquium, May 2020
    • Presentation (Slides, Video)
  • Midwest Political Science Association, Annual Meeting
    • Chicago, IL, April 2020
    • Presentation (cancelled due to the COVID-19)
  • Applied Statistics Workshop
    • Institute for Quantitative Social Science, February 2020
    • Presentation
  • Japanese Society for Quantitative Political Science, Winter Meeting
    • Waseda University, Tokyo, January 2020
    • Presentation

Replication Materials

Shusei Eshima, Kosuke Imai, and Tomoya Sasaki. 2023. “Replication Data for: Keyword-Assisted Topic Models.” https://doi.org/10.7910/DVN/RKNNVL. Harvard Dataverse.