保留正文与补采附件。源页面当前未重新核验；具体限制见交付清单。

Revisit consent button We value your privacy We use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking "Accept All", you consent to our use of cookies. Customise Reject All Accept All Customise Consent Preferences We use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below. The cookies that are categorised as "Necessary" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... Show more NecessaryAlways Active Necessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.

Functional Functional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.

Analytics Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.

Performance Performance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors.

Advertisement Advertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns.

Uncategorised Other uncategorised cookies are those that are being analysed and have not been classified into a category as yet. No cookies to display. Reject All Save My Preferences Accept All Toggle SearchToggle Mobile Navigation Publish AI, ML & data-science insights to a global community of data professionals. Sign inSubmit an Article LatestEditor's PicksNewsletterShipAI

Write For TDS LinkedInXToggle Search Data Science

Topic Modelling with BERTtopic in Python

Hands-on tutorial on modeling political statements with a state-of-the-art transformer-based topic model Petr Koráb April 1, 20245 min read Photo by Harryarts on Freepik Photo by Harryarts on Freepik Topic modeling (i.e., topic identification in a corpus of text data) has developed quickly since the Latent Dirichlet Allocation (LDA) model was published. This classic topic model, however, does not well capture the relationships between words because it is based on the statistical concept of a bag of words. Recent embedding-based Top2Vec and BERTopic models address its drawbacks by exploiting pre-trained language models to generate topics. In this article, we'll use Maarten Grootendorst's (2022) BERTopic to identify the terms representing topics in political speech transcripts. It outperforms most traditional and modern topic models in topic modeling metrics on various corpora and has been used in companies, academia (Chagnon, 2024), and the public sector. We'll explore in Python code:

Learn this step by step with the interactive AI and Data Scientist roadmap.

1. Example data

As an example dataset, we'll use the Empoliticon: Political Speeches-Context & Emotion dataset, released under the Attribution 4.0 International license, as part of the Efat et al. (2023) paper. It contains 2010 transcripts of political speeches from the presidents/ prime ministers of the USA, UK, China, and Russia. To make the topic model more focused, the subset only includes the 556 speeches of leaders from Russia: Source: Emopoliticon: Political Speeches-Context Emotion dataset Source: Emopoliticon: Political Speeches-Context & Emotion dataset

2. Data pre-processing

Working with text datasets is complex. Just cleaning involves several steps that should systematically remove all unnecessary information from the dataset. Check all requirements for this project here. 2.1. Fixing mojibake errors Mojibake is a Japanese word for the confusing text that results from character-encoding errors. Here is an example: Mojibake example Mojibake example It is useful to include this step right at the beginning of the cleaning. Correcting encoding-related errors is simple: 2.2. Cleaning special characters, punctuation, and numbers This step should come right after fixing the encoding errors. The simplest way is to use the cleantext library. Also, consider lower-casing. Does "labor" mean the same as "Labor" in the dataset? In case it does, add a lowercase parameter and apply the cleaning function: 2.3. Define the stopwords removal strategy Removing the standard list of stopwords is generally a necessary step. Depending on the project focus, it might also be useful to clean data from an additional list of stopwords that don't bring any value. As written in the BERTopic's documentation:

Removing stop words as a preprocessing step is not advised as the transformer-based embedding models that we use need the full context to create accurate embeddings. Instead, we use CountVectorizer to preprocess our documents after having generated embeddings during the topic model generation.

3. Topic generation

Having a cleaner dataset, it is now possible to remove English stopwords along with a list of additional stopwords , generate a topic bigram model, and apply it to the data. Note that the nr_topicsparameter is set to 7 for generating 6 topics. The remaining topic is used to keep the outliers.

4. Topic visualization

In the next step, let's visualize the data in a heatmap to present the results better. Here is the outcome: Figure 1: Heatmaps with bigrams and their probabilities, Image by Author Figure 1: Heatmaps with bigrams and their probabilities, Image by Author To do so, we'll extract bigrams and their probabilities from the topic model and create a data frame for each of the 6 topics: Next, this code creates a heatmap in Figure 1.

5. Token frequencies over time

Now, we'll add a perspective on the development of bigrams over time. The goal is to look at which years the bigrams in the Russian leader(s) speeches were most frequently spoken out. The heatmap in Figure 2 displays the frequencies of the 5 most frequent bigrams for each year. Figure 2: Heatmaps with bigrams and their frequencies by year, Image by Author Figure 2: Heatmaps with bigrams and their frequencies by year, Image by Author The arabica library, which is now forthcoming in the Journal of Open Source Software (Koráb & Poměnková, 2024), was developed for this purpose. EDIT Jul 2024 : Arabica has been updated. Check the documentation for the full list of parameters. Here is the code generating the heatmap in Figure 2: Image by rawpixel on Freepik Image by rawpixel on Freepik

Conclusions

This article briefly introduced topic modeling with BERTopic. The model's framework offers many extensions, fine-tuning, and visualization methods (see the documentation). Let's summarize the key findings:

My previous article briefly explains a simpler approach to topic modeling with LDA. The complete code for this tutorial is on my GitHub. If you enjoy my work, you can invite me for coffee and support my writing. You can also subscribe to my email list to get notified about my new articles. Thanks!

References

[1] Blei, Ng, Jordan (2003). Latent Dirichlet Allocation. Journal Of Machine Learning Research 3, pp. 993–1022. [2] Chagnon, Pandolfi, Donatelli, Ushizima (2024). Benchmarking topic models on scientific articles using BERTeley. Natural Language Processing Journal 6. [3] Efat, Atiq, Abeed, Momin, Alam (2023). Empoliticon: NLP And MLBased Approach For Context And Emotion Classification Of Political Speeches From Transcripts. IEEE Access , vol. 11. [4] Grootendorst (2022). Bertopic: Neural Topic Modeling With A Class-Based TF-IDF Procedure. Computer Science. [5] Koráb, Poměnková (2024). Arabica: A Python package for exploratory analysis of text data. In The Journal of Open Source Software. Journal of Open Source Software. https://doi.org/10.5281/zenodo.10866697.

Register Now to Continue Reading

Unlock Towards Data Science with a Free Account

All articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more. Register for freeSign in to your account

Written By

Petr Koráb

See all from Petr Koráb BertopicData SciencePythonText MiningTopic Modeling Share This Article

Towards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program. Write for TDS

Related Articles

Can Reinforcement Learning Generalize Beyond Its Training?

Machine Learning A Case Study in Model Generalization John Morrow January 24, 20235 min read

Analyzing Twitter user network

Data Science Analyzing the Twitter user network using centrality and community detection algorithms. Mananai Saengsuwan June 23, 20223 min read

How Fast GPU Computation Can Be

Deep Learning A comparison of matrix arithmetic calculation in CPU and GPU with Python and PyTorch Andrew Zhu (Shudong Zhu) September 27, 20223 min read

A Better Way to Skill Up

Data Science Brainstorming How to Increase Customer ROI for the Education as a Service Industry Tony Yiu August 29, 201910 min read

Why a Data Science Mentor Can Help You in 2022

Data Science Lessons learned from two years with a data science mentor Rose Day December 10, 20216 min read

3 Rookie Mistakes To Avoid When Building Your Data Science Portfolio

Artificial Intelligence Here's what to do and what to avoid Benjamin Nweke July 30, 20217 min read

Machine Learning Made Easy by PyCaret

Artificial Intelligence Entire machine learning pipeline with 10 lines of code. Soner Yıldırım July 13, 20206 min read

Building a serverless, containerized machine learning model API using AWS Lambda & API Gateway and...

Machine Learning Dockerized Lambdas Sebastian Telsemeyer January 12, 20213 min read

Data Job is Not For Me - What to Do Next?

Data Science What to do when you realize that a Data job is not for you. Cornellius Wijaya January 9, 20214 min read

Your home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals. Subscribe to Our NewsletterWrite For TDS•About•Advertise•Privacy Policy•Terms of Use © Insight Media Group, LLC 2026•The New Stack•roadmap.sh•Creator Network

Recommended Articles

Close