保留正文与补采附件。源页面当前未重新核验；具体限制见交付清单。

Revisit consent button We value your privacy We use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking "Accept All", you consent to our use of cookies. Customise Reject All Accept All Customise Consent Preferences We use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below. The cookies that are categorised as "Necessary" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... Show more NecessaryAlways Active Necessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.

Functional Functional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.

Analytics Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.

Performance Performance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors.

Advertisement Advertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns.

Uncategorised Other uncategorised cookies are those that are being analysed and have not been classified into a category as yet. No cookies to display. Reject All Save My Preferences Accept All Toggle SearchToggle Mobile Navigation Publish AI, ML & data-science insights to a global community of data professionals. Sign inSubmit an Article LatestEditor's PicksNewsletterShipAI

Write For TDS LinkedInXToggle Search Machine Learning

Topic Modelling in Business Intelligence: FASTopic and BERTopic in Code

A comparison of two cutting-edge dynamic topic models solving consumer complaints classification exercise Petr Koráb January 22, 202511 min read Topic Modelling in Business Intelligence: FASTopic and BERTopic in Code Source: Freepic, Image by rawpixel.com Source: Freepic, Image by rawpixel.com Customer reviews about products and services provide valuable information about customer satisfaction. They provide insight into what should be improved across the whole product development. Dynamic topic models in business intelligence can identify key product qualities and other satisfaction factors, cluster them into categories, and evaluate how business decisions materialized in customer satisfaction over time. This is highly valuable information not only for product managers. This article will compare two of the latest topic models to classify customer complaints data. BERTopic by Maarten Grootendorst (2022) and the recent FASTopic by Xiaobao Wu et al. (2024) presented at last year's NeurIPS, are the current leading models for topic analytics of customer data. For these models, we'll explore in Python code:

Learn this step by step with the interactive Machine Learning roadmap.

1. Customer complaints data in companies

Complaints data are generated by interaction with customers and typically recorded in ERP systems. There are many channels where customers can raise a concern about a product or service. Here are just a few examples:

2. Example data

As example data, we'll use the Amazon Dog Food Reviews dataset from Hugging Face , released under the Apache-2.0 license. The subset for topic modeling only contains 3693 customer reviews collected over 02/01/2016: 31/12/2020. Here is what the data looks like: Image 1. Amazon dog food reviews dataset Image 1. Amazon dog food reviews dataset Image 2. General preprocessing steps for customer feedback topic modeling. Image by author Image 2. General preprocessing steps for (customer feedback) topic modeling. Image by author

3. Data preprocessing

Processing data systematically in the right order keeps the essential information and does not add a new bias. Let's go along these steps:

"Delivery" and "deliveries", "box" and "Boxes", or "Price" and "prices" share the same word root, but without lemmatization, topic models would model them as separate factors. That's why product reviews should always be lemmatized in the last step of preprocessing.

Text preprocessing is model-specific:

Source: Freepic, Image by macrovector Source: Freepic, Image by macrovector

4. Topic modeling with top-notch models

Let's now check how the satisfaction factors are distributed across the topics. The questions we ask here are:

The BERTopic and FASTopic papers describe the model architectures in detail. Also, my TDS tutorial on topic modeling explains topic classification with BERTopic on a political speech dataset.

4.1. FASTopic

Import the libraries and the data (complete code and the requirements are here). Then, create a list of clean reviews: python

import pandas as pd
from fastopic import FASTopic
from sklearn.feature_extraction.text import CountVectorizer 
from topmost.preprocessing import Preprocessing             

# create a list of reviews
docs = data['clean_text'].tolist()

In FASTopic, bigram generation is not directly implemented. To solve this, we will make a bigram preprocessing class. The model works with bigrams as with individual tokens, so we join the words in bigrams with underscores. python

# custom preprocessing class with bigram generation
class NgramPreprocessing:
    def __init__(self, ngram_range=(1, 1), 
                       vocab_size=10000, 
                       stopwords='English'): 

        self.ngram_range = ngram_range
        self.preprocessing = Preprocessing(vocab_size=vocab_size, 
                                           stopwords=stopwords)

        # use a custom analyzer to join bigrams with "_"
        self.vectorizer = CountVectorizer(ngram_range=self.ngram_range, 
                                          max_features=vocab_size, 
                                          analyzer=self._custom_analyzer)

        # custom analyzer function to join bigrams with underscores
    def _custom_analyzer(self, doc):
        # tokenize the document and create bigrams
        tokens = CountVectorizer(ngram_range=self.ngram_range).build_analyzer()(doc)

        # replace spaces in bigrams with "_"
        return [token.replace(" ", "_") for token in tokens]

    def preprocess(self, 
                   docs, 
                   pretrained_WE=False):

        parsed_docs = self.preprocessing.preprocess(docs, 
                      pretrained_WE=pretrained_WE)["train_texts"]
        train_bow = self.vectorizer.fit_transform(parsed_docs).toarray()
        rst = {
            "train_bow": train_bow,
            "train_texts": parsed_docs,
            "vocab": self.vectorizer.get_feature_names_out()
        }
        return rst

# initialize preprocessing with bigrams
ngram_preprocessing = NgramPreprocessing(ngram_range=(2, 2))

Let's train the model for eight topics and display the top 20 bigrams for each topic in a data frame. We train on single tokens, then remove the underscores generating the bigrams. python

# model training
model = FASTopic(8, ngram_preprocessing, num_top_words=10000)

# fit model to documents
topic_top_words, doc_topic_dist = model.fit_transform(docs)

# retrieve 20 bigrams for each topic
import pandas as pd

max_bigrams = 20

# Retrieve the bigrams for each topic and select only the word columns
topic_0 = pd.DataFrame(model.get_topic(0, max_bigrams), columns=["Topic_0_word", "Topic_0_prob"])[["Topic_0_word"]]
topic_1 = pd.DataFrame(model.get_topic(1, max_bigrams), columns=["Topic_1_word", "Topic_1_prob"])[["Topic_1_word"]]
topic_2 = pd.DataFrame(model.get_topic(2, max_bigrams), columns=["Topic_2_word", "Topic_2_prob"])[["Topic_2_word"]]
topic_3 = pd.DataFrame(model.get_topic(3, max_bigrams), columns=["Topic_3_word", "Topic_3_prob"])[["Topic_3_word"]]
topic_4 = pd.DataFrame(model.get_topic(4, max_bigrams), columns=["Topic_4_word", "Topic_4_prob"])[["Topic_4_word"]]
topic_5 = pd.DataFrame(model.get_topic(5, max_bigrams), columns=["Topic_5_word", "Topic_5_prob"])[["Topic_5_word"]]
topic_6 = pd.DataFrame(model.get_topic(6, max_bigrams), columns=["Topic_6_word", "Topic_6_prob"])[["Topic_6_word"]]
topic_7 = pd.DataFrame(model.get_topic(7, max_bigrams), columns=["Topic_7_word", "Topic_7_prob"])[["Topic_7_word"]]

# concatenate the dataframes
topics_df = pd.concat([topic_0,topic_1, topic_2, topic_3, topic_4,topic_5,topic_6,topic_7], axis=1)

# remove underscores from bigrams
topics_df = topics_df.applymap(lambda x: x.replace('_', ' ') if isinstance(x, str) else x)

We've modeled the customer satisfaction factors with a dog food product in eight distinct topics. Here are the manually annotated topic names: Image 3: Satisfaction factors modeling with FASTopic. Image by author Image 3: Satisfaction factors modeling with FASTopic. Image by author FASTopic returns relatively distinct topics, sorting the comments of the customers:

It is also useful to check the weight of these categories in the data. The full code is here.

We've modeled the customer satisfaction factors with a dog food product. But why is it beneficial for companies? Dynamic topic models offer a straightforward way of monitoring customer satisfaction over time. They indicate product-related problems and help take the right measures. Once the business decisions are taken into action, topic models check if they have an effect over time. To do so, let's model topic activity over time at a quarterly frequency. python

import plotly.graph_objects as go

# convert date column to datetime
data['time'] = pd.to_datetime(data['time'])

# format date column to quarterly periods
data['date_quarterly'] = data['time'].dt.to_period('Q').astype(str)

periods = data['date_quarterly'].tolist()

# calculate topic activity over time
act = model.topic_activity_over_time(periods)

# visualize topic activity
fig = model.visualize_topic_activity(top_n=8, topic_activity=act, time_slices=periods)

# update legend to display only the topic number
fig.data = sorted(fig.data, key=lambda trace: trace.name)

for trace in fig.data:
    trace.name = trace.name[0]

# update the layout
fig.update_layout(
    width=1200,
    height=600,
    title='',
    legend_title_text='Topic',
    xaxis_tickangle=45         # set x-axis labels to 45-degree angle
)

# show the figure
fig.show()

The delivery problems in topic 7 peaked in Q3 2018. Customers complained about open and rip boxes much more often, but these problems were fixed in early 2019 (see the picture below). Image 4: Topic activity over time, FASTopic. Image by author. Image 4: Topic activity over time, FASTopic. Image by author.

4.2. BERTopic

BERTopic implements bigrams with vectorizer_model, which also works as a data processing pipeline. The code and the requirements are here. python

from bertopic import BERTopic
from umap import UMAP
from sklearn.feature_extraction.text import CountVectorizer
from nltk.corpus import stopwords
import nltk
from nltk import word_tokenize          
from nltk.stem import WordNetLemmatizer
import pandas as pd
import re

nltk.download('stopwords')

# create a list of speeches
docs = data['text'].tolist()

We train on raw data and clean it with the vectorizer. During the training, the vectorizer cleans data from numbers and stopwords, returning lemmatized tokens for the bigram model. python

# create stopwords list
standard_stopwords = list(stopwords.words('english'))

# extended list of English stopwords
stopwords_extended = [ "0o",  ..]      

# additional tokens to remove
additional_stopwords = ['blue','buffalo','dog','food','ha','month','ago'] 

# combine standard, extended stopwords, and additional tokens
full_stopwords = standard_stopwords 
                 + additional_stopwords
                 + stopwords_extended

# define tokenizer retrurning lemmatized text without numbers
class LemmaTokenizer:
    def __init__(self):
        self.wnl = WordNetLemmatizer()
    def __call__(self, doc):
        doc = re.sub(r'd+', '', doc)  # clean numbers
        return [self.wnl.lemmatize(t) for t in word_tokenize(doc)] # lemmatize

# vectorizer makes data processing and generates bigrams
vectorizer_model = CountVectorizer(tokenizer=LemmaTokenizer(),
                                  ngram_range=(2, 2),
                                  stop_words=full_stopwords)

# set-up model
model = BERTopic(n_gram_range=(2,2), # returns bigrams
                nr_topics=9,         # generate 9 topics, leave -1 for outliers
                top_n_words=20,      # return top 20 bigrams
                min_topic_size=20,   # topics contains at least 20 tokens
                vectorizer_model=vectorizer_model,
                umap_model = UMAP(random_state=1))  # setting seed topics reproduce

# fit model to data
topics, probabilities = model.fit_transform(docs)

Next, let's prepare a dataframe with tokens from the model. python

import pandas as pd

# retrieve bigrams for each topic and select only the word columns
topic_0 = pd.DataFrame(model.get_topic(0), columns=["Topic_0_word", "Topic_0_prob"])[["Topic_0_word"]]
topic_1 = pd.DataFrame(model.get_topic(1), columns=["Topic_1_word", "Topic_1_prob"])[["Topic_1_word"]]
topic_2 = pd.DataFrame(model.get_topic(2), columns=["Topic_2_word", "Topic_2_prob"])[["Topic_2_word"]]
topic_3 = pd.DataFrame(model.get_topic(3), columns=["Topic_3_word", "Topic_3_prob"])[["Topic_3_word"]]
topic_4 = pd.DataFrame(model.get_topic(4), columns=["Topic_4_word", "Topic_4_prob"])[["Topic_4_word"]]
topic_5 = pd.DataFrame(model.get_topic(5), columns=["Topic_5_word", "Topic_5_prob"])[["Topic_5_word"]]
topic_6 = pd.DataFrame(model.get_topic(6), columns=["Topic_6_word", "Topic_6_prob"])[["Topic_6_word"]]
topic_7 = pd.DataFrame(model.get_topic(7), columns=["Topic_7_word", "Topic_7_prob"])[["Topic_7_word"]]

# concatenate the dataframes
topics_df = pd.concat([topic_0, topic_1, topic_2, topic_3, topic_4, 
                       topic_5, topic_6,topic_7], axis=1)

The annotated topics show similar categorization to FASTopic. The differences are categorizing Spanish tokens into a separate topic (T7) and filling T1 and T5 with adjectives of positive meaning. Delivery problems in T4 are identical to FASTopic's classification. Image 5: Satisfaction factors modeling with BERTopic. Image by author Image 5: Satisfaction factors modeling with BERTopic. Image by author Again, let's focus on topic activity over time, which gives dynamic topic models additional value for BI. BERTopic uses token frequencies (not topic weights as FASTopic) for topic activity analysis. python

# topic activity over time
import plotly.graph_objects as go

# create timestamps
data['time'] = pd.to_datetime(data['time'])
timestamps = data['time'].to_list()

# generate topics over time, 20 bins correspond to Q frequency
topics_over_time = model.topics_over_time(docs, timestamps, nr_bins=20)

# filter out topic -1 containing outliers
topics_over_time_filtered = topics_over_time[topics_over_time['Topic'] != -1]

# visualize the filtered topics over time
fig = model.visualize_topics_over_time(topics_over_time_filtered)

# update legend to display only the topic number
fig.data = sorted(fig.data, key=lambda trace: trace.name)

for trace in fig.data:
    trace.name = trace.name[0]

# update the layout
fig.update_layout(
    width=1200,
    height=600,
    title='',
    legend_title_text='Topic',
    xaxis_tickangle=45           # set x-axis labels to 45-degree angle
)

# show the figure
fig.show()

Most topics are stable over time, except T4, which categorizes delivery problems. As with FASTopic, BERTopic shows that customers' negative complaints about damaged boxes rose in mid-2018. Image 6: Topic activity over time, BERTopic. Image by author. Image 6: Topic activity over time, BERTopic. Image by author.

Summary

Both models indicated delivery problems in mid-2018, which vanished in early 2019. With a topic model API monitoring customer comments on various channels, these problems can be fixed before they have a harmful effect on the brand. The right data processing is essential for topic models to make sense in the applied world. Cleaning text in the right order minimizes the bias of each cleaning operation. Numbers and emoticons are typically removed first, followed by stopwords. Punctuation is cleaned afterward so that stopwords don't break up into two tokens ("we've" -> "we" + 've"). Additional tokens are removed in the next step in the clean data before lemmatization, which unifies tokens with the same semantics. FASTopic deserves much better documentation, which now provides only basic information. Especially because its (1) simplicity of use and (2) stability in training on small datasets makes it a top-notch alternative to BERTopic. It is mainly practical for small companies like e-shops that typically don't collect large text datasets and seek simple and efficient solutions. Data and full codes for this tutorial here. If you enjoy my work, you can invite me for coffee and support my writing. You can also subscribe to my email list to get notified about my new articles. Thanks!

References

[1] Grootendorst (2022). Bertopic: Neural Topic Modeling With A Class-Based TF-IDF Procedure. Computer Science. [2] Wu, X, Nguyen, T., Ce Zhang, D., Yang Wang, W., Luu, A. T. (2024). FASTopic: A Fast, Adaptive, Stable, and Transferable Topic Modeling Paradigm. arXiv preprint: 2405.17978.

Register Now to Continue Reading

Unlock Towards Data Science with a Free Account

All articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more. Register for freeSign in to your account

Written By

Petr Koráb

See all from Petr Koráb BertopicMachine LearningNLPText MiningTopic Modelling Share This Article

Towards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program. Write for TDS

Related Articles

Dive Into LoRA Adapters

Deep Learning Exploring Parameter Efficient Finetuning (PEFT): Intuitively Understanding Finetuning Using LoRA Mariano Kamp August 25, 202315 min read

The Messages We Send: The Case Against "Deploying" Machine Learning

Machine Learning Why Changing Our Language Enables More Responsible AI Ayodele Odubela June 8, 20214 min read

Regularization. What, Why, When, and How?

Deep Learning In this article, I want to take an in-depth look at regularization. Akash Shastri October 24, 20205 min read

Machine Learning with Google's BigQuery

Machine Learning How to easily create and deploy ML Models with SQL Christianlauer June 23, 20214 min read

PyTorch and MLX for Apple Silicon

Machine Learning A side-by-side CNN implementation and comparison Mike Cvet March 8, 20249 min read

Using Complex Networks to improve Machine Learning methods

Machine Learning Using the High-Level Data Classification Algorithm to leverage topological characteristics of the data Tiago Toledo Jr. October 26, 20216 min read

4 Useful clustering methods you should know in 2021

Machine Learning Form groups of similar observations based on distance Rukshan Pramoditha March 30, 20217 min read

What is Stationarity in Time Series and why should you care

Artificial Intelligence Time Series are Everywhere - Make sure you know how to Handle them Dario Radečić January 11, 20206 min read

The Machine Learning “Advent Calendar” Day 13: LASSO and Ridge Regression in Excel

Artificial Intelligence Why regularization is not about changing the model, but about choosing better coefficients Angela and Kezhan Shi December 13, 20259 min read

Your home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals. Subscribe to Our NewsletterWrite For TDS•About•Advertise•Privacy Policy•Terms of Use © Insight Media Group, LLC 2026•The New Stack•roadmap.sh•Creator Network

Recommended Articles

Close