![Revisit consent button](../images/body-df29beacd7496220cc94.svg) We value your privacy We use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking "Accept All", you consent to our use of cookies. Customise Reject All Accept All Customise Consent Preferences ![](../images/body-2159bb67f3de8ad870eb.svg) We use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below. The cookies that are categorised as "Necessary" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... Show more NecessaryAlways Active Necessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data. * Cookie BCTempID * Duration 10 minutes * Description No description available. * Cookie __cf_bm * Duration 1 hour * Description This cookie, set by Cloudflare, is used to support Cloudflare Bot Management. * Cookie AWSALBCORS * Duration 7 days * Description Amazon Web Services set this cookie for load balancing. * Cookie _cfuvid * Duration session * Description Cloudflare sets this cookie to track users across sessions to optimize user experience by maintaining session consistency and providing personalized services * Cookie li_gc * Duration 6 months * Description Linkedin set this cookie for storing visitor's consent regarding using cookies for non-essential purposes. * Cookie __hssrc * Duration session * Description This cookie is set by Hubspot whenever it changes the session cookie. The __hssrc cookie set to 1 indicates that the user has restarted the browser, and if the cookie does not exist, it is assumed to be a new session. * Cookie __hssc * Duration 1 hour * Description HubSpot sets this cookie to keep track of sessions and to determine if HubSpot should increment the session number and timestamps in the __hstc cookie. * Cookie wpEmojiSettingsSupports * Duration session * Description WordPress sets this cookie when a user interacts with emojis on a WordPress site. It helps determine if the user's browser can display emojis properly. * Cookie BCSessionID * Duration 1 year 1 month 4 days * Description Blueconic sets this cookie as a unique identifier for the BlueConic profile. * Cookie _octo * Duration 1 year * Description No description available. * Cookie logged_in * Duration 1 year * Description No description available. * Cookie __Secure-YEC * Duration past * Description YouTube sets this cookie to stores the user's video player preferences using embedded YouTube video * Cookie __eoi * Duration 6 months * Description Description is currently not available. * Cookie AWSALBTGCORS * Duration 7 days * Description No description available. * Cookie login-status-p * Duration past * Description Description is currently not available. * Cookie AWSALBTG * Duration 7 days * Description No description available. * Cookie csrf_token * Duration session * Description No description available. * Cookie token_v2 * Duration 1 day * Description Description is currently not available. * Cookie D * Duration 1 year * Description Description is currently not available. * Cookie PHPSESSID * Duration session * Description This cookie is native to PHP applications. The cookie stores and identifies a user's unique session ID to manage user sessions on the website. The cookie is a session cookie and will be deleted when all the browser windows are closed. * Cookie VISITOR_PRIVACY_METADATA * Duration 6 months * Description YouTube sets this cookie to store the user's cookie consent state for the current domain. * Cookie cookietest * Duration session * Description The cookietest cookie is typically used to determine whether the user's browser accepts cookies, essential for website functionality and user experience. * Cookie __Host-airtable-session * Duration 1 year * Description This cookie is used to enable us to integrate the services of Airtable. * Cookie __Host-airtable-session.sig * Duration 1 year * Description This cookie is used to enable us to integrate the services of Airtable. * Cookie m * Duration 1 year 1 month 4 days * Description Stripe sets this cookie for fraud prevention purposes. It identifies the device used to access the website, allowing the website to be formatted accordingly. * Cookie BIGipServer* * Duration session * Description Marketo sets this cookie to collect information about the user's online activity and build a profile about their interests to provide advertisements relevant to the user. * Cookie __cfruid * Duration session * Description Cloudflare sets this cookie to identify trusted web traffic. * Cookie _GRECAPTCHA * Duration 6 months * Description Google Recaptcha service sets this cookie to identify bots to protect the website against malicious spam attacks. * Cookie __Secure-YNID * Duration 6 months * Description Google cookie used to protect user security and prevent fraud, especially during the login process. * Cookie cookieyes-consent * Duration 1 year * Description CookieYes sets this cookie to remember users' consent preferences so that their preferences are respected on subsequent visits to this site. It does not collect or store any personal information about the site visitors. Functional Functional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features. * Cookie lidc * Duration 1 day * Description LinkedIn sets the lidc cookie to facilitate data center selection. * Cookie brw * Duration 1 year * Description No description available. * Cookie brwConsent * Duration 5 minutes * Description Description is currently not available. * Cookie WMF-Uniq * Duration 1 year * Description Description is currently not available. * Cookie loom_anon_comment * Duration 1 year * Description No description available. * Cookie loom_referral_video * Duration session * Description Description is currently not available. * Cookie VISITOR_INFO1_LIVE * Duration 6 months * Description A cookie set by YouTube to measure bandwidth that determines whether the user gets the new or old player interface. * Cookie yt-remote-connected-devices * Duration Never Expires * Description YouTube sets this cookie to store the user's video preferences using embedded YouTube videos. * Cookie ytidb::LAST_RESULT_ENTRY_KEY * Duration Never Expires * Description The cookie ytidb::LAST_RESULT_ENTRY_KEY is used by YouTube to store the last search result entry that was clicked by the user. This information is used to improve the user experience by providing more relevant search results in the future. * Cookie yt-remote-device-id * Duration Never Expires * Description YouTube sets this cookie to store the user's video preferences using embedded YouTube videos. * Cookie yt-remote-session-name * Duration session * Description The yt-remote-session-name cookie is used by YouTube to store the user's video player preferences using embedded YouTube video. * Cookie yt-remote-fast-check-period * Duration session * Description The yt-remote-fast-check-period cookie is used by YouTube to store the user's video player preferences for embedded YouTube videos. * Cookie yt-remote-session-app * Duration session * Description The yt-remote-session-app cookie is used by YouTube to store user preferences and information about the interface of the embedded YouTube video player. * Cookie yt-remote-cast-available * Duration session * Description The yt-remote-cast-available cookie is used to store the user's preferences regarding whether casting is available on their YouTube video player. * Cookie yt-remote-cast-installed * Duration session * Description The yt-remote-cast-installed cookie is used to store the user's video player preferences using embedded YouTube video. * Cookie cp_session * Duration 3 months * Description Codepen sets this cookie for Help systems found in the website. * Cookie loid * Duration 1 year 1 month 4 days * Description This cookie is set by the Reddit. The cookie enables the sharing of content from the website onto the social media platform. Analytics Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc. * Cookie __hstc * Duration 6 months * Description Hubspot set this main cookie for tracking visitors. It contains the domain, initial timestamp (first visit), last timestamp (last visit), current timestamp (this visit), and session number (increments for each subsequent session). * Cookie hubspotutk * Duration 6 months * Description HubSpot sets this cookie to keep track of the visitors to the website. This cookie is passed to HubSpot on form submission and used when deduplicating contacts. * Cookie _ga * Duration 1 year 1 month 4 days * Description Google Analytics sets this cookie to calculate visitor, session and campaign data and track site usage for the site's analytics report. The cookie stores information anonymously and assigns a randomly generated number to recognise unique visitors. * Cookie _ga_* * Duration 1 year 1 month 4 days * Description Google Analytics sets this cookie to store and count page views. * Cookie __Host-psifi.analyticsTrace * Duration 6 hours * Description Description is currently not available. * Cookie __Host-psifi.analyticsTraceV2 * Duration 6 hours * Description Description is currently not available. * Cookie _gh_sess * Duration session * Description GitHub sets this cookie for temporary application and framework state between pages like what step the user is on in a multiple step form. * Cookie YSC * Duration session * Description YSC cookie is set by Youtube and is used to track the views of embedded videos on Youtube pages. * Cookie ajs_anonymous_id * Duration 1 year * Description This cookie is set by Segment to count the number of people who visit a certain site by tracking if they have visited before. * Cookie vuid * Duration 1 year 1 month 4 days * Description Vimeo installs this cookie to collect tracking information by setting a unique ID to embed videos on the website. Performance Performance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors. * Cookie AWSALB * Duration 7 days * Description AWSALB is an application load balancer cookie set by Amazon Web Services to map the session to the target. * Cookie acq * Duration past * Description Description is currently not available. * Cookie acq.sig * Duration past * Description Description is currently not available. * Cookie ptc * Duration 2 years * Description No description available. Advertisement Advertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns. * Cookie muc_ads * Duration 1 year 1 month 4 days * Description Twitter sets this cookie to collect user behaviour and interaction data to optimize the website. * Cookie guest_id_marketing * Duration 1 year 1 month 4 days * Description Twitter sets this cookie to identify and track the website visitor. * Cookie guest_id_ads * Duration 1 year 1 month 4 days * Description Twitter sets this cookie to identify and track the website visitor. * Cookie personalization_id * Duration 1 year 1 month 4 days * Description Twitter sets this cookie to integrate and share features for social media and also store information about how the user uses the website, for tracking and targeting. * Cookie guest_id * Duration 1 year 1 month 4 days * Description Twitter sets this cookie to identify and track the website visitor. It registers if a user is signed in to the Twitter platform and collects information about ad preferences. * Cookie bcookie * Duration 1 year * Description LinkedIn sets this cookie from LinkedIn share buttons and ad tags to recognize browser IDs. * Cookie __Secure-ROLLOUT_TOKEN * Duration 6 months * Description YouTube sets this cookie to manage feature rollout and experimentation. It helps Google control which new features or interface changes are shown to users as part of testing and staged rollouts, ensuring consistent experience for a given user during an experiment. * Cookie yt.innertube::nextId * Duration Never Expires * Description YouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen. * Cookie yt.innertube::requests * Duration Never Expires * Description YouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen. * Cookie session_tracker * Duration session * Description This cookie is set by the Reddit. This cookie is used to identify trusted web traffic. It also helps in adverstising on the website. * Cookie edgebucket * Duration session * Description Reddit sets this cookie to save the information about a log-on Reddit user, for the purpose of advertisement recommendations and updating the content. * Cookie did * Duration 1 year * Description Arbor sets this cookie to show targeted ads to site visitors.This cookie expires after 2 months or 1 year. Uncategorised Other uncategorised cookies are those that are being analysed and have not been classified into a category as yet. No cookies to display. Reject All Save My Preferences Accept All Toggle Search[](https://towardsdatascience.com/)Toggle Mobile Navigation [](https://towardsdatascience.com/) Publish AI, ML & data-science insights to a global community of data professionals. Sign in[Submit an Article](https://write.insightmediagroup.io) [Latest](https://towardsdatascience.com/latest/)[Editor's Picks](https://towardsdatascience.com/tag/editors-pick/)[Newsletter](https://towardsdatascience.com/tag/the-variable/)[ShipAI](https://towardsdatascience.com/shipai/) * * * [Write For TDS](https://towardsdatascience.com/submissions/) [LinkedIn](https://www.linkedin.com/company/towards-data-science/)[X](https://x.com/TDataScience)Toggle Search [Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/) # Topic Modelling in Business Intelligence: FASTopic and BERTopic in Code A comparison of two cutting-edge dynamic topic models solving consumer complaints classification exercise [Petr Koráb](https://towardsdatascience.com/author/petrkorab/) January 22, 202511 min read Topic Modelling in Business Intelligence: FASTopic and BERTopic in Code ![Source: Freepic, Image by rawpixel.com](../images/000.jpeg) Source: [Freepic](https://www.freepik.com/free-vector/illustration-network-shareing_2808109.htm#fromView=search&page=2&position=3&uuid=c11e1dc6-4be2-45d5-9c7b-29bc16bc69f0), Image by rawpixel.com **Customer reviews** about products and services provide valuable information about customer satisfaction. They provide insight into what should be improved across the whole product development. Dynamic topic models in business intelligence can identify key product qualities and other satisfaction factors, cluster them into categories, and evaluate how business decisions materialized in customer satisfaction over time. This is highly valuable information not only for product managers. This article will compare two of the latest topic models to classify customer complaints data. **BERTopic** by [Maarten Grootendorst (2022)](https://arxiv.org/pdf/2203.05794) and the recent [**FASTopic**](https://pypi.org/project/fastopic/) by [Xiaobao Wu et al. (2024)](https://arxiv.org/pdf/2405.17978) presented at last year's [NeurIPS](https://neurips.cc/virtual/2024/poster/96416), are the current leading models for topic analytics of customer data. For these models, we'll explore in Python code: * how to effectively **preprocess data** * how to train a **Bigram topic model** for customer complaint analysis * how to model **topic activity** over time. Learn this step by step with the interactive [Machine Learning](https://roadmap.sh/machine-learning) roadmap. # 1. Customer complaints data in companies Complaints data are generated by interaction with customers and typically recorded in [ERP systems](https://www.sap.com/products/erp/what-is-erp.html). There are many channels where customers can raise a concern about a product or service. Here are just a few examples: * **Email** : email communication is stored for the BI team, e.g., in the SQL database. * **After-purchase** **survey** : feedback sent to customers after product purchase. Companies either send the emails themselves or use a price comparison website (e.g., [Billiger ](https://www.billiger-mietwagen.de/)in Germany) where customers order the product. * **Phone transcriptions** : after prior consent from a customer, some companies record the phone communication with customers, which is then available for the BI team. * **Google reviews:** customers leave comments and reviews on products and services worldwide. Google enables authorized users to [export the data ](https://takeout.google.com/?pli=1)not only for text mining purposes. * **Review platforms:** independent review platforms **** (such as [Trustpilot](https://www.trustpilot.com/)) offer customers a place to provide feedback to brands and companies. This data is available through [various APIs.](https://developers.trustpilot.com/service-reviews-api) * **Social media** **conversations** : Instagram, X, and Facebook are full of product or brand-related comments. The simplest way is to use an official API to collect the data. For Instagram and Facebook, go to the [developers' portal](https://developers.facebook.com/) to receive an API key. X works the [same way.](https://developer.x.com/en/docs/x-api) ## 2. Example data As example data, we'll use the [**Amazon Dog Food Reviews**](https://huggingface.co/datasets/NiyatiC/amazon_food_reviews) dataset from **Hugging Face** , released under the [Apache-2.0 license](https://github.com/huggingface/datasets/blob/main/LICENSE). The subset for topic modeling only contains 3693 customer reviews collected over 02/01/2016: 31/12/2020. Here is what the data looks like: ![Image 1. Amazon dog food reviews dataset](../images/001.png) Image 1. Amazon dog food reviews dataset ![Image 2. General preprocessing steps for \(customer feedback\) topic modeling. Image by author](../images/002.png) Image 2. General preprocessing steps for (customer feedback) topic modeling. Image by author ## 3. Data preprocessing Processing data systematically in the right order keeps the essential information and does not add a new bias. Let's go along these steps: * _**#1: Numbers:**_ digits are typically the characters to remove in the first step. * _**#2: Emoticons:**_ product reviews are typically full of them. For topic modeling in customer reviews, emojis don't have much significance. * _**#3: Stopwords:**_ apart from [standard stopwords](https://www.geeksforgeeks.org/removing-stop-words-nltk-python/), it is common to remove an [extended ](https://github.com/PetrKorab/Arabica/blob/main/stopwords_extended.py)stopwords list for one or more languages. * _**#4: Punctuation:**_ general language has a myriad of special characters and punctuation, which should be cleaned in this step. * _**#5: Additional stopwords:**_ depending on the use case, some additional words are also useful to remove. With the Amazon dog food reviews, these are _"dog", "food", "blue", "buffalo", "ha", "month"_ , and _"ago"._ > "Delivery" and "deliveries", "box" and "Boxes", or "Price" and "prices" share the same word root, but without lemmatization, topic models would model them as separate factors. That's why product reviews should always be lemmatized in the last step of preprocessing. * _**#6: Lemmatization:**_ groups words into a single form (the lemma), keeping the word root information and semantics. Text preprocessing is model-specific: * _**FASTopic**_ works with clean data on input; some cleaning (stopwords) can be done during the training. The simplest and most effective way is to use the [_**Washer: The no-code app for text data cleaning**_](https://washer.textminingstories.com/) offering a no-code way of processing data for text mining projects. * _**BERTopic:**_ the[ documentation](https://maartengr.github.io/BERTopic/faq.html#how-do-i-reduce-topic-outliers) recommends that " r _emoving stop words as a preprocessing step is not advised as the transformer-based embedding models that we use need the full context to create accurate embeddings"._ It uses transformers based on real text, not clean text without stopwords, lemmas, or tokens. For this reason, cleaning operations should be included in the model training. ![Source: Freepic, Image by macrovector](../images/003.jpeg) Source: [Freepic](https://www.freepik.com/free-vector/romantic-train-trip-background_5972223.htm#fromView=search&page=1&position=24&uuid=1b2ccb61-1b56-453e-b67a-befe1eba6858), Image by macrovector ## 4. Topic modeling with top-notch models Let's now check how the satisfaction factors are distributed across the topics. The questions we ask here are: * _What were the key problems and qualities customers reported on the product?_ * _How has product satisfaction changed over time?_ The [BERTopi](https://arxiv.org/pdf/2203.05794)c and [FASTopic](https://arxiv.org/pdf/2405.17978) papers describe the model architectures in detail. Also, my [TDS tutorial](https://towardsdatascience.com/topic-modelling-with-berttopic-in-python-8a80d529de34) on topic modeling explains topic classification with BERTopic on a political speech dataset. ### 4.1. FASTopic Import the libraries and the data (complete code and the requirements are [here](https://github.com/PetrKorab/Topic-Modelling-in-Business-Intelligence-BERTopic-and-FASTopic-in-Code/tree/main)). Then, create a list of clean reviews: python ``` import pandas as pd from fastopic import FASTopic from sklearn.feature_extraction.text import CountVectorizer from topmost.preprocessing import Preprocessing # create a list of reviews docs = data['clean_text'].tolist() ``` In FASTopic, bigram generation is not directly implemented. To solve this, we will make a bigram preprocessing class. The model works with bigrams as with individual tokens, so we join the words in bigrams with underscores. python ``` # custom preprocessing class with bigram generation class NgramPreprocessing: def __init__(self, ngram_range=(1, 1), vocab_size=10000, stopwords='English'): self.ngram_range = ngram_range self.preprocessing = Preprocessing(vocab_size=vocab_size, stopwords=stopwords) # use a custom analyzer to join bigrams with "_" self.vectorizer = CountVectorizer(ngram_range=self.ngram_range, max_features=vocab_size, analyzer=self._custom_analyzer) # custom analyzer function to join bigrams with underscores def _custom_analyzer(self, doc): # tokenize the document and create bigrams tokens = CountVectorizer(ngram_range=self.ngram_range).build_analyzer()(doc) # replace spaces in bigrams with "_" return [token.replace(" ", "_") for token in tokens] def preprocess(self, docs, pretrained_WE=False): parsed_docs = self.preprocessing.preprocess(docs, pretrained_WE=pretrained_WE)["train_texts"] train_bow = self.vectorizer.fit_transform(parsed_docs).toarray() rst = { "train_bow": train_bow, "train_texts": parsed_docs, "vocab": self.vectorizer.get_feature_names_out() } return rst # initialize preprocessing with bigrams ngram_preprocessing = NgramPreprocessing(ngram_range=(2, 2)) ``` Let's train the model for eight topics and display the top 20 bigrams for each topic in a data frame. We train on single tokens, then remove the underscores generating the bigrams. python ``` # model training model = FASTopic(8, ngram_preprocessing, num_top_words=10000) # fit model to documents topic_top_words, doc_topic_dist = model.fit_transform(docs) # retrieve 20 bigrams for each topic import pandas as pd max_bigrams = 20 # Retrieve the bigrams for each topic and select only the word columns topic_0 = pd.DataFrame(model.get_topic(0, max_bigrams), columns=["Topic_0_word", "Topic_0_prob"])[["Topic_0_word"]] topic_1 = pd.DataFrame(model.get_topic(1, max_bigrams), columns=["Topic_1_word", "Topic_1_prob"])[["Topic_1_word"]] topic_2 = pd.DataFrame(model.get_topic(2, max_bigrams), columns=["Topic_2_word", "Topic_2_prob"])[["Topic_2_word"]] topic_3 = pd.DataFrame(model.get_topic(3, max_bigrams), columns=["Topic_3_word", "Topic_3_prob"])[["Topic_3_word"]] topic_4 = pd.DataFrame(model.get_topic(4, max_bigrams), columns=["Topic_4_word", "Topic_4_prob"])[["Topic_4_word"]] topic_5 = pd.DataFrame(model.get_topic(5, max_bigrams), columns=["Topic_5_word", "Topic_5_prob"])[["Topic_5_word"]] topic_6 = pd.DataFrame(model.get_topic(6, max_bigrams), columns=["Topic_6_word", "Topic_6_prob"])[["Topic_6_word"]] topic_7 = pd.DataFrame(model.get_topic(7, max_bigrams), columns=["Topic_7_word", "Topic_7_prob"])[["Topic_7_word"]] # concatenate the dataframes topics_df = pd.concat([topic_0,topic_1, topic_2, topic_3, topic_4,topic_5,topic_6,topic_7], axis=1) # remove underscores from bigrams topics_df = topics_df.applymap(lambda x: x.replace('_', ' ') if isinstance(x, str) else x) ``` We've modeled the customer satisfaction factors with a dog food product in eight distinct topics. Here are the manually annotated topic names: ![Image 3: Satisfaction factors modeling with FASTopic. Image by author](../images/004.png) Image 3: Satisfaction factors modeling with FASTopic. Image by author FASTopic returns relatively distinct topics, sorting the comments of the customers: * **0: Negative health effects,** _"sensitive stomach", "small bite", "stomach issue", "lose weight", "refuse eat", "taste wild", "digestive issue", "upset stomach", "stop eat", "gain weight"_ * **1: Food quality,** _"love flavor", "quality ingredient", "good ingredient", "healthy ingredient", "ingredient quality", "flavor good", "taste great", "healthy love", "great healthy", "good healthy", "good health", ..._ * **2: Positive health effects,** _"healthy fur", "awesome pup", "eye bright"_ * **3: Digestion effects,** _"smell bad", "runny poop", "horrible gas", "diarrhea vet", "terrible diarrhea," "sick week", "sick buy", "day vomit"_ * **4: Pricing,** _"great price", "good price", "love price", "price great", "love cheap", "price deliver", "great deal", "price increase", "free shipping", ..._ * **5: Other,** other factors. * **6: Fur effects,** _"coat shiny", "fur baby", "skin issue", "shiny coat", "love coat", "coat soft"_ * **7: Delivery,** _"open box", "bag rip", "big bag", "hole bag", "open bag", "inside box", "bag open", "bag hole", "heavy bag", "rip open", ..._ It is also useful to check the weight of these categories in the data. The full code is [here](https://github.com/PetrKorab/Topic-Modelling-in-Business-Intelligence-BERTopic-and-FASTopic-in-Code/blob/main/FASTopic.ipynb). > We've modeled the customer satisfaction factors with a dog food product. But why is it beneficial for companies? Dynamic topic models offer a straightforward way of monitoring customer satisfaction over time. They indicate product-related problems and help take the right measures. Once the business decisions are taken into action, topic models check if they have an effect over time. To do so, let's model topic activity over time at a quarterly frequency. python ``` import plotly.graph_objects as go # convert date column to datetime data['time'] = pd.to_datetime(data['time']) # format date column to quarterly periods data['date_quarterly'] = data['time'].dt.to_period('Q').astype(str) periods = data['date_quarterly'].tolist() # calculate topic activity over time act = model.topic_activity_over_time(periods) # visualize topic activity fig = model.visualize_topic_activity(top_n=8, topic_activity=act, time_slices=periods) # update legend to display only the topic number fig.data = sorted(fig.data, key=lambda trace: trace.name) for trace in fig.data: trace.name = trace.name[0] # update the layout fig.update_layout( width=1200, height=600, title='', legend_title_text='Topic', xaxis_tickangle=45 # set x-axis labels to 45-degree angle ) # show the figure fig.show() ``` The delivery problems in topic 7 peaked in Q3 2018. Customers complained about open and rip boxes much more often, but these problems were fixed in early 2019 (see the picture below). ![Image 4: Topic activity over time, FASTopic. Image by author.](../images/005.png) Image 4: Topic activity over time, FASTopic. Image by author. ### 4.2. BERTopic BERTopic implements bigrams with _[vectorizer_model](https://scikit-learn.org/1.5/modules/generated/sklearn.feature_extraction.text.CountVectorizer.html),_ which also works as a data processing pipeline. The code and the requirements are [here](https://github.com/PetrKorab/Topic-Modelling-in-Business-Intelligence-BERTopic-and-FASTopic-in-Code/tree/main). python ``` from bertopic import BERTopic from umap import UMAP from sklearn.feature_extraction.text import CountVectorizer from nltk.corpus import stopwords import nltk from nltk import word_tokenize from nltk.stem import WordNetLemmatizer import pandas as pd import re nltk.download('stopwords') # create a list of speeches docs = data['text'].tolist() ``` We train on raw data and clean it with the vectorizer. During the training, the vectorizer cleans data from numbers and stopwords, returning lemmatized tokens for the bigram model. python ``` # create stopwords list standard_stopwords = list(stopwords.words('english')) # extended list of English stopwords stopwords_extended = [ "0o", ..] # additional tokens to remove additional_stopwords = ['blue','buffalo','dog','food','ha','month','ago'] # combine standard, extended stopwords, and additional tokens full_stopwords = standard_stopwords + additional_stopwords + stopwords_extended # define tokenizer retrurning lemmatized text without numbers class LemmaTokenizer: def __init__(self): self.wnl = WordNetLemmatizer() def __call__(self, doc): doc = re.sub(r'd+', '', doc) # clean numbers return [self.wnl.lemmatize(t) for t in word_tokenize(doc)] # lemmatize # vectorizer makes data processing and generates bigrams vectorizer_model = CountVectorizer(tokenizer=LemmaTokenizer(), ngram_range=(2, 2), stop_words=full_stopwords) # set-up model model = BERTopic(n_gram_range=(2,2), # returns bigrams nr_topics=9, # generate 9 topics, leave -1 for outliers top_n_words=20, # return top 20 bigrams min_topic_size=20, # topics contains at least 20 tokens vectorizer_model=vectorizer_model, umap_model = UMAP(random_state=1)) # setting seed topics reproduce # fit model to data topics, probabilities = model.fit_transform(docs) ``` Next, let's prepare a dataframe with tokens from the model. python ``` import pandas as pd # retrieve bigrams for each topic and select only the word columns topic_0 = pd.DataFrame(model.get_topic(0), columns=["Topic_0_word", "Topic_0_prob"])[["Topic_0_word"]] topic_1 = pd.DataFrame(model.get_topic(1), columns=["Topic_1_word", "Topic_1_prob"])[["Topic_1_word"]] topic_2 = pd.DataFrame(model.get_topic(2), columns=["Topic_2_word", "Topic_2_prob"])[["Topic_2_word"]] topic_3 = pd.DataFrame(model.get_topic(3), columns=["Topic_3_word", "Topic_3_prob"])[["Topic_3_word"]] topic_4 = pd.DataFrame(model.get_topic(4), columns=["Topic_4_word", "Topic_4_prob"])[["Topic_4_word"]] topic_5 = pd.DataFrame(model.get_topic(5), columns=["Topic_5_word", "Topic_5_prob"])[["Topic_5_word"]] topic_6 = pd.DataFrame(model.get_topic(6), columns=["Topic_6_word", "Topic_6_prob"])[["Topic_6_word"]] topic_7 = pd.DataFrame(model.get_topic(7), columns=["Topic_7_word", "Topic_7_prob"])[["Topic_7_word"]] # concatenate the dataframes topics_df = pd.concat([topic_0, topic_1, topic_2, topic_3, topic_4, topic_5, topic_6,topic_7], axis=1) ``` The annotated topics show similar categorization to FASTopic. The differences are categorizing Spanish tokens into a separate topic (T7) and filling T1 and T5 with adjectives of positive meaning. Delivery problems in T4 are identical to FASTopic's classification. ![Image 5: Satisfaction factors modeling with BERTopic. Image by author](../images/006.png) Image 5: Satisfaction factors modeling with BERTopic. Image by author Again, let's focus on topic activity over time, which gives dynamic topic models additional value for BI. BERTopic uses token frequencies (not topic weights as FASTopic) for topic activity analysis. python ``` # topic activity over time import plotly.graph_objects as go # create timestamps data['time'] = pd.to_datetime(data['time']) timestamps = data['time'].to_list() # generate topics over time, 20 bins correspond to Q frequency topics_over_time = model.topics_over_time(docs, timestamps, nr_bins=20) # filter out topic -1 containing outliers topics_over_time_filtered = topics_over_time[topics_over_time['Topic'] != -1] # visualize the filtered topics over time fig = model.visualize_topics_over_time(topics_over_time_filtered) # update legend to display only the topic number fig.data = sorted(fig.data, key=lambda trace: trace.name) for trace in fig.data: trace.name = trace.name[0] # update the layout fig.update_layout( width=1200, height=600, title='', legend_title_text='Topic', xaxis_tickangle=45 # set x-axis labels to 45-degree angle ) # show the figure fig.show() ``` Most topics are stable over time, except T4, which categorizes delivery problems. As with FASTopic, BERTopic shows that customers' negative complaints about damaged boxes rose in mid-2018. ![Image 6: Topic activity over time, BERTopic. Image by author.](../images/007.png) Image 6: Topic activity over time, BERTopic. Image by author. ## Summary Both models indicated delivery problems in mid-2018, which vanished in early 2019. With a topic model API monitoring customer comments on various channels, these problems can be fixed before they have a harmful effect on the brand. The right **data processing is essential** for topic models to make sense in the applied world. Cleaning text in the right order minimizes the bias of each cleaning operation. Numbers and emoticons are typically removed first, followed by stopwords. Punctuation is cleaned afterward so that stopwords don't break up into two tokens ("we've" -> "we" + 've"). Additional tokens are removed in the next step in the clean data before lemmatization, which unifies tokens with the same semantics. **FASTopic** deserves much better [documentation](https://github.com/BobXWu/FASTopic), which now provides only basic information. Especially because its (1) simplicity of use and (2) stability in training on small datasets makes it a top-notch alternative to BERTopic. It is mainly practical for small companies like e-shops that typically don't collect large text datasets and seek simple and efficient solutions. Data and full codes for this tutorial [here](https://github.com/PetrKorab/Topic-Modelling-in-Business-Intelligence-BERTopic-and-FASTopic-in-Code/tree/main). _If you enjoy my work, you can invite me_[ _for coffee_](https://www.buymeacoffee.com/petrkorab) _and support my writing. You can also subscribe to my_[ _email list_](https://medium.com/subscribe/@petrkorab) _to get notified about my new articles. Thanks!_ ## References [1] Grootendorst (2022). Bertopic: Neural Topic Modeling With A Class-Based TF-IDF Procedure. [_Computer Science._](https://arxiv.org/abs/2203.05794) [2] Wu, X, Nguyen, T., Ce Zhang, D., Yang Wang, W., Luu, A. T. (2024). FASTopic: [A Fast, Adaptive, Stable, and Transferable Topic Modeling Paradigm](https://arxiv.org/abs/2405.17978). arXiv preprint: 2405.17978. ## Register Now to Continue Reading ### Unlock Towards Data Science with a Free Account All articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more. Register for freeSign in to your account * * * ![](../images/body-9e78938b5e8b0a70213c.jpeg) Written By ## [Petr Koráb](https://towardsdatascience.com/author/petrkorab/) [See all from Petr Koráb](https://towardsdatascience.com/author/petrkorab/) [Bertopic](https://towardsdatascience.com/tag/bertopic/)[Machine Learning](https://towardsdatascience.com/tag/machine-learning/)[NLP](https://towardsdatascience.com/tag/nlp/)[Text Mining](https://towardsdatascience.com/tag/text-mining/)[Topic Modelling](https://towardsdatascience.com/tag/topic-modelling/) Share This Article * [](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Ftowardsdatascience.com%2Ftopic-modelling-in-business-intelligence-fastopic-and-bertopic-in-code-2d3949260a37%2F&title=Topic%20Modelling%20in%20Business%20Intelligence%3A%20FASTopic%20and%20BERTopic%20in%20Code) * [](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Ftowardsdatascience.com%2Ftopic-modelling-in-business-intelligence-fastopic-and-bertopic-in-code-2d3949260a37%2F&title=Topic%20Modelling%20in%20Business%20Intelligence%3A%20FASTopic%20and%20BERTopic%20in%20Code) * [](https://x.com/share?url=https%3A%2F%2Ftowardsdatascience.com%2Ftopic-modelling-in-business-intelligence-fastopic-and-bertopic-in-code-2d3949260a37%2F&text=Topic%20Modelling%20in%20Business%20Intelligence%3A%20FASTopic%20and%20BERTopic%20in%20Code) Towards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program. [Write for TDS](https://towardsdatascience.com/submissions/) ## Related Articles * [![](../images/008.png)](https://towardsdatascience.com/dive-into-lora-adapters-38f4da488ede/) ## [Dive Into LoRA Adapters](https://towardsdatascience.com/dive-into-lora-adapters-38f4da488ede/) [Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/) Exploring Parameter Efficient Finetuning (PEFT): Intuitively Understanding Finetuning Using LoRA [Mariano Kamp](https://towardsdatascience.com/author/mkamp/) August 25, 202315 min read * [![](../images/009.jpeg)](https://towardsdatascience.com/the-messages-we-send-the-case-against-deploying-machine-learning-3d123a18f13d/) ## [The Messages We Send: The Case Against "Deploying" Machine Learning](https://towardsdatascience.com/the-messages-we-send-the-case-against-deploying-machine-learning-3d123a18f13d/) [Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/) Why Changing Our Language Enables More Responsible AI [Ayodele Odubela](https://towardsdatascience.com/author/ayodeleodubela/) June 8, 20214 min read * [![](../images/010.png)](https://towardsdatascience.com/regularization-what-why-when-and-how-d4a329b6b27f/) ## [Regularization. What, Why, When, and How?](https://towardsdatascience.com/regularization-what-why-when-and-how-d4a329b6b27f/) [Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/) In this article, I want to take an in-depth look at regularization. [Akash Shastri](https://towardsdatascience.com/author/akashgshastri/) October 24, 20205 min read * [![](../images/011.jpeg)](https://towardsdatascience.com/machine-learning-with-googles-bigquery-6602e734bf74/) ## [Machine Learning with Google's BigQuery](https://towardsdatascience.com/machine-learning-with-googles-bigquery-6602e734bf74/) [Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/) How to easily create and deploy ML Models with SQL [Christianlauer](https://towardsdatascience.com/author/christianlauer90/) June 23, 20214 min read * [![](../images/012.png)](https://towardsdatascience.com/pytorch-and-mlx-for-apple-silicon-4f35b9f60e39/) ## [PyTorch and MLX for Apple Silicon](https://towardsdatascience.com/pytorch-and-mlx-for-apple-silicon-4f35b9f60e39/) [Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/) A side-by-side CNN implementation and comparison [Mike Cvet](https://towardsdatascience.com/author/mikecvet/) March 8, 20249 min read * [![](../images/body-63beb50f664a055ed739.jpg)](https://towardsdatascience.com/using-complex-networks-to-improve-machine-learning-methods-fe2c034f6de3/) ## [Using Complex Networks to improve Machine Learning methods](https://towardsdatascience.com/using-complex-networks-to-improve-machine-learning-methods-fe2c034f6de3/) [Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/) Using the High-Level Data Classification Algorithm to leverage topological characteristics of the data [Tiago Toledo Jr.](https://towardsdatascience.com/author/tiagotoledojr/) October 26, 20216 min read * [![](../images/014.jpeg)](https://towardsdatascience.com/4-useful-clustering-methods-you-should-know-in-2021-ac61301e968e/) ## [4 Useful clustering methods you should know in 2021](https://towardsdatascience.com/4-useful-clustering-methods-you-should-know-in-2021-ac61301e968e/) [Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/) Form groups of similar observations based on distance [Rukshan Pramoditha](https://towardsdatascience.com/author/rukshanpramoditha/) March 30, 20217 min read * [![](../images/015.jpg)](https://towardsdatascience.com/what-is-stationarity-in-time-series-and-why-should-you-care-f3b45082356b/) ## [What is Stationarity in Time Series and why should you care](https://towardsdatascience.com/what-is-stationarity-in-time-series-and-why-should-you-care-f3b45082356b/) [Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/) Time Series are Everywhere - Make sure you know how to Handle them [Dario Radečić](https://towardsdatascience.com/author/radecicdario/) January 11, 20206 min read * [![](../images/016.jpg)](https://towardsdatascience.com/the-machine-learning-advent-calendar-day-13-lasso-and-ridge-regression-in-excel/) ## [The Machine Learning “Advent Calendar” Day 13: LASSO and Ridge Regression in Excel](https://towardsdatascience.com/the-machine-learning-advent-calendar-day-13-lasso-and-ridge-regression-in-excel/) [Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/) Why regularization is not about changing the model, but about choosing better coefficients [Angela and Kezhan Shi](https://towardsdatascience.com/author/angela.shi/) December 13, 20259 min read * [YouTube](https://www.youtube.com/c/TowardsDataScience) * [X](https://x.com/TDataScience) * [LinkedIn](https://www.linkedin.com/company/towards-data-science/) * [Threads](https://www.threads.net/@towardsdatascience) * [Bluesky](https://bsky.app/profile/towardsdatascience.com) [](https://towardsdatascience.com/) Your home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals. Subscribe to Our Newsletter[Write For TDS](https://towardsdatascience.com/submissions/)•[About](https://towardsdatascience.com/about-towards-data-science-d691af11cc2f/)•[Advertise](https://contact.towardsdatascience.com/advertise-with-towards-data-science)•[Privacy Policy](https://towardsdatascience.com/privacy-policy/)•[Terms of Use](https://towardsdatascience.com/website-terms-of-use/) © Insight Media Group, LLC 2026•[The New Stack](https://thenewstack.io)•[roadmap.sh](https://roadmap.sh)•[Creator Network](https://insightmediagroup.io/creator-network) #### Recommended Articles Close * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607686/image?etag=1782161464448&width=500&height=500) How Relevance Models Foreshadowed Transformers for NLP ](https://towardsdatascience.com/how-relevance-models-foreshadowed-transformers-for-nlp/ "How Relevance Models Foreshadowed Transformers for NLP") * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500) Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1 ](https://towardsdatascience.com/javascript-fatigue-you-dont-need-js-to-build-chatgpt/ "Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1") * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500) The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs ](https://towardsdatascience.com/subset-sum-problem-solved-in-linear-time-for-dense-enough-inputs/ "The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs") * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/3660/image?etag=1780176246070&width=500&height=500) Does Your Company Have a Data Strategy? ](https://towardsdatascience.com/does-your-company-have-a-data-strategy-e8944dbec4ae/ "Does Your Company Have a Data Strategy?")