保留正文与补采附件。源页面当前未重新核验；具体限制见交付清单。

Revisit consent button We value your privacy We use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking "Accept All", you consent to our use of cookies. Customise Reject All Accept All Customise Consent Preferences We use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below. The cookies that are categorised as "Necessary" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... Show more NecessaryAlways Active Necessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.

Functional Functional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.

Analytics Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.

Performance Performance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors.

Advertisement Advertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns.

Uncategorised Other uncategorised cookies are those that are being analysed and have not been classified into a category as yet. No cookies to display. Reject All Save My Preferences Accept All Toggle SearchToggle Mobile Navigation Publish AI, ML & data-science insights to a global community of data professionals. Sign inSubmit an Article LatestEditor's PicksNewsletterShipAI

Write For TDS LinkedInXToggle Search Machine Learning

The Machine Learning “Advent Calendar” Day 22: Embeddings in Excel

Understanding text embeddings through simple models and Excel Angela and Kezhan Shi December 22, 20257 min read Embeddings in Excel Embeddings in Excel For the remaining days of this series, we will talk about deep learning. And when people talk about deep learning, we immediately think of these images of deep neural networks architectures, with many layers, neurons, and parameters. In practice, the real shift introduced by deep learning is elsewhere. It is about learning data representations. In this article, we focus on text embeddings , explain their role in the machine learning landscape, and show how they can be understood and explored in Excel.

1. Classic Machine earning vs. Deep learning

We will discuss, in this part, why embedding is introduced. Learn this step by step with the interactive Machine Learning roadmap.

1.1 Where does deep learning fit?

To understand embeddings, we first need to clarify the place of deep learning. We will use the term classic machine learning to describe methods that do not rely on deep architectures. All the previous articles deal with classic machine learning, that can be described in two complementary ways. Learning paradigms

Model families

Across this series, we have already studied the learning algorithms behind these models. In particular, we have seen that gradient descent applies to all weight-based models, from linear regression to neural networks. Deep learning is often reduced to neural networks with many layers. But this explanation is incomplete. From an optimization point of view, deep learning does not introduce a new learning rule. So what does it introduce?

1.2 Deep learning as data representation learning

Deep learning is about how features are created. Instead of manually designing features, deep learning learns representations automatically , often through multiple successive transformations. This also raises an important conceptual question: Where is the boundary between feature engineering and model learning? Some examples make this clearer:

Deep learning continues this idea, but at scale. From this perspective, deep learning belongs to:

1.3 Images and convolutional neural networks

Images are represented as pixels. From a technical point of view, image data is already numerical and structured: a grid of numbers. However, the information contained in these pixels is not structured in a way that classical models can easily exploit. Pixels do not explicitly encode: edges, shapes, textures, or objects. Convolutional Neural Networks (CNNs) are designed to create information from pixels. They apply filters to detect local patterns, then progressively combine them into higher-level representations. I have published a this article showing how CNNs can be implemented in Excel to make this process explicit. CNN in Excel - all images by author For images, the challenge is not to make the data numerical, but to extract meaningful representations from already numerical data.

1.4 Text data: a different problem

Text presents a fundamentally different challenge. Unlike images, text is not numerical by nature. Before modeling context or order, the first problem is more basic: How do we represent words numerically? Creating a numerical representation for text the first step. In deep learning for text, this step is handled by embeddings. Embeddings transform discrete symbols (words) into vectors that models can work with. Once embeddings exist, we can then model: context, order and relationships between words. In this article, we focus on this first and essential step:
how embeddings create numerical representations for text , and how this process can be explored in Excel.

2. Two ways to learn text embeddings

In this article, we will use the IMDB movie reviews dataset to illustrate both approaches. The dataset is distributed under the Apache License 2.0. There are two main ways to learn embeddings for text, and we will do both with this dataset:

In both cases, the goal is the same:
to transform words into numerical vectors that can be used by machine learning models. If you test UNDERSTAND_ML, you may notice a difference. Before comparing these two approaches, we first need to clarify what embeddings are and how they relate to classic machine learning. IMDB dataset image by author -Apache License 2.0

2.1 Embeddings and classic machine learning

In classic machine learning, categorical data is usually handled with:

How they can be used depend on the nature of the models. Distance-based models cannot effectively use one-hot encoding, because all categories end up being equally distant from each other. Label encoding could work only if we can attribute meaningful numerical values for the categories, which is generally not the case in classic models. Weight-based models can use one-hot encoding, because the model learns a weight for each category. In contrast, with label encoding, the numerical values are fixed and cannot be adjusted to represent meaningful relationships. Tree-based models treat all variables as categorical splits rather than numerical magnitudes, which makes label encoding acceptable in practice. However, most implementations, including scikit-learn, still require numerical inputs. As a result, categories must be converted to numbers, either through label encoding or one-hot encoding. If the numerical values carried semantic meaning, this would again be beneficial. Overall, this highlights a limitation of classic approaches:
category values are fixed and not learned. Embeddings extend this idea by learning the representation itself.
Each word is associated with a trainable vector, turning the representation of categories into a learning problem rather than a preprocessing step.

2.2 Supervised embeddings

In supervised learning, embeddings are learned as part of a prediction task. For example, the IMDB dataset has labels about the in sentiment analysis. So we can create a very simple architecture: In our case, we can use a very simple architecture: each word is mapped to a one-dimensional embedding This is possible because the objective is binary sentiment classification. Once training is complete, we can export the embeddings and explore them in Excel. When plotting the embeddings on the x-axis and word frequency on the y-axis, a clear pattern appears:

Depending on the initialization, the sign can be inverted, since the logistic regression layer also has parameters that influence the final prediction. Finally, in Excel, we reconstruct the full pipeline that corresponds to the architecture we define early. Input column
The input text (a review) is cut into words, and each row corresponds to one word. Embedding search
Using a lookup function, the embedding value associated with each word is retrieved from the embedding table learned during training. Global average
The global average embedding is computed by averaging the embeddings of all words seen so far. This corresponds to a very simple sentence representation: the mean of word vectors. Probability prediction
The averaged embedding is then passed through a logistic function to produce a sentiment probability. What we observe

As more words are added, the global average embedding stabilizes, and the sentiment prediction becomes more confident.

2.3 Word2Vec: embeddings from co-occurrence

In Word2Vec, similarity does not mean that two words have the same meaning.
It means that they appear in similar contexts. Word2Vec learns word embeddings by looking at which words tend to co-occur within a fixed window in the text. Two words are considered similar if they often appear around the same neighboring words , even if their meanings are opposite. As shown in the Excel sheet below, we compute the cosine similarity for the word good and retrieve the most similar words. From the model’s perspective, the surrounding words are almost identical. The only thing that changes is the adjective itself. As a result, Word2Vec learns that “good” and “bad” play a similar role in language , even though their meanings are opposite. So, Word2Vec captures distributional similarity , not semantic polarity. A useful way to think about it is: Words are close if they are used in the same places.

2.4 How embeddings are used

In modern systems such as RAG (Retrieval-Augmented Generation) , embeddings are often used to retrieve documents or passages for question answering. However, this approach has limitations. Most commonly used embeddings are trained in a self-supervised way, based on co-occurrence or contextual prediction objectives. As a result, they capture general language similarity, not task-specific meaning. This means that:

Other embedding strategies can be used, including task-adapted or supervised embeddings, but they often remain self-supervised at their core. Understanding how embeddings are created, what they encode, and what they do not encode is therefore essential before using them in downstream systems such as RAG.

Conclusion

Embeddings are learned numerical representations of words that make similarity measurable. Whether learned through supervision or through co-occurrence, embeddings map words to vectors based on how they are used in data. By exporting them to Excel, we can inspect these representations directly, compute similarities, and understand what they capture and what they do not. This makes embeddings less mysterious and clarifies their role as a foundation for more complex systems such as retrieval or RAG. ··· All the Excel files are available through this Kofi link. Your support means a lot to me. The price will increase during the month, so early supporters get the best value. All Excel/Google sheet files for ML and DL

Register Now to Continue Reading

Unlock Towards Data Science with a Free Account

All articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more. Register for freeSign in to your account

Written By

Angela and Kezhan Shi

See all from Angela and Kezhan Shi Artificial IntelligenceData ScienceDeep LearningText Embedding Share This Article

Towards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program. Write for TDS

Related Articles

Python Basics 2 - Working with List, Tuples, Dictionaries

Machine Learning Day 2 of #100DaysOfMLCode Durgesh Samariya July 30, 20204 min read

Demystifying the Correlation Matrix in Data Science

Data Science Understanding the Connections Between Variables: A Comprehensive Guide to Correlation Matrices and Their Applications Niklas Lang November 13, 202414 min read

Analysis of car accidents in Barcelona using Pandas, Matplotlib, and Folium

Data Science Open Data Barcelona is Barcelona´s data service which contains around 400 datasets, covering a wide rage of topics such as population... Amanda Iglesias Moreno September 1, 20199 min read

Cross-Entropy Demystified.

Deep Learning For the longest time, I had not completely understood Cross-Entropy loss. Why did we take exponents (softmax)? Why did we then take the... Akash Shastri October 6, 20206 min read

ML goes after chemistry and material sciences -highlights of a review of interest to everybody...

Artificial Intelligence Scientists have just set up a roadmap for the future of ML applied to chemistry and material sciences, in an article that also tracks the... Luciano Abriata September 8, 202214 min read

Computer Vision: Part 1

Artificial Intelligence An introduction to image data processing and feature extraction Aditya Mehta November 7, 20205 min read

Tabyl - a frequency table for the modern R user

Data Science Out with the old, in with the new! Zvonimir Boban May 20, 20236 min read

Polars + NVIDIA GPU Tutorial

Machine Learning Using Polars with NVIDIA GPU can speed up your data pipelines Ivo Bernardo September 17, 20244 min read

How do Decision Trees and Random Forests Work?

Machine Learning 1a Decision Trees Decision trees and random forests are two commonly used algorithms in predictive... Dick Brown May 6, 202110 min read

Your home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals. Subscribe to Our NewsletterWrite For TDS•About•Advertise•Privacy Policy•Terms of Use © Insight Media Group, LLC 2026•The New Stack•roadmap.sh•Creator Network

Recommended Articles

Close