{
  "source_file": "0930",
  "source_platform": "Dev.to",
  "author": "Javier Aguirre",
  "title_listed": "Bag of Words & TF-IDF: How NLP Turns Text into Numbers",
  "keywords": "",
  "domain": "文本处理",
  "content_type": "技术教程",
  "is_paid_flag": "否",
  "url": "https://dev.to/javiagu13/bag-of-words-tf-idf-how-nlp-turns-text-into-numbers-4b65",
  "publish_date": "2026-09-14",
  "corpus_stage": "",
  "notes": "",
  "id": "97f025734024",
  "collected_at": "2026-10-02T13:54:07",
  "title": "Bag of Words & TF-IDF: How NLP Turns Text into Numbers - DEV Community",
  "body_markdown": "If you’ve been following this series, you’ve already seen NLP do some impressive things with language.\nYou’ve seen how tokenization breaks raw text into individual pieces a program can work with. You’ve seen POS tagging assign grammatical labels to words. You’ve seen dependency parsing map the relationships between words in a sentence. And you’ve seen word sense disambiguation use context to determine whether “bank” refers to a financial institution or the side of a river.\nThose techniques are far from academic curiosities. They’re useful in their own right and continue to play important roles in many real-world systems. Search engines, information extraction pipelines, document processing systems, and domain-specific NLP applications often rely on linguistic analysis to improve accuracy and relevance.\nBut there is another challenge we haven’t addressed yet.\nMany of the algorithms used for search, ranking, classification, clustering, and machine learning operate on numerical representations rather than words and linguistic labels alone. To compare documents mathematically, measure similarity, train models, or rank search results, we need a way to convert text into vectors of numbers.\nThis process is known as\ntext vectorization\n.\nBag of Words\nand\nTF-IDF\nare two of the foundational approaches to solving that problem. They don’t replace tokenization, POS tagging, dependency parsing, or other linguistic techniques. Instead, they provide a numerical representation that can be used alongside them in larger NLP systems.\nThese methods are simple enough to understand in an afternoon, powerful enough to have driven production systems for decades, and still widely used today. Modern search engines, retrieval systems, and machine learning pipelines often combine classical techniques such as TF-IDF and BM25 with linguistic analysis and neural models rather than choosing one approach over another.\nIn this post, we’ll explore Bag of Words and TF-IDF from first principles: the intuition behind them, the mathematics that makes them work, and practical Python examples you can run yourself.\nThe Core Problem: Computers Can’t Read Text\nImagine you’re building a spam filter. You want your program to look at an email and decide: spam or not spam?\nTo do that, you need to compare emails somehow. You need to ask questions like:\ndoes this email look more like the spam emails I’ve seen, or more like the legitimate ones?\nBut you can’t compare raw text directly in any meaningful mathematical way. What would it even mean to subtract one sentence from another? How do you calculate the “distance” between\n“Win a free prize!”\nand\n“Your invoice is attached”\n?\nWhat you need is a way to represent each document as a set of numbers (a\nvector\n) so that you can apply all the usual mathematical tools to them: measuring distance, finding similarities, feeding them into classifiers.\nThis transformation from raw text into numerical vectors is called\ntext vectorization\n. It’s the foundation that everything else in NLP sits on.\nOne important thing to set expectations correctly before we dive in: the classical methods we’re covering today (Bag of Words and TF-IDF) capture\nshared vocabulary\n, not meaning. Two documents get similar vectors if they use many of the same words, not simply because they express the same idea. A document about “automobiles” and one about “cars” could look quite different to these methods, even though they cover the same subject. That’s one of the limitations we’ll come back to, and a big part of why word embeddings and transformers were developed later.\nBag of Words and TF-IDF are the two classic approaches to text vectorization.\nBag of Words: Counting What’s There\nThe Basic Idea\nThe bag of words model is built on a beautifully simple premise:\na document can be represented by the words it contains, and how often they appear.\nThat’s it. You throw away sentence structure, word order, grammar: everything except the raw inventory of words and their counts.\nWhy “bag”? Because imagine tipping all the words out of a document into a bag and shaking it. The structure is gone. The order is gone. What’s left is just a jumbled collection of words. That jumbled collection is your representation.\nHere’s a concrete example. Suppose you have three short documents:\nDocument 1:\n“The cat sat on the mat”\nDocument 2:\n”The cat sat on the hat”\nDocument 3:\n”The dog lay on the mat”\nThe first step is to build a\nvocabulary\n: the complete list of unique words across all documents.\nVocabulary: [the, cat, sat, on, mat, hat, dog, lay]\nEnter fullscreen mode\nExit fullscreen mode\nNow you represent each document as a vector of word counts — one number per word in the vocabulary:\nEach row is now a vector of numbers. Documents 1 and 2 share most of the same words, so their vectors look similar. Document 3 shares fewer words with the others, which shows up in the numbers too.\nYou’ve just turned text into maths.\nNotice something about these vectors: most of the entries are zero. Each document only contains a handful of the words in the full vocabulary, so most positions are empty. This is called a\nsparse vector\n, and it’s a defining characteristic of both Bag of Words and TF-IDF. In a real corpus with a vocabulary of tens or hundreds of thousands of words, a typical document might use only a few hundred of them, meaning the vast majority of entries in its vector are zero.\nThis sparsity has practical consequences. Sparse vectors can be stored and computed with efficiently using specialised data structures, but they’re very high-dimensional, and all those zeros mean the vectors don’t capture much about\nwhat words mean\n, only\nwhich words appear\n. Later in this series we’ll look at\ndense embeddings\n(compact vectors where every entry carries meaning) which take a fundamentally different approach to representing text. But sparse vectors remain useful and widely deployed, as we’ll see.\nBag of Words (BoW) — Sparse Vector Example\nWhat Bag of Words Is Actually Good For\nOnce your documents are vectors, you can do real computation on them. You can measure how similar two documents are by comparing their vectors. You can feed them into a machine learning classifier. You can cluster them, search them, rank them.\nThe bag of words model works surprisingly well for many tasks:\nText classification\n: Is this review positive or negative? The presence of words like\n“great,” “loved,” “excellent”\nversus\n“terrible,” “disappointed,” “broken”\ncarries a lot of signal.\nSpam detection\n: Certain words and phrases appear much more often in spam than in legitimate email.\nDocument similarity\n: Two documents covering the same topic will tend to use many of the same words.\nThe key insight is that\nfor a lot of real-world tasks, what matters most is which words are present, not the exact order they appear in.\nThe Obvious Limitations\nBag of words is simple by design, and that simplicity comes with real costs.\nWord order is lost entirely.\n“The dog bit the man” and “The man bit the dog” produce identical bag of words vectors, even though they mean opposite things.\nCommon words drown out meaningful ones.\nWords like\n“the,” “a,” “is,” “and”\nappear in almost every document. They dominate the word counts but carry almost no useful information. You can partially fix this with a\nstop word list\n(a predefined list of common words to ignore) but the underlying problem remains.\nEvery word is treated as equally important.\nA document about machine learning that mentions “neural” twice gets the same weight as one that mentions it twenty times. And a rare technical term that appears once gets the same weight as\n“the”\nappearing fifty times.\nThat last problem is exactly what TF-IDF was designed to solve.\nTF-IDF: Not All Words Are Created Equal\nThe Intuition\nThink about the word “machine” in a collection of technology articles. It probably appears in a lot of documents. It’s somewhat useful for identifying tech content, but it’s not very distinctive.\nNow think about the word “backpropagation.” If that word appears in a document, you know a lot about what that document is about. It’s rare in most text, but highly specific and meaningful in context.\nThe core intuition behind TF-IDF is:\na word is important if it appears a lot in this specific document, but not in many other documents.\nWords that appear everywhere are not very useful for distinguishing one document from another. Words that appear in only a few documents are highly distinctive. A good word representation should reward distinctive words and discount common ones.\nTF-IDF\n(Term Frequency-Inverse Document Frequency) formalises this intuition with a simple formula.\nTerm Frequency\nTerm frequency\nmeasures how often a word appears in a document.\nThe simplest version is a raw count: if\n“neural”\nappears 4 times in a document, its term frequency is 4. In practice, implementations vary — some use the raw count, some normalise by dividing by the total number of words in the document (so longer documents don’t automatically produce higher scores), and some apply logarithmic scaling to reduce the impact of words that appear very many times. Scikit-learn’s\nTfidfVectorizer\n, for instance, uses a specific variant under the hood. The core idea is the same across all of them: measure how prominent this word is within this document.\nThe normalised version looks like this:\nTF(word, document) = (number of times word appears in document) / (total words in document)\nEnter fullscreen mode\nExit fullscreen mode\nFor example, if\n“neural”\nappears 4 times in a 200-word document:\nTF = 4 / 200 = 0.02\nEnter fullscreen mode\nExit fullscreen mode\nThis measures\nlocal\nimportance: how much does this word dominate this particular document?\nInverse Document Frequency\nInverse document frequency\nmeasures how rare or common a word is across the entire collection.\nIDF(word) = log( total number of documents / number of documents containing word )\nEnter fullscreen mode\nExit fullscreen mode\nIf a word appears in every document, the fraction inside the log becomes 1, and log(1) = 0. The IDF is zero. That word gets no weight at all.\nIf a word appears in only 1 out of 1,000 documents, the fraction is 1,000, and log(1,000) is a large number. That word gets a high IDF score: it’s highly distinctive.\nThe logarithm is there to compress the scale. Without it, a word appearing in 1 out of 1,000,000 documents would get a weight a thousand times higher than a word appearing in 1 out of 1,000. The log smooths that out.\nPutting Them Together\nThe TF-IDF score for a word in a document is simply:\nTF-IDF(word, document) = TF(word, document) × IDF(word)\nEnter fullscreen mode\nExit fullscreen mode\nWords that are frequent in this document\nand\nrare across the collection get high scores. Words that are common everywhere get scores near zero.\nLet’s work through a simple example. Suppose you have a collection of 1,000 news articles.\nEven though “the” has the highest term frequency, it gets a TF-IDF of zero because it appears in every document. “Candidacy” is rarer and more distinctive, so it gets the highest score despite appearing less often in this specific article.\nThis is exactly what you’d want. If you’re trying to understand what makes this article distinct, “the” tells you nothing. “Candidacy” tells you a lot.\nTF-IDF Scoring VIsualizaion\nThe TF-IDF Vectorizer: From Concept to Tool\nIn practice, you rarely compute TF-IDF by hand. You use a\nTF-IDF vectorizer\n: a ready-made tool that takes a collection of documents and transforms each one into a TF-IDF vector automatically.\nThe vectorizer handles all the bookkeeping: building the vocabulary, computing term frequencies, calculating IDF scores across the entire corpus, and assembling the final document-term matrix.\nThe output is the same shape as a bag of words matrix (rows are documents, columns are words) but now each cell contains a TF-IDF score instead of a raw count.\nTF-IDF in Python\nPython’s scikit-learn library includes a\nTfidfVectorizer\nthat makes this trivially easy.\nfrom sklearn.feature_extraction.text import TfidfVectorizer  \n\n# Your documents  \ndocuments = [  \n    \"The cat sat on the mat\",  \n    \"The cat sat on the hat\",  \n    \"The dog lay on the mat\"  \n]  \n\n# Create and fit the vectorizer  \nvectorizer = TfidfVectorizer()  \ntfidf_matrix = vectorizer.fit_transform(documents)  \n\n# See the vocabulary  \nprint(vectorizer.get_feature_names_out())  \n# ['cat', 'dog', 'hat', 'lay', 'mat', 'on', 'sat', 'the']  \n\n# See the TF-IDF scores for document 1  \nprint(tfidf_matrix[0].toarray())\nEnter fullscreen mode\nExit fullscreen mode\nThe\nfit_transform\ncall does everything at once: it learns the vocabulary and IDF weights from your documents (\nfit\n), then transforms each document into a TF-IDF vector (\ntransform\n).\nIf you later want to transform new documents using the same vocabulary and IDF weights (which you almost always do when building a real system) you call\ntransform\nalone:\nnew_docs = [\"The cat lay on the hat\"]  \nnew_vectors = vectorizer.transform(new_docs)\nEnter fullscreen mode\nExit fullscreen mode\nYou can also use\nCountVectorizer\nif you just want raw word counts (the bag of words approach):\nfrom sklearn.feature_extraction.text import CountVectorizer  \n\nbow_vectorizer = CountVectorizer()  \nbow_matrix = bow_vectorizer.fit_transform(documents)\nEnter fullscreen mode\nExit fullscreen mode\nBoth vectorizers support useful parameters for cleaning your text automatically:\nvectorizer = TfidfVectorizer(  \n    stop_words='english',       # Remove common English stop words  \n    max_features=10000,         # Only keep the 10,000 most frequent words  \n    ngram_range=(1, 2),         # Include both single words and two-word phrases  \n    min_df=2                    # Ignore words that appear in fewer than 2 documents  \n)\nEnter fullscreen mode\nExit fullscreen mode\nA Practical Example: Finding Similar Documents\nHere’s a slightly more realistic use case: using TF-IDF vectors to find which documents are most similar to a query.\nfrom sklearn.feature_extraction.text import TfidfVectorizer  \nfrom sklearn.metrics.pairwise import cosine_similarity  \nimport numpy as np  \n\n# A small collection of descriptions  \ndocuments = [  \n    \"Machine learning algorithms learn patterns from data\",  \n    \"Deep learning uses neural networks with many layers\",  \n    \"Natural language processing handles text and speech\",  \n    \"Computer vision processes images and video\",  \n    \"Neural networks are inspired by the human brain\"  \n]  \n\n# Fit and transform  \nvectorizer = TfidfVectorizer()  \ntfidf_matrix = vectorizer.fit_transform(documents)  \n\n# A search query  \nquery = [\"neural networks and deep learning\"]  \nquery_vector = vectorizer.transform(query)  \n\n# Calculate similarity between query and all documents  \nsimilarities = cosine_similarity(query_vector, tfidf_matrix).flatten()  \n\n# Rank documents by similarity  \nranked = np.argsort(similarities)[::-1]  \nprint(\"Most similar documents:\")  \nfor i in ranked:  \n    print(f\"  {similarities[i]:.3f} — {documents[i]}\")\nEnter fullscreen mode\nExit fullscreen mode\nOutput:\nMost similar documents:  \n  0.712 — Neural networks are inspired by the human brain  \n  0.634 — Deep learning uses neural networks with many layers  \n  0.000 — Machine learning algorithms learn patterns from data  \n  0.000 — Natural language processing handles text and speech  \n  0.000 — Computer vision processes images and video\nEnter fullscreen mode\nExit fullscreen mode\nThe query\n“neural networks and deep learning”\ncorrectly identifies the two most relevant documents. Documents that don’t mention neural networks at all score zero.\nNotice what TF-IDF is actually doing here: it’s matching shared terms, not understanding concepts. It found the right documents because the query and those documents happen to use the same words. If one of our documents had said “artificial neurons arranged in layers” instead of “neural networks,” TF-IDF would have given it a score of zero. The terminology doesn’t overlap, so the vectors don’t either. This is the core limitation of all sparse, vocabulary-based approaches. It works well when related documents use consistent terminology, which is true in many real-world settings. When it breaks down, that’s where dense embeddings and semantic search come in.\nThis term-matching approach is essentially how document search worked for decades — and in many systems, it still does.\nBag of Words vs TF-IDF: When to Use Which\nBoth approaches transform text into vectors. The difference is what those numbers mean and how much signal they carry.\nUse Bag of Words (raw counts) when:\n— Your task is simple and you want maximum interpretability — You’re working with very short texts where document length doesn’t vary much — You’re doing a quick baseline before trying something more sophisticated\nUse TF-IDF when:\n— Your documents vary in length (longer documents shouldn’t automatically dominate) — You want distinctive terms to be weighted more heavily than ubiquitous ones — You’re doing search, document similarity, or text classification where specificity matters\nIn practice, TF-IDF almost always outperforms raw bag of words for text classification and search tasks. It’s the default choice when you want a classical vectorization approach.\nWhere These Methods Fit in Modern NLP\nIf TF-IDF has been around since the 1970s, you might wonder whether it’s still worth learning.\nThe honest answer: yes, absolutely. And not just for historical context.\nTransformer models like BERT and GPT brought\ncontextual embeddings\nto NLP, which capture meaning, sentence structure, and semantic relationships in ways sparse methods can’t. That was a genuine step forward for many tasks. But it didn’t make sparse retrieval disappear. The story is more nuanced than “new methods replaced old ones.”\nIn practice, many production systems today use\nhybrid architectures\n: they run a sparse retrieval step (BM25, TF-IDF) alongside a dense neural retrieval step, then combine the results. Each component handles what it’s good at. Sparse methods are fast, interpretable, and excellent at exact keyword matching. Dense methods are better at semantic similarity: finding documents that mean the same thing even when they use different words. Neither has made the other obsolete. Classical NLP techniques — tokenization, lemmatization, POS tagging, dependency parsing, TF-IDF, and the retrieval pipelines built around them — remain important parts of real production systems.\nNLP Vectoriation Progression\nTF-IDF’s direct successor,\nBM25\n, is one of the most widely deployed ranking algorithms in the world right now. BM25 takes the core TF-IDF idea and fixes two specific weaknesses: it handles diminishing returns on term frequency (the 50th mention of a word shouldn’t count as much as the 5th), and it normalises more gracefully for document length. Elasticsearch, OpenSearch, and most modern search engines use BM25 as their default relevance model. It’s also the sparse retrieval component in many of those hybrid systems mentioned above. We’ll cover BM25 in the next post. Once you’ve understood TF-IDF, it clicks almost immediately.\nUnderstanding Bag of Words and TF-IDF gives you the conceptual foundation to understand why every subsequent approach works differently, and what specific problem each one was built to solve. Dense embeddings exist because sparse methods can’t capture meaning across different vocabulary. BM25 exists because raw TF-IDF has specific edge cases in term weighting and length normalisation. Each step in the progression was motivated by a concrete limitation of what came before — not a wholesale replacement of it.\nEvery serious NLP practitioner knows these methods. They’re the starting point for a reason.\nWhat Comes Next\nWe’ve now covered how text gets turned into numbers — the prerequisite for almost every NLP task that follows.\nThe immediate next step is\nBM25\n: the direct evolution of TF-IDF that fixes its two most significant weaknesses and has become the default ranking algorithm inside most modern search engines. If you search anything on Elasticsearch or use a hybrid retrieval system, BM25 is almost certainly involved. Because it builds directly on what you’ve just learned, it’s the natural next post.\nAfter that, we’ll go deeper into what you can\ndo\nwith these vectors — the machine learning models that sit on top of text vectorization, and the neural architectures that eventually pushed beyond classical methods altogether. That includes\nword embeddings\nlike Word2Vec and GloVe, which represent words as dense vectors rather than sparse counts, and eventually the contextual embeddings that power modern AI.\nLearn This at Fondra Labs\nThis post is part of our\nNLP Foundations\nseries, where we build up practical AI knowledge one concept at a time, from text processing basics all the way to the systems powering modern AI.\nAt\nFondra Labs\n, we teach AI from production reality, not hype. Every topic in this series is here because it genuinely matters when you sit down to build something real.\nIf this was useful, explore the rest of the blog. We cover machine learning, deep learning, NLP, and the practical skills that turn understanding into building.\nOriginally published at\nhttps://fondralabs.com\n.",
  "metadata": {
    "title": "Bag of Words & TF-IDF: How NLP Turns Text into Numbers - DEV Community",
    "description": "If you’ve been following this series, you’ve already seen NLP do some impressive things with... Tagged with ai, nlp, machinelearning.",
    "keywords": "ai, nlp, machinelearning, software, coding, development, engineering, inclusive, community",
    "author": null,
    "og:type": "article",
    "og:url": "https://dev.to/javiagu13/bag-of-words-tf-idf-how-nlp-turns-text-into-numbers-4b65",
    "og:title": "Bag of Words & TF-IDF: How NLP Turns Text into Numbers",
    "og:description": "If you’ve been following this series, you’ve already seen NLP do some impressive things with...",
    "og:site_name": "DEV Community",
    "og:image": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fuuvhq0n0u196mkho7d3d.png",
    "twitter:site": "@thepracticaldev",
    "twitter:creator": "@",
    "twitter:title": "Bag of Words & TF-IDF: How NLP Turns Text into Numbers",
    "twitter:description": "If you’ve been following this series, you’ve already seen NLP do some impressive things with...",
    "twitter:card": "summary_large_image",
    "twitter:widgets:new-embed-design": "on",
    "twitter:image:src": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fuuvhq0n0u196mkho7d3d.png"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [
    {
      "file": "attachments/97f025734024/code/body-000.txt",
      "chars": 52,
      "language": "text"
    },
    {
      "file": "attachments/97f025734024/code/body-001.txt",
      "chars": 92,
      "language": "text"
    },
    {
      "file": "attachments/97f025734024/code/body-002.txt",
      "chars": 20,
      "language": "text"
    },
    {
      "file": "attachments/97f025734024/code/body-003.txt",
      "chars": 83,
      "language": "text"
    },
    {
      "file": "attachments/97f025734024/code/body-004.txt",
      "chars": 56,
      "language": "text"
    },
    {
      "file": "attachments/97f025734024/code/body-005.txt",
      "chars": 521,
      "language": "text"
    },
    {
      "file": "attachments/97f025734024/code/body-006.txt",
      "chars": 85,
      "language": "text"
    },
    {
      "file": "attachments/97f025734024/code/body-007.txt",
      "chars": 153,
      "language": "text"
    },
    {
      "file": "attachments/97f025734024/code/body-008.txt",
      "chars": 350,
      "language": "text"
    },
    {
      "file": "attachments/97f025734024/code/body-009.txt",
      "chars": 1030,
      "language": "text"
    },
    {
      "file": "attachments/97f025734024/code/body-010.txt",
      "chars": 332,
      "language": "text"
    }
  ],
  "images": [
    {
      "status": "ok",
      "url": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fwvnkjop0qav0suswyq3r.png",
      "bytes": 120438,
      "reused": true,
      "local_path": "attachments/97f025734024/images/001.png",
      "src": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fwvnkjop0qav0suswyq3r.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fm8paiqph698rkqh9h4fj.png",
      "bytes": 22046,
      "reused": true,
      "local_path": "attachments/97f025734024/images/002.png",
      "src": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fm8paiqph698rkqh9h4fj.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Frw0tggphc1cjwk4191pk.png",
      "bytes": 61736,
      "reused": true,
      "local_path": "attachments/97f025734024/images/003.png",
      "src": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Frw0tggphc1cjwk4191pk.png",
      "alt": "Bag of Words (BoW) — Sparse Vector Example",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fppos8m0imwgi92n9wr8c.png",
      "bytes": 41008,
      "reused": true,
      "local_path": "attachments/97f025734024/images/004.png",
      "src": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fppos8m0imwgi92n9wr8c.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Frvubdc4jiyihuje8or4p.png",
      "bytes": 55790,
      "reused": true,
      "local_path": "attachments/97f025734024/images/005.png",
      "src": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Frvubdc4jiyihuje8or4p.png",
      "alt": "TF-IDF Scoring VIsualizaion",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Frfo80z2xkqnrqib879us.png",
      "bytes": 54818,
      "reused": true,
      "local_path": "attachments/97f025734024/images/006.png",
      "src": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Frfo80z2xkqnrqib879us.png",
      "alt": "NLP Vectoriation Progression",
      "downloaded": true
    }
  ],
  "audio": [],
  "videos": [],
  "links": [
    {
      "href": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fwvnkjop0qav0suswyq3r.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fm8paiqph698rkqh9h4fj.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Frw0tggphc1cjwk4191pk.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fppos8m0imwgi92n9wr8c.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Frvubdc4jiyihuje8or4p.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Frfo80z2xkqnrqib879us.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://fondralabs.com/nlp-foundations.html",
      "label": "NLP Foundations",
      "type": "a"
    },
    {
      "href": "https://fondralabs.com/",
      "label": "Fondra Labs",
      "type": "a"
    },
    {
      "href": "https://fondralabs.com/blog/nlp-foundations/bag-of-words-tf-idf-how-nlp-turns-text-into-numbers.html",
      "label": "https://fondralabs.com",
      "type": "a"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:48:54Z",
  "source_url": "https://dev.to/javiagu13/bag-of-words-tf-idf-how-nlp-turns-text-into-numbers-4b65",
  "papers": [],
  "actual_video_files": [],
  "body_chars": 21264,
  "expected_body_images": 6,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/97f025734024/pages/source.html",
  "local_html": "attachments/97f025734024/pages/article.html",
  "body_base_path": "attachments/97f025734024/pages/",
  "body_markdown_path": "attachments/97f025734024/pages/article.md"
}