{
  "source_file": "0930",
  "source_platform": "Dev.to",
  "author": "Javier Aguirre",
  "title_listed": "BM25 Explained: The Algorithm Behind Modern Search",
  "keywords": "",
  "domain": "文本处理",
  "content_type": "技术教程",
  "is_paid_flag": "否",
  "url": "https://dev.to/javiagu13/bm25-explained-the-algorithm-behind-modern-search-2jo3",
  "publish_date": "2026-09-16",
  "corpus_stage": "",
  "notes": "",
  "id": "b60f10ab64c6",
  "collected_at": "2026-10-02T13:54:06",
  "title": "BM25 Explained: The Algorithm Behind Modern Search - DEV Community",
  "body_markdown": "In the last post, we covered TF-IDF: the foundational technique for turning text into numbers and scoring how relevant a word is to a document. If you haven’t read that one yet, it’s worth doing first. BM25 builds directly on those ideas, and the motivation for it becomes much clearer once you’ve seen what TF-IDF gets right and where it falls short.\nHere’s a quick recap of where we left off.\nTF-IDF scores a word by multiplying two things together: how often it appears in a document (term frequency), and how rare it is across the entire collection (inverse document frequency). The result is a weight that rewards distinctive words and discounts common ones. It works well. It’s been used in real search systems for decades.\nBut it has two specific problems that become hard to ignore when you’re building anything at scale.\nThe first problem: TF-IDF has no ceiling on term frequency. If a document mentions the word “python” twice, it scores higher than one that mentions it once. Fine. But if a document mentions “python” fifty times, it scores twenty-five times higher than the document that mentions it once. At some point, more repetitions stop adding meaningful evidence that a document is relevant. The relationship isn’t linear, but TF-IDF treats it as if it is.\nThe second problem: TF-IDF doesn’t handle document length well. A long document will naturally contain more words, including more repetitions of relevant terms, than a short one. That gives longer documents a systematic advantage in raw TF scores, even if the shorter document is actually more focused and relevant.\nBM25\nwas designed to fix both of these problems. It takes the same core intuition as TF-IDF (score documents based on term frequency and rarity) but adds two refinements that make it significantly more robust in practice. Those refinements are what we’ll spend most of this post unpacking.\nWhat Is Information Retrieval?\nBefore getting into BM25 specifically, it helps to place it in context.\nInformation retrieval\nis the field concerned with finding relevant material from a large collection in response to a query. It’s what a search engine does. You type a question or a set of keywords, and the system returns a ranked list of documents it believes are most relevant to what you asked.\nThe core challenge in information retrieval is\nranking\n: given a query and a large collection of documents, how do you decide which documents to return, and in what order?\nRanking algorithms take a query and a document and produce a score. Higher score means more relevant. The entire ranked list is built by scoring every candidate document and sorting them. The quality of the search experience depends almost entirely on how good that scoring function is.\nTF-IDF was one of the earliest practical scoring functions. BM25 is its direct successor, and it remains the most widely used lexical retrieval algorithm in production systems today.\nLexical retrieval\nrefers to retrieval based on matching the actual words in the query against the actual words in the documents. No semantic understanding is involved. If the query says “python tutorial” and a document contains the words “python” and “tutorial,” that document gets a score. If a document covers the same topic but uses the phrase “programming language guide,” it may score much lower or not at all, because the words don’t overlap.\nLexical retrieval is the category that both TF-IDF and BM25 belong to. It is fast, transparent, and very effective when users search using the same terminology that appears in the documents. Its limitations are the motivation for semantic and hybrid retrieval approaches, which we’ll get to later in this series.\nWhat Is BM25?\nBM25\nstands for Best Match 25. The “25” refers to the fact that it was the 25th iteration in a series of retrieval functions developed through the Okapi research project at City University London in the 1980s and 1990s. You will sometimes see it called\nOkapi BM25\nafter that project. The name has stuck.\nThe BM25 algorithm scores a document against a query by looking at each query term individually, scoring the document for each term, and summing those scores. If your query is “machine learning tutorial,” BM25 computes a score for “machine,” a score for “learning,” and a score for “tutorial,” then adds them together to get the document’s final relevance score.\nThe score for each individual term is where the interesting work happens.\nThe BM25 Formula\nLet’s build up the formula in pieces, because each piece has a clear purpose.\nThe full BM25 score for a single query term in a document is:\nscore(term, document) = IDF(term) × [ TF(term, document) × (k1 + 1) ] / [ TF(term, document) + k1 × (1 - b + b × (doc_length / avg_doc_length)) ]\nEnter fullscreen mode\nExit fullscreen mode\nIf we display it in a mathematical formula it would be written as follows:\nBM25 Formula\nThat looks intimidating. It isn’t, once you see what each part does. Also, remember, for AI engineering knowing the mathematics is not as important as understanding the underlaying behaviours. Therefore, let me guide you through the general understanding of it.\nThe IDF Part\nThe IDF component in BM25 works similarly to TF-IDF. It measures how rare the term is across the entire collection. Rare terms that appear in only a few documents get high IDF scores. Common terms that appear everywhere get low scores.\nThe exact formula used for IDF varies slightly across implementations, but the principle is the same as in TF-IDF. We won’t dwell on it here because you already understand it from the previous post.\nThe TF Part: Saturation\nThis is the first of BM25’s two key improvements.\nIn TF-IDF, the term frequency contribution grows linearly and without limit. In BM25, the contribution of term frequency\nsaturates\nas the count increases. The 5th occurrence of a word adds meaningful evidence of relevance. The 50th occurrence adds very little on top of that.\nThe saturation is controlled by a parameter called\nk1\n. In practice, k1 is typically set between 1.2 and 2.0.\nWhen k1 is high, the saturation kicks in slowly: more repetitions keep adding score for longer. When k1 is low, the function saturates quickly and additional repetitions matter very little.\nLet’s see what this looks like concretely. Suppose k1 = 1.5 and ignore length normalisation for now.\nThe TF-IDF contribution grows without limit. The BM25 contribution approaches a ceiling (around 2.0 with k1 = 1.5) and never exceeds it. A document that mentions a term fifty times is not scored fifty times higher than one that mentions it once. That’s a much more realistic model of relevance.\nThe Length Normalisation Part\nThis is BM25’s second key improvement.\nA long document will naturally repeat terms more often than a short one, just by virtue of having more words. TF-IDF doesn’t account for this at all. BM25 adjusts for it by comparing each document’s length to the average document length across the collection.\nThis adjustment is controlled by a parameter called\nb\n, typically set around 0.75.\nWhen b = 1.0, full length normalisation is applied: longer documents are penalised proportionally. When b = 0, no length normalisation is applied, and BM25 behaves more like TF-IDF in this respect. In practice, b = 0.75 provides a partial normalisation: long documents are penalised somewhat, but not to the point where a genuinely comprehensive document is dismissed just for being thorough.\nThe intuition is: if a short document and a long document both mention a query term the same number of times, the short document is probably more focused on that topic. BM25 rewards that focus.\nBM25 term frequency saturation vs TF-IDF linear growth\nA Worked Example\nLet’s make this concrete with the same setup used in the previous post: a collection of 1,000 news articles, and a query for the word “candidacy.”\nSuppose we have two documents:\nDocument A\n(120 words): mentions “candidacy” 3 times.\nDocument B\n(600 words): mentions “candidacy” 8 times.\nWhich document should rank higher for the query “candidacy”?\nIntuitively, Document A is probably more focused on the topic. It’s short, and “candidacy” appears three times in just 120 words. Document B mentions it more often in absolute terms, but it’s a much longer piece where the term is less central.\nTF-IDF would give Document B a higher raw score just because of the higher count. BM25 adjusts for both the diminishing returns of higher counts and the longer document length.\nLet’s assume the average document length in the collection is 300 words, k1 = 1.5, and b = 0.75.\nFor Document A (3 occurrences, 120 words):\nLength ratio = 120 / 300 = 0.4  \nLength adjustment = 1 - 0.75 + 0.75 × 0.4 = 0.55  \nTF contribution = (3 × 2.5) / (3 + 1.5 × 0.55) = 7.5 / 3.825 ≈ 1.96\nEnter fullscreen mode\nExit fullscreen mode\nFor Document B (8 occurrences, 600 words):\nLength ratio = 600 / 300 = 2.0  \nLength adjustment = 1 - 0.75 + 0.75 × 2.0 = 1.75  \nTF contribution = (8 × 2.5) / (8 + 1.5 × 1.75) = 20 / 10.625 ≈ 1.88\nEnter fullscreen mode\nExit fullscreen mode\nDocument A scores slightly higher than Document B, despite fewer raw mentions. The shorter, more focused document wins. That’s the behaviour we wanted.\nBM25 in Python\nThere is no BM25 in scikit-learn, but several reliable libraries implement it. The most commonly used is\nrank_bm25\n.\npip install rank-bm25\nEnter fullscreen mode\nExit fullscreen mode\nHere’s a minimal working example:\nfrom rank_bm25 import BM25Okapi  \n\n# Your documents, pre-tokenized (split into word lists)  \ndocuments = [  \n    \"machine learning algorithms learn patterns from data\".split(),  \n    \"deep learning uses neural networks with many layers\".split(),  \n    \"natural language processing handles text and speech\".split(),  \n    \"computer vision processes images and video\".split(),  \n    \"neural networks are inspired by the human brain\".split()  \n]  \n\n# Build the BM25 index  \nbm25 = BM25Okapi(documents)  \n\n# Query  \nquery = \"neural networks deep learning\".split()  \n\n# Get scores for each document  \nscores = bm25.get_scores(query)  \nprint(scores)  \n\n# Get the top N most relevant documents  \ntop_docs = bm25.get_top_n(query, documents, n=2)  \nfor doc in top_docs:  \n    print(\" \".join(doc))\nEnter fullscreen mode\nExit fullscreen mode\nOutput:\n[0.    2.31  0.    0.    1.84]  \ndeep learning uses neural networks with many layers  \nneural networks are inspired by the human brain\nEnter fullscreen mode\nExit fullscreen mode\nA few things worth noting here.\nThe documents are passed as lists of tokens, not raw strings. That means you handle tokenization yourself before passing anything to BM25. In practice this is a feature, not a limitation: it means you can apply any preprocessing you want first (lowercasing, stemming, stop word removal, domain-specific tokenization) before the BM25 index sees the text.\nYou can also adjust k1 and b directly:\nbm25 = BM25Okapi(documents, k1=1.2, b=0.8)\nEnter fullscreen mode\nExit fullscreen mode\nA More Realistic Example\nIn practice, you’d want to preprocess your text before indexing. Here’s a slightly more complete pipeline:\nfrom rank_bm25 import BM25Okapi  \nimport re  \n\ndef tokenize(text):  \n    # Lowercase and split on non-alphanumeric characters  \n    return re.findall(r'\\b[a-z]+\\b', text.lower())  \n\nraw_documents = [  \n    \"Machine learning algorithms learn patterns from data\",  \n    \"Deep learning uses neural networks with many layers\",  \n    \"Natural language processing handles text and speech\",  \n    \"Computer vision processes images and video\",  \n    \"Neural networks are inspired by the human brain\"  \n]  \n\ntokenized_docs = [tokenize(doc) for doc in raw_documents]  \nbm25 = BM25Okapi(tokenized_docs)  \n\nquery_tokens = tokenize(\"neural networks and deep learning\")  \nscores = bm25.get_scores(query_tokens)  \n\nranked = sorted(zip(scores, raw_documents), reverse=True)  \nfor score, doc in ranked:  \n    print(f\"{score:.3f} — {doc}\")\nEnter fullscreen mode\nExit fullscreen mode\nOutput:\n2.312 — Deep learning uses neural networks with many layers  \n1.843 — Neural networks are inspired by the human brain  \n0.000 — Machine learning algorithms learn patterns from data  \n0.000 — Natural language processing handles text and speech  \n0.000 — Computer vision processes images and video\nEnter fullscreen mode\nExit fullscreen mode\nThe same two documents come out on top as in the TF-IDF example from the previous post, which makes sense: the query terms overlap with those documents. The difference is in how the scoring behaves as documents get longer or repeat terms more.\nWhy NLP Fundamentals Pay Off Here\nThe tokenization in the example above is deliberately simple: lowercase and split on whitespace. In a real system, the preprocessing step is where everything we’ve covered earlier in this series starts earning its keep.\nBM25 scores what it receives. The quality of retrieval depends heavily on the quality of the tokens you feed into it. That means the linguistic techniques from earlier posts aren’t abstract exercises. They’re practical inputs to a better search index.\nHere’s how several of them connect directly to BM25:\nTokenization\nis the foundation. Every token becomes a term in the index. Poor tokenization (splitting on whitespace only, failing to handle punctuation, not lowercasing consistently) introduces noise that degrades matching. A query for “run” and a document containing “Run.” may not match if normalisation wasn’t applied.\nLemmatization and stemming\ncollapse inflected forms of a word into a common base. Without them, “run,” “runs,” “running,” and “ran” are four different terms in the index. A query for “running” won’t match a document that only says “ran.” With lemmatization, all four map to “run” and matching improves significantly.\nStop word removal\neliminates high-frequency words that carry little retrieval value: “the,” “a,” “is,” “on.” Removing them reduces index size and prevents these terms from consuming scoring weight that should go to meaningful content words.\nPOS tagging\ndoesn’t improve BM25 directly, but it enables smarter filtering decisions. If you know which tokens are nouns, verbs, and adjectives, you can make deliberate choices: keep only content words, remove function words that stop word lists miss, or apply different normalisation strategies to different parts of speech. Better filtering leads to a cleaner index, and a cleaner index leads to better retrieval scores.\nThe gains compound. A BM25 system built on top of careful tokenization, lemmatization, stop word removal, and POS-informed filtering will consistently outperform one built on raw whitespace splitting, using the same scoring parameters and the same documents.\nMultilingual systems take this further\nLanguage-specific preprocessing becomes especially important when working outside English. Korean, for example, attaches grammatical particles such as 는, 은, 이, and 가 directly to words. Without morphological analysis, a query and a document may refer to exactly the same concept but appear as completely different surface forms. No amount of BM25 parameter tuning fixes a vocabulary mismatch that happens before the scoring function ever runs.\nSimilar considerations exist for Japanese (word segmentation), Chinese (character vs. word tokenization), German (compound word splitting), Arabic (root-based morphology), and many other languages. In each case, the preprocessing layer, informed by the linguistic structure of the language, is what allows BM25 to function well.\nThis is one of the clearest demonstrations that NLP fundamentals continue to matter even when working with relatively simple retrieval algorithms. The preprocessing stack and the scoring function are not separate concerns. They are two parts of the same system, and the quality of one directly determines the ceiling of the other.\nBoth methods score documents based on term frequency and inverse document frequency. The difference is in the details of the TF component.\nFor text classification and feature extraction tasks, TF-IDF is still the standard choice. Its simplicity and compatibility with scikit-learn’s ecosystem make it practical and easy to work with.\nFor search and ranking tasks (where you’re scoring documents against queries and returning a ranked list) BM25 is the better default. Its saturation and length normalisation make it more robust to the kinds of variation you see in real document collections.\nWhere BM25 Fits in Real Systems\nBM25 is not just a theoretical improvement over TF-IDF. It is the default ranking function in Elasticsearch and OpenSearch, two of the most widely deployed search platforms in the world. When a developer sets up a full-text search index without configuring a custom scorer, BM25 is what’s running underneath.\nIt’s also the sparse retrieval component in most hybrid search architectures. A hybrid search system typically runs two retrieval steps in parallel: a BM25 step that matches exact keywords quickly, and a dense neural retrieval step that finds semantically similar documents even when the vocabulary doesn’t overlap. The results from both steps are then combined and re-ranked.\nThis combination exists because BM25 and dense retrieval are genuinely complementary. BM25 is extremely fast and reliable for exact keyword matching. It handles queries with rare, specific terminology (product codes, names, technical terms) very well. Dense retrieval handles conceptual similarity and synonym matching better, but is slower and more resource-intensive. Neither is strictly better than the other. Production systems that care about search quality tend to use both.\nBM25 hybrid search architecture lexical dense retrieval.\nThe practical takeaway: if you are building anything involving search over text, BM25 is the right starting point for the lexical component. It is well-understood, well-supported, fast, and transparent. You can inspect its scores, tune its parameters, and reason about why a particular document ranked where it did. That interpretability is valuable, especially in early development.\nThe Limits of Lexical Retrieval\nBM25 improves on TF-IDF in important ways, but it is still fundamentally a lexical retrieval method. It matches words. It does not understand meaning.\nIf a user searches for “heart attack symptoms” and a document discusses “myocardial infarction warning signs,” BM25 will likely score that document low. The terms don’t overlap. A dense retrieval system trained on medical text would probably find it.\nThis is the core limitation that motivates the move toward semantic and dense retrieval methods. BM25 gets you very far. It is reliable, efficient, and handles the majority of real-world queries well. But for queries that require understanding synonyms, paraphrases, or conceptual relationships, you need something beyond lexical matching.\nThat’s where word embeddings come in. In the next post, we’ll cover how words get represented as dense vectors that capture something closer to meaning, and how that opens the door to a fundamentally different kind of retrieval.\nLearn This at Fondra Labs\nThis post is part of our\nNLP Foundations\nseries, where we build up practical AI knowledge one concept at a time, from text processing basics all the way to the systems powering modern AI.\nAt\nFondra Labs\n, we teach AI from production reality, not hype. Every topic in this series is here because it genuinely matters when you sit down to build something real.\nIf this was useful, explore the rest of the blog. We cover machine learning, deep learning, NLP, and the practical skills that turn understanding into building.\nOriginally published at\nhttps://fondralabs.com\n.",
  "metadata": {
    "title": "BM25 Explained: The Algorithm Behind Modern Search - DEV Community",
    "description": "In the last post, we covered TF-IDF: the foundational technique for turning text into numbers and... Tagged with ai, nlp, machinelearning.",
    "keywords": "ai, nlp, machinelearning, software, coding, development, engineering, inclusive, community",
    "author": null,
    "og:type": "article",
    "og:url": "https://dev.to/javiagu13/bm25-explained-the-algorithm-behind-modern-search-2jo3",
    "og:title": "BM25 Explained: The Algorithm Behind Modern Search",
    "og:description": "In the last post, we covered TF-IDF: the foundational technique for turning text into numbers and...",
    "og:site_name": "DEV Community",
    "og:image": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Foo7nlt7qqk1hgs3tshwd.png",
    "twitter:site": "@thepracticaldev",
    "twitter:creator": "@",
    "twitter:title": "BM25 Explained: The Algorithm Behind Modern Search",
    "twitter:description": "In the last post, we covered TF-IDF: the foundational technique for turning text into numbers and...",
    "twitter:card": "summary_large_image",
    "twitter:widgets:new-embed-design": "on",
    "twitter:image:src": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Foo7nlt7qqk1hgs3tshwd.png"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [
    {
      "file": "attachments/b60f10ab64c6/code/body-000.txt",
      "chars": 146,
      "language": "text"
    },
    {
      "file": "attachments/b60f10ab64c6/code/body-001.txt",
      "chars": 152,
      "language": "text"
    },
    {
      "file": "attachments/b60f10ab64c6/code/body-002.txt",
      "chars": 152,
      "language": "text"
    },
    {
      "file": "attachments/b60f10ab64c6/code/body-003.txt",
      "chars": 22,
      "language": "text"
    },
    {
      "file": "attachments/b60f10ab64c6/code/body-004.txt",
      "chars": 788,
      "language": "text"
    },
    {
      "file": "attachments/b60f10ab64c6/code/body-005.txt",
      "chars": 135,
      "language": "text"
    },
    {
      "file": "attachments/b60f10ab64c6/code/body-006.txt",
      "chars": 43,
      "language": "text"
    },
    {
      "file": "attachments/b60f10ab64c6/code/body-007.txt",
      "chars": 822,
      "language": "text"
    },
    {
      "file": "attachments/b60f10ab64c6/code/body-008.txt",
      "chars": 296,
      "language": "text"
    }
  ],
  "images": [
    {
      "status": "ok",
      "url": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F8qptm0se0oohzsy21zo9.png",
      "bytes": 109004,
      "reused": true,
      "local_path": "attachments/b60f10ab64c6/images/001.png",
      "src": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F8qptm0se0oohzsy21zo9.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fejen8upccodk89xlevc8.png",
      "bytes": 31574,
      "reused": true,
      "local_path": "attachments/b60f10ab64c6/images/002.png",
      "src": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fejen8upccodk89xlevc8.png",
      "alt": "BM25 Formula",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fhippyi0ejoun559wu8f0.png",
      "bytes": 31552,
      "reused": true,
      "local_path": "attachments/b60f10ab64c6/images/003.png",
      "src": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fhippyi0ejoun559wu8f0.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fysy4getk1o7sitoyjs9d.png",
      "bytes": 57932,
      "reused": true,
      "local_path": "attachments/b60f10ab64c6/images/004.png",
      "src": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fysy4getk1o7sitoyjs9d.png",
      "alt": "BM25 term frequency saturation vs TF-IDF linear growth",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fug7atmmfbf67jtwvgz8g.png",
      "bytes": 72080,
      "reused": true,
      "local_path": "attachments/b60f10ab64c6/images/005.png",
      "src": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fug7atmmfbf67jtwvgz8g.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F2qv7nfrxhjbfyx6ytz64.png",
      "bytes": 49138,
      "reused": true,
      "local_path": "attachments/b60f10ab64c6/images/006.png",
      "src": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F2qv7nfrxhjbfyx6ytz64.png",
      "alt": "BM25 hybrid search architecture lexical dense retrieval.",
      "downloaded": true
    }
  ],
  "audio": [],
  "videos": [],
  "links": [
    {
      "href": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F8qptm0se0oohzsy21zo9.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fejen8upccodk89xlevc8.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fhippyi0ejoun559wu8f0.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fysy4getk1o7sitoyjs9d.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fug7atmmfbf67jtwvgz8g.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F2qv7nfrxhjbfyx6ytz64.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://fondralabs.com/nlp-foundations.html",
      "label": "NLP Foundations",
      "type": "a"
    },
    {
      "href": "https://fondralabs.com/",
      "label": "Fondra Labs",
      "type": "a"
    },
    {
      "href": "https://fondralabs.com/blog/nlp-foundations/bm25-explained-the-algorithm-behind-modern-search.html",
      "label": "https://fondralabs.com",
      "type": "a"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:48:54Z",
  "source_url": "https://dev.to/javiagu13/bm25-explained-the-algorithm-behind-modern-search-2jo3",
  "papers": [],
  "actual_video_files": [],
  "body_chars": 19635,
  "expected_body_images": 6,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/b60f10ab64c6/pages/source.html",
  "local_html": "attachments/b60f10ab64c6/pages/article.html",
  "body_base_path": "attachments/b60f10ab64c6/pages/",
  "body_markdown_path": "attachments/b60f10ab64c6/pages/article.md"
}