{
  "source_file": "0930",
  "source_platform": "Hashnode",
  "author": "Muhammad Fahad Bashir",
  "title_listed": "6. Understanding Retrievers in detail -LangChain Retriever Methods",
  "keywords": "",
  "domain": "大模型算法",
  "content_type": "深度技术解析",
  "is_paid_flag": "否",
  "url": "https://mfahadbashir.hashnode.dev/6-understanding-retrievers-in-detail-langchain-retriever-methods",
  "publish_date": "2024-09-26",
  "corpus_stage": "",
  "notes": "",
  "id": "88ca38158907",
  "collected_at": "2026-10-02T14:10:58",
  "title": "6. Understanding Retrievers in detail -LangChain Retriever Methods",
  "body_markdown": "6. Understanding Retrievers in detail -LangChain Retriever Methods\nPublished\nSeptember 26, 2024\n•\n7\nmin read\n•\nView as Markdown\nM\nMuhammad Fahad Bashir\nMachine Learning, Artificial Intelligence\nPart of series\nImplementing RAG systems from Scratch in-depth\nThis is the 6th article in my\nRetrieval-Augmented Generation (RAG) series\n. In previous articles, we explored what RAG is and how to create a vector store. Now, we’ll cover the next critical step—\nretrieving data from vector stores\n.\nThis is a core component of RAG because\nproper retrieval of data\nensures that the Large Language Model (LLM) can generate accurate and contextually relevant outputs. Without effective retrieval, even the most sophisticated models will struggle to provide useful responses.\nWhat is a Retriever, and Why is It Important?\nRetrievers\nare responsible for fetching relevant information from your indexed documents based on a query. In LangChain, a\nretriever\nis an interface that returns documents from a data store (like a vector store) based on an unstructured query.\nUnlike a vector store, the primary role of a retriever is to\nreturn\ndocuments, not store them. While vector stores store documents as vectors,\nretrievers search through those vectors\nto identify the most relevant information based on the similarity to the query.\nIn simpler terms:\nVector Store\n: Stores data as vectors (numerical representations).\nRetriever\n: Finds the most similar vectors (data) to a query and returns relevant information.\nHow Does Data Retrieval Work?\nIn a vector store, data is stored in the form of\nembeddings\n(vectors). When we submit a query, it is\nconverted into a vector\n. The retriever then performs a\nsimilarity search\nto identify which stored vectors are most similar to the query. This similarity is typically measured using mathematical techniques like\ndot product\nor\ncosine similarity\n.\nSince we cannot interact with raw vectors (numerical representations),\nretrievers\nplay a crucial role in enabling us to interact with our data in a meaningful way.\nTypes of Retrievers in LangChain\nLangChain provides multiple types of retrievers, each suited for different use cases. Some of the most common retrievers include:\nVector Store Retriever\n: A lightweight wrapper around the vector store. It transforms the vector store into a retriever by enabling it to conform to the retrieval interface.\nParent Document Retriever\n: Instead of returning individual chunks, this retriever finds chunks that are most similar in embedding space but retrieves the entire parent document.\nMulti-Vector Retriever\n: Uses multiple vectors to represent the document for richer retrieval results.\nSelf-Query Retriever\n: Automatically generates a query to retrieve the most relevant documents.\nContextual Compression Retriever\n: Retrieves data while compressing the context based on the query.\nThe following\nblog post\nwill dive deep into retrieval and cover more advanced methods for overcoming those edge cases.\nRetrieval Techniques\nWhen retrieving data from vector stores, different techniques are applied depending on the requirements:\nSemantic Search (Cosine Similarity)\n: This method retrieves data based on how semantically close the query is to the stored documents. It helps bring back all the\nrelevant data\n.\nMaximum Marginal Relevance (MMR)\n: Unlike semantic search, MMR ensures\ndiversity\nby reducing repetition. It retrieves the most\ndiverse data\n, rather than just the most similar. This is useful when you want to avoid redundancy in results.\nOther retrieval methods include:\nSupport Vector Machine (SVM)\nTF/IDF (Term Frequency-Inverse Document Frequency)\nAdvanced Retrieval Strategies\nRetrieval can get quite complex depending on your requirements. Here are a few advanced strategies:\nHard Rules\n: You can enforce specific conditions, such as only retrieving documents from a certain timeframe or related to specific topics.\nDocument Linking\n: Retrieving documents that are linked to the retrieved context (e.g., via document taxonomy).\nMultiple Embeddings\n: Generating and using multiple embeddings for each document to ensure richer retrieval results.\nEnsemble of Retrievers\n: Combining multiple retrievers to enhance the quality of results.\nWeighted Retrieval\n: Assigning higher weights to more recent or relevant documents, allowing you to fine-tune the retrieval process.\nHow to Choose a Retrieval Method?\nChoosing the right retrieval method depends on the type of response you're interested in and the nature of your data:\nMMR\n: Useful when you want to avoid repetition and prioritize diversity in results.\nCosine Similarity (Semantic Search)\n: Best when you want to gather all relevant information, without concern for diversity.\nFilters\n: Add specificity to your retrieval by applying filters like “documents published after 2020” or “documents related to Pakistan.”\nI. What is Semantic Search?\nSemantic Search\nis a technique used in information retrieval to find documents or text passages that are\nsemantically similar\nto the query. Rather than focusing on exact keyword matches, semantic search tries to understand the\nmeaning\nbehind the query and the documents.\nThis approach leverages\nembeddings\n—numerical vectors that represent the meaning of text. When a query is made, it is converted into an embedding, and the retriever then compares this query embedding with the embeddings of the documents in the vector store.\nCosine Similarity\nis often used to compare these embeddings. It measures how close the two vectors are, with higher cosine similarity indicating that the query and document share similar meanings.\nIn summary:\nSemantic Search\nlooks for\nmeaning\nrather than exact words.\nIt uses techniques like\ncosine similarity\nto find documents that share similar semantic concepts.\nExample of Semantic Search\nIf your query is \"What's the capital of Pakistan?\", a semantic search system could retrieve documents that contain related information like \"Islamabad is the capital of Pakistan,\" even if the exact wording differs.\nThis flexibility makes semantic search a powerful tool for retrieving relevant information, even when the phrasing of the query doesn't exactly match the stored documents.\nii. What is Maximum Marginal Relevance (MMR)?\nMaximum Marginal Relevance (MMR)\nis a retrieval technique used to ensure that the results returned are not only relevant but also\ndiverse\n. When querying a vector store, especially when retrieving multiple documents, you might face the problem of\nredundant responses\n. For instance, all retrieved documents could be highly similar, making them redundant in providing additional value to the LLM.\nMMR tackles this issue by balancing two factors:\nRelevance\n: Ensuring the returned documents are highly similar to the query.\nDiversity\n: Introducing variation in the responses, so that you avoid retrieving documents that are too similar to each other.\nThis is particularly useful when you want to avoid\nrepetition\nand ensure that the results provide a\nbroader spectrum of information\n.\nretriever = vector_store.as_retriever(search_type=\n\"mmr\"\n)\ndocs = retriever.get_relevant_documents(query)\nMMR is ideal when you need to balance both\nsimilarity\nand\nuniqueness\nin the retrieved results.\nMaking Decisions and implementing\nFrom different types of retrievers in langchain, we will be using\nVector Store as a Retriever\nbecause this is simple and easy to understand.\nHow to Use a Vector Store as a Retriever\nIn LangChain, using a\nvector store as a retriever\nis straightforward. Essentially, the vector store acts as the backend for the retriever, which performs similarity searches and returns relevant documents. Here’s a basic process:\nStep 1: Create or Load a Vector Store\n– You need to first create or load a vector store containing document embeddings (e.g., using Chroma, Pinecone, or Qdrant as vector stores).\nStep 2: Wrap the Vector Store in a Retriever\n– LangChain offers a\nVectorStoreRetriever\n, which acts as a lightweight wrapper around the vector store. This allows you to search and retrieve documents based on vector similarity.\nretriever = vector_store.as_retriever()\nStep 3: Query the Vector Store\n– Once the vector store is set up as a retriever, you can pass a query, which is transformed into a vector and matched with the stored embeddings.\nretriever.get_relevant_documents(query)\nThis process allows the\nvector store\nto perform its primary function—storing and indexing data—while the\nretriever\nfocuses on fetching the relevant data based on similarity.\nPractical Example\nin the following, we have covered from start loading to performing same query on two techniques\n1. using similarity search\nquery =\n\"What is PhysioFlex?\"\nk=\n4\n#number of result we want\nretrieved_docs = vector_store.similarity_search(query,k=k)\nfor\ndoc\nin\nretrieved_docs:\n    print(doc.page_content)\nusing MMR\nk =\n4\n# Set the number of top results to return\nlambda_mult =\n0.5\n# Balances between relevance and diversity (between 0 and 1)\nquery =\n\"What is PhysioFlex?\"\nretrieved_docs_mmr = vector_store.max_marginal_relevance_search(query, k=k, lambda_mult=lambda_mult)\n# Output the retrieved documents\nfor\ndoc\nin\nretrieved_docs_mmr:\n    print(doc.page_content)\nTo view the complete\ncheck the notebook\n. Copy and play around\nConclusion\nRetrievers are a key element in RAG systems. Whether you're retrieving relevant documents based on a semantic query or ensuring diversity with techniques like MMR, the retriever plays an essential role in ensuring that your LLM generates contextually accurate and relevant outputs. By choosing the right retrieval technique and applying filters effectively, you can maximize the performance of your RAG system.\n#\nrag\n#\nllm\n#\ntutorial\n#\nstep-by-step\n#\nseries\n#\nlangchain\n#\nchromadb\n#\nvector-embeddings\n#\nvector-similarity\n#\nretrieval-augmented-generation\n200\nviews",
  "metadata": {
    "title": "6. Understanding Retrievers in detail -LangChain Retriever Methods",
    "description": "Discover the role of retrievers in RAG, their types, retrieval methods, and using vector stores as retrievers in LangChain",
    "keywords": null,
    "author": null,
    "og:title": "6. Understanding Retrievers in detail -LangChain Retriever Methods",
    "og:description": "Discover the role of retrievers in RAG, their types, retrieval methods, and using vector stores as retrievers in LangChain",
    "og:url": "https://mfahadbashir.hashnode.dev/6-understanding-retrievers-in-detail-langchain-retriever-methods",
    "og:site_name": "Muhammad Fahad Bashir",
    "og:image": "https://cdn.hashnode.com/res/hashnode/image/upload/v1727324148342/496cfee8-e8e2-44e2-8cb7-52d085b0028f.png",
    "og:image:width": "1200",
    "og:image:height": "630",
    "og:type": "article",
    "twitter:card": "summary_large_image",
    "twitter:title": "6. Understanding Retrievers in detail -LangChain Retriever Methods",
    "twitter:description": "Discover the role of retrievers in RAG, their types, retrieval methods, and using vector stores as retrievers in LangChain",
    "twitter:image": "https://cdn.hashnode.com/res/hashnode/image/upload/v1727324148342/496cfee8-e8e2-44e2-8cb7-52d085b0028f.png",
    "article:published_time": "2024-09-26T05:05:04.584Z",
    "article:author": "https://mfahadbashir.hashnode.dev/@mfahadbashir"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [
    {
      "file": "attachments/88ca38158907/code/body-000.txt",
      "chars": 104,
      "language": "text"
    },
    {
      "file": "attachments/88ca38158907/code/body-001.txt",
      "chars": 41,
      "language": "text"
    },
    {
      "file": "attachments/88ca38158907/code/body-002.txt",
      "chars": 41,
      "language": "text"
    },
    {
      "file": "attachments/88ca38158907/code/body-003.txt",
      "chars": 174,
      "language": "text"
    },
    {
      "file": "attachments/88ca38158907/code/body-004.txt",
      "chars": 354,
      "language": "text"
    }
  ],
  "images": [
    {
      "status": "ok",
      "url": "https://cdn.hashnode.com/res/hashnode/image/upload/v1727324148342/496cfee8-e8e2-44e2-8cb7-52d085b0028f.png",
      "bytes": 157504,
      "reused": true,
      "local_path": "attachments/88ca38158907/images/000.png",
      "src": "https://cdn.hashnode.com/res/hashnode/image/upload/v1727324148342/496cfee8-e8e2-44e2-8cb7-52d085b0028f.png",
      "alt": "6. Understanding Retrievers in detail -LangChain Retriever Methods",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://miro.medium.com/v2/resize:fit:1400/1*-sVJnwUyoqPrXMvscJAL9w.jpeg",
      "bytes": 56982,
      "reused": true,
      "local_path": "attachments/88ca38158907/images/002.jpeg",
      "src": "https://miro.medium.com/v2/resize:fit:1400/1*-sVJnwUyoqPrXMvscJAL9w.jpeg",
      "alt": "Retrieval-Augmented Generation (RAG) using LangChain, LlamaIndex, and  OpenAI | by Prasad Mahamulkar | Medium",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://blog.dataiku.com/hs-fs/hubfs/dftt%202.webp?width=673&height=316&name=dftt%202.webp",
      "bytes": 23737,
      "reused": true,
      "local_path": "attachments/88ca38158907/images/003.webp",
      "src": "https://blog.dataiku.com/hs-fs/hubfs/dftt%202.webp?width=673&height=316&name=dftt%202.webp",
      "alt": "Semantic Search: An Overlooked NLP Superpower",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9d859f3c-bc35-40e3-b06c-354a7e200bcb_751x437.png",
      "bytes": 19854,
      "reused": true,
      "local_path": "attachments/88ca38158907/images/004.png",
      "src": "https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9d859f3c-bc35-40e3-b06c-354a7e200bcb_751x437.png",
      "alt": "Hands-On LangChain for LLM Applications Development: Information Retrieval",
      "downloaded": true
    }
  ],
  "audio": [],
  "videos": [],
  "links": [
    {
      "href": "https://mfahadbashir.hashnode.dev/6-understanding-retrievers-in-detail-langchain-retriever-methods.md",
      "label": "View as Markdown",
      "type": "a"
    },
    {
      "href": "https://hashnode.com/@mfahadbashir",
      "label": "M",
      "type": "a"
    },
    {
      "href": "https://www.linkedin.com/in/muhammad-fahad-bashir-09568216a/",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://mfahadbashir.hashnode.dev/series/rag-series",
      "label": "Part of series Implementing RAG systems from Scratch in-depth",
      "type": "a"
    },
    {
      "href": "https://dev.to/rutamstwt/langchain-retrieval-ida",
      "label": "blog post",
      "type": "a"
    },
    {
      "href": "https://colab.research.google.com/drive/1sjKZVxZpzWtPM3IQ03fdzughgqWpVu73?usp=sharing",
      "label": "check the notebook",
      "type": "a"
    },
    {
      "href": "https://mfahadbashir.hashnode.dev/tag/rag",
      "label": "# rag",
      "type": "a"
    },
    {
      "href": "https://mfahadbashir.hashnode.dev/tag/llm",
      "label": "# llm",
      "type": "a"
    },
    {
      "href": "https://mfahadbashir.hashnode.dev/tag/tutorial",
      "label": "# tutorial",
      "type": "a"
    },
    {
      "href": "https://mfahadbashir.hashnode.dev/tag/step-by-step",
      "label": "# step-by-step",
      "type": "a"
    },
    {
      "href": "https://mfahadbashir.hashnode.dev/tag/series",
      "label": "# series",
      "type": "a"
    },
    {
      "href": "https://mfahadbashir.hashnode.dev/tag/langchain",
      "label": "# langchain",
      "type": "a"
    },
    {
      "href": "https://mfahadbashir.hashnode.dev/tag/chromadb",
      "label": "# chromadb",
      "type": "a"
    },
    {
      "href": "https://mfahadbashir.hashnode.dev/tag/vector-embeddings",
      "label": "# vector-embeddings",
      "type": "a"
    },
    {
      "href": "https://mfahadbashir.hashnode.dev/tag/vector-similarity",
      "label": "# vector-similarity",
      "type": "a"
    },
    {
      "href": "https://mfahadbashir.hashnode.dev/tag/retrieval-augmented-generation",
      "label": "# retrieval-augmented-generation",
      "type": "a"
    }
  ],
  "retry_at": "2026-10-03T08:08:28",
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:50:45Z",
  "source_url": "https://mfahadbashir.hashnode.dev/6-understanding-retrievers-in-detail-langchain-retriever-methods",
  "papers": [],
  "actual_video_files": [],
  "body_chars": 9811,
  "expected_body_images": 4,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/88ca38158907/pages/source.html",
  "local_html": "attachments/88ca38158907/pages/article.html",
  "body_base_path": "attachments/88ca38158907/pages/",
  "body_markdown_path": "attachments/88ca38158907/pages/article.md"
}