![Revisit consent button](../images/body-df29beacd7496220cc94.svg) We value your privacy We use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking "Accept All", you consent to our use of cookies. Customise Reject All Accept All Customise Consent Preferences ![](../images/body-2159bb67f3de8ad870eb.svg) We use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below. The cookies that are categorised as "Necessary" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... Show more NecessaryAlways Active Necessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data. * Cookie BCTempID * Duration 10 minutes * Description No description available. * Cookie __cf_bm * Duration 1 hour * Description This cookie, set by Cloudflare, is used to support Cloudflare Bot Management. * Cookie AWSALBCORS * Duration 7 days * Description Amazon Web Services set this cookie for load balancing. * Cookie _cfuvid * Duration session * Description Cloudflare sets this cookie to track users across sessions to optimize user experience by maintaining session consistency and providing personalized services * Cookie li_gc * Duration 6 months * Description Linkedin set this cookie for storing visitor's consent regarding using cookies for non-essential purposes. * Cookie __hssrc * Duration session * Description This cookie is set by Hubspot whenever it changes the session cookie. The __hssrc cookie set to 1 indicates that the user has restarted the browser, and if the cookie does not exist, it is assumed to be a new session. * Cookie __hssc * Duration 1 hour * Description HubSpot sets this cookie to keep track of sessions and to determine if HubSpot should increment the session number and timestamps in the __hstc cookie. * Cookie wpEmojiSettingsSupports * Duration session * Description WordPress sets this cookie when a user interacts with emojis on a WordPress site. It helps determine if the user's browser can display emojis properly. * Cookie BCSessionID * Duration 1 year 1 month 4 days * Description Blueconic sets this cookie as a unique identifier for the BlueConic profile. * Cookie _octo * Duration 1 year * Description No description available. * Cookie logged_in * Duration 1 year * Description No description available. * Cookie __Secure-YEC * Duration past * Description YouTube sets this cookie to stores the user's video player preferences using embedded YouTube video * Cookie __eoi * Duration 6 months * Description Description is currently not available. * Cookie AWSALBTGCORS * Duration 7 days * Description No description available. * Cookie login-status-p * Duration past * Description Description is currently not available. * Cookie AWSALBTG * Duration 7 days * Description No description available. * Cookie csrf_token * Duration session * Description No description available. * Cookie token_v2 * Duration 1 day * Description Description is currently not available. * Cookie D * Duration 1 year * Description Description is currently not available. * Cookie PHPSESSID * Duration session * Description This cookie is native to PHP applications. The cookie stores and identifies a user's unique session ID to manage user sessions on the website. The cookie is a session cookie and will be deleted when all the browser windows are closed. * Cookie VISITOR_PRIVACY_METADATA * Duration 6 months * Description YouTube sets this cookie to store the user's cookie consent state for the current domain. * Cookie cookietest * Duration session * Description The cookietest cookie is typically used to determine whether the user's browser accepts cookies, essential for website functionality and user experience. * Cookie __Host-airtable-session * Duration 1 year * Description This cookie is used to enable us to integrate the services of Airtable. * Cookie __Host-airtable-session.sig * Duration 1 year * Description This cookie is used to enable us to integrate the services of Airtable. * Cookie m * Duration 1 year 1 month 4 days * Description Stripe sets this cookie for fraud prevention purposes. It identifies the device used to access the website, allowing the website to be formatted accordingly. * Cookie BIGipServer* * Duration session * Description Marketo sets this cookie to collect information about the user's online activity and build a profile about their interests to provide advertisements relevant to the user. * Cookie __cfruid * Duration session * Description Cloudflare sets this cookie to identify trusted web traffic. * Cookie _GRECAPTCHA * Duration 6 months * Description Google Recaptcha service sets this cookie to identify bots to protect the website against malicious spam attacks. * Cookie __Secure-YNID * Duration 6 months * Description Google cookie used to protect user security and prevent fraud, especially during the login process. * Cookie cookieyes-consent * Duration 1 year * Description CookieYes sets this cookie to remember users' consent preferences so that their preferences are respected on subsequent visits to this site. It does not collect or store any personal information about the site visitors. Functional Functional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features. * Cookie lidc * Duration 1 day * Description LinkedIn sets the lidc cookie to facilitate data center selection. * Cookie brw * Duration 1 year * Description No description available. * Cookie brwConsent * Duration 5 minutes * Description Description is currently not available. * Cookie WMF-Uniq * Duration 1 year * Description Description is currently not available. * Cookie loom_anon_comment * Duration 1 year * Description No description available. * Cookie loom_referral_video * Duration session * Description Description is currently not available. * Cookie VISITOR_INFO1_LIVE * Duration 6 months * Description A cookie set by YouTube to measure bandwidth that determines whether the user gets the new or old player interface. * Cookie yt-remote-connected-devices * Duration Never Expires * Description YouTube sets this cookie to store the user's video preferences using embedded YouTube videos. * Cookie ytidb::LAST_RESULT_ENTRY_KEY * Duration Never Expires * Description The cookie ytidb::LAST_RESULT_ENTRY_KEY is used by YouTube to store the last search result entry that was clicked by the user. This information is used to improve the user experience by providing more relevant search results in the future. * Cookie yt-remote-device-id * Duration Never Expires * Description YouTube sets this cookie to store the user's video preferences using embedded YouTube videos. * Cookie yt-remote-session-name * Duration session * Description The yt-remote-session-name cookie is used by YouTube to store the user's video player preferences using embedded YouTube video. * Cookie yt-remote-fast-check-period * Duration session * Description The yt-remote-fast-check-period cookie is used by YouTube to store the user's video player preferences for embedded YouTube videos. * Cookie yt-remote-session-app * Duration session * Description The yt-remote-session-app cookie is used by YouTube to store user preferences and information about the interface of the embedded YouTube video player. * Cookie yt-remote-cast-available * Duration session * Description The yt-remote-cast-available cookie is used to store the user's preferences regarding whether casting is available on their YouTube video player. * Cookie yt-remote-cast-installed * Duration session * Description The yt-remote-cast-installed cookie is used to store the user's video player preferences using embedded YouTube video. * Cookie cp_session * Duration 3 months * Description Codepen sets this cookie for Help systems found in the website. * Cookie loid * Duration 1 year 1 month 4 days * Description This cookie is set by the Reddit. The cookie enables the sharing of content from the website onto the social media platform. Analytics Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc. * Cookie __hstc * Duration 6 months * Description Hubspot set this main cookie for tracking visitors. It contains the domain, initial timestamp (first visit), last timestamp (last visit), current timestamp (this visit), and session number (increments for each subsequent session). * Cookie hubspotutk * Duration 6 months * Description HubSpot sets this cookie to keep track of the visitors to the website. This cookie is passed to HubSpot on form submission and used when deduplicating contacts. * Cookie _ga * Duration 1 year 1 month 4 days * Description Google Analytics sets this cookie to calculate visitor, session and campaign data and track site usage for the site's analytics report. The cookie stores information anonymously and assigns a randomly generated number to recognise unique visitors. * Cookie _ga_* * Duration 1 year 1 month 4 days * Description Google Analytics sets this cookie to store and count page views. * Cookie __Host-psifi.analyticsTrace * Duration 6 hours * Description Description is currently not available. * Cookie __Host-psifi.analyticsTraceV2 * Duration 6 hours * Description Description is currently not available. * Cookie _gh_sess * Duration session * Description GitHub sets this cookie for temporary application and framework state between pages like what step the user is on in a multiple step form. * Cookie YSC * Duration session * Description YSC cookie is set by Youtube and is used to track the views of embedded videos on Youtube pages. * Cookie ajs_anonymous_id * Duration 1 year * Description This cookie is set by Segment to count the number of people who visit a certain site by tracking if they have visited before. * Cookie vuid * Duration 1 year 1 month 4 days * Description Vimeo installs this cookie to collect tracking information by setting a unique ID to embed videos on the website. Performance Performance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors. * Cookie AWSALB * Duration 7 days * Description AWSALB is an application load balancer cookie set by Amazon Web Services to map the session to the target. * Cookie acq * Duration past * Description Description is currently not available. * Cookie acq.sig * Duration past * Description Description is currently not available. * Cookie ptc * Duration 2 years * Description No description available. Advertisement Advertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns. * Cookie muc_ads * Duration 1 year 1 month 4 days * Description Twitter sets this cookie to collect user behaviour and interaction data to optimize the website. * Cookie guest_id_marketing * Duration 1 year 1 month 4 days * Description Twitter sets this cookie to identify and track the website visitor. * Cookie guest_id_ads * Duration 1 year 1 month 4 days * Description Twitter sets this cookie to identify and track the website visitor. * Cookie personalization_id * Duration 1 year 1 month 4 days * Description Twitter sets this cookie to integrate and share features for social media and also store information about how the user uses the website, for tracking and targeting. * Cookie guest_id * Duration 1 year 1 month 4 days * Description Twitter sets this cookie to identify and track the website visitor. It registers if a user is signed in to the Twitter platform and collects information about ad preferences. * Cookie bcookie * Duration 1 year * Description LinkedIn sets this cookie from LinkedIn share buttons and ad tags to recognize browser IDs. * Cookie __Secure-ROLLOUT_TOKEN * Duration 6 months * Description YouTube sets this cookie to manage feature rollout and experimentation. It helps Google control which new features or interface changes are shown to users as part of testing and staged rollouts, ensuring consistent experience for a given user during an experiment. * Cookie yt.innertube::nextId * Duration Never Expires * Description YouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen. * Cookie yt.innertube::requests * Duration Never Expires * Description YouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen. * Cookie session_tracker * Duration session * Description This cookie is set by the Reddit. This cookie is used to identify trusted web traffic. It also helps in adverstising on the website. * Cookie edgebucket * Duration session * Description Reddit sets this cookie to save the information about a log-on Reddit user, for the purpose of advertisement recommendations and updating the content. * Cookie did * Duration 1 year * Description Arbor sets this cookie to show targeted ads to site visitors.This cookie expires after 2 months or 1 year. Uncategorised Other uncategorised cookies are those that are being analysed and have not been classified into a category as yet. No cookies to display. Reject All Save My Preferences Accept All Toggle Search[](https://towardsdatascience.com/)Toggle Mobile Navigation [](https://towardsdatascience.com/) Publish AI, ML & data-science insights to a global community of data professionals. Sign in[Submit an Article](https://write.insightmediagroup.io) [Latest](https://towardsdatascience.com/latest/)[Editor's Picks](https://towardsdatascience.com/tag/editors-pick/)[Newsletter](https://towardsdatascience.com/tag/the-variable/)[ShipAI](https://towardsdatascience.com/shipai/) * * * [Write For TDS](https://towardsdatascience.com/submissions/) [LinkedIn](https://www.linkedin.com/company/towards-data-science/)[X](https://x.com/TDataScience)Toggle Search [NLP](https://towardsdatascience.com/category/artificial-intelligence/nlp/) # The Ultimate Guide to RAGs - Each Component Dissected A visual tour of what it takes to build CHAD-level LLM pipelines [Avishek Biswas](https://towardsdatascience.com/author/neural.avb/) October 29, 202412 min read ### A visual tour of what it takes to build production-ready LLM pipelines ![Let's learn RAGs! \(Image by Author\)](../images/000.jpeg) Let's learn RAGs! (Image by Author) If you have worked with Large Language Models, there is a great chance that you have at least heard the term RAG - Retrieval Augmented Generation. The idea of RAGs are pretty simple - suppose you want to ask a question to a LLM, instead of just relying on the LLM's pre-trained knowledge, you first retrieve relevant information from an external knowledge base. This retrieved information is then provided to the LLM along with the question, allowing it to generate a more informed and up-to-date response. ![Comparing standard LLM calls with RAG \(Source: Image by Author\)](../images/001.jpeg) Comparing standard LLM calls with RAG (Source: Image by Author) **So, why use Retrieval Augmented Generation?** When providing **accurate and up-to-date information** is key, you cannot rely on the LLM's inbuilt knowledge. RAGs are a cheap practical way to use LLMs to generate content about recent topics or niche topics **without needing to finetune** them on your own and burn away your life's savings. Even when LLMs internal knowledge may be enough to answer questions, it might be a good idea to use RAGs anyway, [since recent studies have shown that they could help reduce LLMs hallucinations.](https://arxiv.org/html/2404.08189v1) ## The different components of a bare-bones RAG Before we dive into the advanced portion of this article, let's review the basics. Generally RAGs consist of two pipelines - **preprocessing and inferencing**. **Inferencing** is all about using data from your existing database to answer questions from a user query. **Preprocessing** is the process of setting up the database in the correct way so that retrieval is done correctly later on. Here is a diagramatic look into the entire basic barebones RAG pipeline. ![The Basic RAG pipeline \(Image by Author\)](../images/002.png) The Basic RAG pipeline (Image by Author) ### The Indexing or Preprocessing Steps This is the offline preprocessing stage, where we would set up our database. 1. **Identify Data Source** : Choose a relevant data source based on the application, such as Wikipedia, books, or manuals. Since this is domain dependent, I am going to skip over this step in this article. Go choose any data you want to use, knock yourself out! 2. **Chunking the Data** : Break down the dataset into smaller, manageable documents or chunks. 3. **Convert to Searchable Format** : Transform each chunk into a numerical vector or similar searchable representation. 4. **Insert into Database** : Store these searchable chunks in a custom database, though external databases or search engines could also be used. ### The Inferencing Steps During the Query Inferencing stage, the following components stand out. 1. **Query Processing** : A method to convert the user's query into a format suitable for search. 2. **Retrieval/Search Strategy** : A similarity search mechanism to retrieve the most relevant documents. 3. **Post-Retrieval Answer Generation** : Use retrieved documents as context to generate the answer with an LLM. Great - so we identified several key modules required to build a RAG. Believe it or not, each of these components have a lot of additional research to make this simple RAG turn into CHAD-rag. Let's look into each of the major components in this list, starting with chunking. Please accept cookies to access this content _**By the way, this article is based on this 17-minute Youtube video I made on the same topic, covering all the topics in this article. Feel free to check it out after reading this Medium article!**_ * * * ## 1. Chunking Chunking is the process of breaking down large documents into smaller, manageable pieces. It might sound simple, but trust me, the way you chunk your data can make or break your RAG pipeline. Whatever chunks you create during preprocessing will eventually get retrieved during inference. If you make the size of chunks too small - like each sentence - then it might be difficult to retrieve them through search because they capture very little information. If the chunk size is too big - like inserting entire Wikipedia articles - the retrieved passages might end up confusing the LLM because you are sending large bodies of texts at once. ![Depending on the different levels of chunking, your results may vary! \(Image by author\)](../images/003.jpeg) Depending on the different levels of chunking, your results may vary! (Image by author) Some frameworks use LLMs to do chunking, for example by extracting simple factoids or propositions from the text corpus, and treat them as documents. This could be expensive because the larger your dataset, the more LLM calls you'll have to make. ![Proposition Chunking \(Source: Dense X Retrieval Paper. License: Free\)](../images/004.png) Proposition Chunking (Source: [Dense X Retrieval Paper](https://arxiv.org/pdf/2312.06648). License: Free) ### Structural Chunking ![If your data has inherent boundaries \(like HTML or Code\), sometimes it is best to just utilize it. \(Image by Author\)](../images/005.jpeg) If your data has inherent boundaries (like HTML or Code), sometimes it is best to just utilize it. (Image by Author) Quite often we may also deal with datasets that inherently have a known structure or format. For example, if you want to insert code into your database, you can simply split each script by the function names or class definitions. For HTML pages like Wikipedia articles, you can split by the heading tags - for example, split by the H2 tags to isolate each sub-chapter. ### Contextual Chunking But there are some glaring issues with the types of chunking we have discussed so far. Suppose your dataset consists of tens of thousands of paragraphs extracted from all Sherlock Holmes books. Now the user has queried something general like what was the first crime in Study in Scarlet? What do you think is going to happen? The problem is that since each documented is an isolated piece of information, we don't know which chunks are from the book Study in Scarlet. Therefore, later on during retrieval, we will end up fetch a bunch of passages about the topic "crime" without knowing if it's relevant to the book. To resolve this, we can use something known as contextual chunking. **Enter Contextual Chunking** [A recent blogpost](https://anthropic.com/news/contextual-retrieval) from Anthropic describes it as prepending chunk-specific explanatory context to each chunk before embedding. Basically, while we are indexing, we would also include additional information relevant to the chunk - like the name of the book, the chapter, maybe a summary of the events in the book. Adding this context will allow the retriever to find references to Study in Scarlett and crimes when searching, hopefully getting the right documents from the database! ![Contextual Chunking adds additional information to the chunks than just the text body \(Image by author\)](../images/006.jpeg) Contextual Chunking adds additional information to the chunks than just the text body (Image by author) There are other ways to solve the problem of finding the right queries - like metadata filtering, We will talk about this later when we talk about Databases. * * * ## 2. Data Conversion Next, we come to the data-conversion stage. Note that whatever strategy we used to convert the documents during preprocessing, we need to use it to search for similarity later, so these two components are tightly coupled. Two of the most common approaches that have emerged in this space are **embedding based methods** and **keyword-frequency based methods** like TF-IDF or BM-25. ### Embedding Based Methods We'll start with embedding-based methods. Here, we use pretrained transformer models to transform the text into high-dimensional vector representations, capturing semantic meaning about the text. Embeddings are great for capturing semantic relationships, handling synonyms, and understanding context-dependent meanings. However, embedding can be computationally intensive, and can sometimes overlook exact matches that simpler methods would easily catch. ![Embeddings \(Image by Author\)](../images/007.jpeg) Embeddings (Image by Author) ### When does Semantic Search fail? For example, suppose you have a database of manuals containing information about specific refrigerators. When you ask a query mentioning a very specific niche model or a serial number, embeddings will fetch documents that kind of resemble your query, but may fail to exactly match it. This brings us to the alternative of embeddings retrieval - keyword based retrieval. ### Keyword Based Methods Two popular keyword-based methods are TF-IDF and BM25. These algorithms focus on statistical relationships between terms in documents and queries. TF-IDF weighs the importance of a word based on its frequency in a document relative to its frequency in the entire corpus. Every document in our dataset is be represented by a array of TF-IDF scores for each word in the vocabulary. The indices of the high values in this document vector tell us which words that are likely to be most characteristic of that document's content, because these words appear more frequently in this document and less frequently in others. For example, the documents related to this Godrej A241gX , will have a high TF-IDF score for the phrase Godrej and A241gX, making it more likely for us to retrieve this using TF-IDF. ![TF-IDF relies on the ratio of the occurence of terms in a document compared to the entire corpus. \(Image by author\)](../images/008.jpeg) TF-IDF relies on the ratio of the occurence of terms in a document compared to the entire corpus. (Image by author) BM25, an evolution of TF-IDF, incorporates document length normalization and term saturation. Meaning that it adjusts the TF-IDF score based on if the document itself is longer or shorter than the average document length in the collection. Term saturation means that as a particular word appears too often in the database, it's importance decreases. TF-IDF and BM-25 are great finding documents with specific keyword occurrences when they exactly occur. And embeddings are great for finding documents with similar semantic meaning. **A common thing these days is to retrieve using both keyword and embedding based methods, and combine them, giving us the best of both worlds.** Later on when we discuss Reciprocal Rank Fusion and Deduplication, we will look into how to combine these different retrieval methods. * * * ## 3. Databases Up next, let's talk about Databases. The most common type of database that is used in RAGs are **Vector Databases**. Vector databases store documents by indexing them with their vector representation, be in from an embedding, or TF-IDF. Vector databases specialize in fast similarity check with query vectors, making them ideal for RAG. Popular vector databases that you may want to look into are Pinecone, Milvus, ChromaDB, MongoDB, and they all have their pros and cons and pricing model. An alternative to vector databases are **graph databases**. Graph databases store information as a network of documents with each document connected to others through relationships. ![Modern Vector Databases allow attribute filtering with semantic search \(Image by Author\)](../images/009.jpeg) Modern Vector Databases allow attribute filtering with semantic search (Image by Author) Many modern vector and graph database also allow properties from relational databases, most notably metadata or attribute filtering. If you know the question is about the 5th Harry Potter book, it would be really nice to filter your entire database first to only contain documents from the 5th Harry Potter book, and not run embeddings search through the entire dataset. Optimal metadata filtering in Vector Databases is a pretty amazing area in Computer Science research, and a seperate article would be best for a in-depth discussion about this. * * * ## 4. Query transformation **Next, let's move to inferencing starting with query transformation - which is any preprocessing step we do to the user's actual query before doing any similarity search. Think of it like improving the user's question to get better answers.** In general, we want to avoid searching directly with the user query. User inputs are usually very noisy and they can type random stuff - we want an additional transformation layer that interprets the user query and turns it into a search query. ### A simple example why Query Rewriting is important The most common technique to do this transformation is query rewriting. Imagine someone asks, "**What happened to the artist who painted the Mona Lisa?** " If we do semantic or keyword searches, the retrieved information will be all about the Mona Lisa, not about the artist. A query rewriting system would use an LLM to rewrite this query. The LLM might transform this into "**Leonardo da Vinci Mona Lisa artist** ", which will be a much fruitful search. ![Direct Retrieval vs Query Rewriting \(Image by Author\)](../images/010.jpeg) Direct Retrieval vs Query Rewriting (Image by Author) Sometimes we would also use **Contextual Query Writing** , where we might use additional contexts, like using the older conversation transcript from the user, or if we know that our application covers documents from 10 different books, maybe we can have a classifier LLM that classifies the user query to detect which of the 10 books we are working with. If our database is in a different language, we can also translate the query. [There are also powerful techniques like HYDE](https://arxiv.org/abs/2212.10496), which stands for **Hypothetical Document Embedding**. HYDE uses a language model to generate a hypothetical answer to the query, and do similarity search with this hypothetical answer to retrieve relevant documents. ![Hypothetical Document Embeddings \(Image by Author\)](../images/011.jpeg) Hypothetical Document Embeddings (Image by Author) Another technique is **Multi-Query Expansion** where we generate multiple queries from the single user query and perform parallel searches to retrieve multiple sets of documents. The received documents can then later go through a de-duplication step or rank fusion to remove redundant documents. A recent approach called [**Astute RAG**](https://arxiv.org/abs/2410.07176) tries to consolidate externally input knowledge with the LLM's own internal knowledge before generating answers. There are also **Multi-Hop techniques like Baleen programs**. They work by performing an initial search, analyzing the top results to find frequently co-occurring terms, and then adding these terms to the original query. This adaptive approach can help bridge the vocabulary gap between user queries and document content, and help retrieve better documents. * * * ## 5. Post Retrieval Processing Now that we've retrieved our potentially relevant documents, we can add another post-retrieval processing step before feeding information to our language model for generating the answer. For example, we can do information selection and emphasis, where an LLM selects portion of the retrieved documents that could be useful for finding the answer. We might highlight key sentences, or do semantic filtering where we remove unimportant paragraphs, or do context summarization by fusing multiple documents into one. The goal here is to avoid overwhelming our LLM with too much information, which could lead to less focused or accurate responses. ![You can use smaller LLMs to flag relevant info from retrieved documents before consolidating the context prompt for the final LLM call \(Image by Author\)](../images/012.jpeg) You can use smaller LLMs to flag relevant info from retrieved documents before consolidating the context prompt for the final LLM call (Image by Author) Often we do multiple queries with query expansion, or use multiple retrieval algorithms like Embeddings+BM-25 to separately fetch multiple documents. To remove duplicates, we often use reranking methods like Reciprocal Rank Fusion. RRF combines the rankings from all the different approaches, giving higher weight to documents that consistently rank well across multiple methods. In the end, the top K high ranking documents are passed to the LLM. ![Reciprocal Rank Fusion is a classic Search Engine algorithm to combine item ranks obtained from multiple ranking algorithms \(Image by author\)](../images/body-11a5a7deec41993130f2.jpeg) Reciprocal Rank Fusion is a classic Search Engine algorithm to combine item ranks obtained from multiple ranking algorithms (Image by author) [FLARE or forward-looking active retrieval augmented generation](https://arxiv.org/abs/2305.06983) is an iterative post-retrieval strategy. Starting with the user input and initial retrieval results, an LLM iteratively guesses the next sentence. Then we check if the generated guess contains any low probability tokens indicated here with an underline - if so, we call the retriever to retrieve useful documents from the dataset and make necessary corrections. * * * ## Final Thoughts Please accept cookies to access this content For a more visual breakdown of the different components of RAGs, do checkout my Youtube video on this topic. The field of LLMs and RAGs are rapidly evolving - a thorough understanding of the RAG framework is incredibly essential to appreciate the pros and cons of each approach and weigh which approaches work best for YOUR use-case. The next time you are thinking of designing a RAG system, do stop and ask yourself these questions - * What are my data sources? * How should I chunk my data? Is there inherent structure that comes with my data domain? Do my chunks need additional context (contextual chunking)? * Do I need semantic retrieval (embeddings) or more exact-match retrieval (BM-25)? What type of queries am I expecting from the user? * What database should I use? Is my data a graph? Does it need metadata-filtering? How much money do I want to spend on databases? * How can I best rewrite the user query for easy search hits? Can an LLM rewrite the queries? Should I use HYDE? If LLMs already have enough domain knowledge about my target field, can I use Astute? * Can I combine multiple different retrieval algorithms and then do rank fusion? (honestly, just do it if you can afford it cost-wise and latency-wise) ## The Author Check out my Youtube channel where I post content about Deep Learning, Machine Learning, Paper Reviews, Tutorials, and just about anything related to AI (except news, there are WAY too many Youtube channels for AI news). Here are some of my links: **Youtube Channel:** **Patreon:** Give me a follow on Medium and a clap if you enjoyed this! ## References Vector Databases: Metadata Filtering: Contextual Chunking: Propositions / Dense X Retrieval: Hypothetical Document Embeddigs (HYDE): FLARE: ## Register Now to Continue Reading ### Unlock Towards Data Science with a Free Account All articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more. Register for freeSign in to your account * * * ![](../images/body-849620ef9680d77fc51e.jpeg) Written By ## [Avishek Biswas](https://towardsdatascience.com/author/neural.avb/) [See all from Avishek Biswas](https://towardsdatascience.com/author/neural.avb/) [AI](https://towardsdatascience.com/tag/ai/)[Genai](https://towardsdatascience.com/tag/genai/)[Llm](https://towardsdatascience.com/tag/llm/)[NLP](https://towardsdatascience.com/tag/nlp/)[Rags](https://towardsdatascience.com/tag/rags/) Share This Article * [](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Ftowardsdatascience.com%2Fthe-ultimate-guide-to-rags-each-component-dissected-3cd51c4c0212%2F&title=The%20Ultimate%20Guide%20to%20RAGs%20-%20Each%20Component%20Dissected) * [](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Ftowardsdatascience.com%2Fthe-ultimate-guide-to-rags-each-component-dissected-3cd51c4c0212%2F&title=The%20Ultimate%20Guide%20to%20RAGs%20-%20Each%20Component%20Dissected) * [](https://x.com/share?url=https%3A%2F%2Ftowardsdatascience.com%2Fthe-ultimate-guide-to-rags-each-component-dissected-3cd51c4c0212%2F&text=The%20Ultimate%20Guide%20to%20RAGs%20-%20Each%20Component%20Dissected) Towards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program. [Write for TDS](https://towardsdatascience.com/submissions/) ## Related Articles * [![](../images/013.jpg)](https://towardsdatascience.com/let-us-extract-some-topics-from-text-data-part-iv-bertopic-46ddf3c91622/) ## [Let us Extract some Topics from Text Data - Part IV: BERTopic](https://towardsdatascience.com/let-us-extract-some-topics-from-text-data-part-iv-bertopic-46ddf3c91622/) [Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/) Learn more about the family member of BERT for topic modelling [Seungjun (Josh) Kim](https://towardsdatascience.com/author/joshnjuny/) December 19, 202210 min read * [![](../images/014.jpg)](https://towardsdatascience.com/exposing-nlp-routines-for-job-hunters-90e6aed661a3/) ## [Exposing NLP routines for Job hunters](https://towardsdatascience.com/exposing-nlp-routines-for-job-hunters-90e6aed661a3/) [NLP](https://towardsdatascience.com/category/artificial-intelligence/nlp/) Providing a resource for job seekers to check keyword matches [David Moore](https://towardsdatascience.com/author/cognitivedave/) April 7, 20217 min read * [![](../images/015.jpg)](https://towardsdatascience.com/semantic-highlighting-d2fba3da2822/) ## [Semantic Highlighting](https://towardsdatascience.com/semantic-highlighting-d2fba3da2822/) [NLP](https://towardsdatascience.com/category/artificial-intelligence/nlp/) Building and adding a User Interface for NLP [David Moore](https://towardsdatascience.com/author/cognitivedave/) February 27, 20214 min read * [![](../images/016.jpg)](https://towardsdatascience.com/understanding-outliers-in-text-data-with-transformers-cleanlab-and-topic-modeling-db3585415a19/) ## [Understanding Outliers in Text Data with Transformers, Cleanlab, and Topic Modeling](https://towardsdatascience.com/understanding-outliers-in-text-data-with-transformers-cleanlab-and-topic-modeling-db3585415a19/) [NLP](https://towardsdatascience.com/category/artificial-intelligence/nlp/) An open-source python workflow to audit text datasets Many text corpora contain heterogeneous documents, some... [Elías Snorrason](https://towardsdatascience.com/author/eliassno/) October 6, 202210 min read * [![](../images/017.jpeg)](https://towardsdatascience.com/a-walkthrough-of-nvidias-latest-multi-modal-llm-family-fdc067b59596/) ## [A Walkthrough of Nvidia's Latest Multi-Modal LLM Family](https://towardsdatascience.com/a-walkthrough-of-nvidias-latest-multi-modal-llm-family-fdc067b59596/) [Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/) From LLaVA, Flamingo, to NVLM [Mengliu Zhao](https://towardsdatascience.com/author/mengliuz/) October 10, 20246 min read * [![](../images/018.png)](https://towardsdatascience.com/nlp-with-rapids-yes-its-possible-1830287210ed/) ## [NLP With RAPIDS, Yes It's Possible!](https://towardsdatascience.com/nlp-with-rapids-yes-its-possible-1830287210ed/) [Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/) Principal Component Analysis on Sentence Embeddings [Louise Ferbach](https://towardsdatascience.com/author/lsferbach/) April 10, 20217 min read * [![](../images/019.jpeg)](https://towardsdatascience.com/what-gpt-4-brings-to-the-ai-table-74e392a32ac3/) ## [What GPT-4 Brings to the AI Table](https://towardsdatascience.com/what-gpt-4-brings-to-the-ai-table-74e392a32ac3/) [Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/) A language model and more [Chiedozie Onyearugbulem](https://towardsdatascience.com/author/doziesixtus/) April 14, 20237 min read * [![](../images/020.jpg)](https://towardsdatascience.com/a-detailed-novice-introduction-to-natural-language-processing-nlp-90b7be1b7e54/) ## [A Detailed, Novice Introduction to Natural Language Processing (NLP)](https://towardsdatascience.com/a-detailed-novice-introduction-to-natural-language-processing-nlp-90b7be1b7e54/) [Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/) The Ultimate Code-Based Guide to Getting Started with NLP in Python [Rahulraj Singh](https://towardsdatascience.com/author/rahulraj24/) July 29, 20212 min read * [![](../images/021.jpeg)](https://towardsdatascience.com/building-a-chatbot-for-slack-from-scratch-part-1-understanding-language-1f085b2eda6c/) ## [Building a Game of Thrones chatbot for Slack from scratch: Part 1 Understanding Language](https://towardsdatascience.com/building-a-chatbot-for-slack-from-scratch-part-1-understanding-language-1f085b2eda6c/) [Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/) How to incorporate the latest deep learning techniques to improve conversational goal-oriented-dialogue. [Isaac Godfried](https://towardsdatascience.com/author/igodfried/) March 25, 201913 min read * [YouTube](https://www.youtube.com/c/TowardsDataScience) * [X](https://x.com/TDataScience) * [LinkedIn](https://www.linkedin.com/company/towards-data-science/) * [Threads](https://www.threads.net/@towardsdatascience) * [Bluesky](https://bsky.app/profile/towardsdatascience.com) [](https://towardsdatascience.com/) Your home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals. Subscribe to Our Newsletter[Write For TDS](https://towardsdatascience.com/submissions/)•[About](https://towardsdatascience.com/about-towards-data-science-d691af11cc2f/)•[Advertise](https://contact.towardsdatascience.com/advertise-with-towards-data-science)•[Privacy Policy](https://towardsdatascience.com/privacy-policy/)•[Terms of Use](https://towardsdatascience.com/website-terms-of-use/) © Insight Media Group, LLC 2026•[The New Stack](https://thenewstack.io)•[roadmap.sh](https://roadmap.sh)•[Creator Network](https://insightmediagroup.io/creator-network) #### Recommended Articles Close * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500) Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1 ](https://towardsdatascience.com/javascript-fatigue-you-dont-need-js-to-build-chatgpt/ "Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1") * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500) The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs ](https://towardsdatascience.com/subset-sum-problem-solved-in-linear-time-for-dense-enough-inputs/ "The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs") * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/3660/image?etag=1780176246070&width=500&height=500) Does Your Company Have a Data Strategy? ](https://towardsdatascience.com/does-your-company-have-a-data-strategy-e8944dbec4ae/ "Does Your Company Have a Data Strategy?") * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/4970/image?etag=1780565225463&width=500&height=500) Productivity Tips, Data Career Insights, and Other Recent Must-Reads ](https://towardsdatascience.com/productivity-tips-data-career-insights-and-other-recent-must-reads-5c47eb7a2fe0/ "Productivity Tips, Data Career Insights, and Other Recent Must-Reads")