{ "source_file": "0930", "source_platform": "Medium", "author": "Hesam Sheikh", "title_listed": "Understanding Buffer of Thoughts (BoT) - Reasoning with Large Language Models", "keywords": "", "domain": "大模型算法", "content_type": "深度技术解析", "is_paid_flag": "否", "url": "https://towardsdatascience.com/understanding-buffer-of-thoughts-bot-reasoning-with-large-language-models-391919d2f76f/", "publish_date": "2024-06-15", "corpus_stage": "", "notes": "", "id": "093822e7cfa9", "collected_at": "2026-10-02T06:31:43", "title": "Understanding Buffer of Thoughts (BoT) - Reasoning with Large Language Models | Towards Data Science", "body_markdown": "![Revisit consent button](../images/body-df29beacd7496220cc94.svg)\nWe value your privacy\nWe use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking \"Accept All\", you consent to our use of cookies.\nCustomise Reject All Accept All\nCustomise Consent Preferences ![](../images/body-2159bb67f3de8ad870eb.svg)\nWe use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below.\nThe cookies that are categorised as \"Necessary\" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... Show more\nNecessaryAlways Active\nNecessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.\n * Cookie\nBCTempID\n * Duration\n10 minutes\n * Description\nNo description available.\n\n\n * Cookie\n__cf_bm\n * Duration\n1 hour\n * Description\nThis cookie, set by Cloudflare, is used to support Cloudflare Bot Management. \n\n\n * Cookie\nAWSALBCORS\n * Duration\n7 days\n * Description\nAmazon Web Services set this cookie for load balancing.\n\n\n * Cookie\n_cfuvid\n * Duration\nsession\n * Description\nCloudflare sets this cookie to track users across sessions to optimize user experience by maintaining session consistency and providing personalized services\n\n\n * Cookie\nli_gc\n * Duration\n6 months\n * Description\nLinkedin set this cookie for storing visitor's consent regarding using cookies for non-essential purposes.\n\n\n * Cookie\n__hssrc\n * Duration\nsession\n * Description\nThis cookie is set by Hubspot whenever it changes the session cookie. The __hssrc cookie set to 1 indicates that the user has restarted the browser, and if the cookie does not exist, it is assumed to be a new session.\n\n\n * Cookie\n__hssc\n * Duration\n1 hour\n * Description\nHubSpot sets this cookie to keep track of sessions and to determine if HubSpot should increment the session number and timestamps in the __hstc cookie.\n\n\n * Cookie\nwpEmojiSettingsSupports\n * Duration\nsession\n * Description\nWordPress sets this cookie when a user interacts with emojis on a WordPress site. It helps determine if the user's browser can display emojis properly.\n\n\n * Cookie\nBCSessionID\n * Duration\n1 year 1 month 4 days\n * Description\nBlueconic sets this cookie as a unique identifier for the BlueConic profile.\n\n\n * Cookie\n_octo\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nlogged_in\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\n__Secure-YEC\n * Duration\npast\n * Description\nYouTube sets this cookie to stores the user's video player preferences using embedded YouTube video\n\n\n * Cookie\n__eoi\n * Duration\n6 months\n * Description\nDescription is currently not available.\n\n\n * Cookie\nAWSALBTGCORS\n * Duration\n7 days\n * Description\nNo description available.\n\n\n * Cookie\nlogin-status-p\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nAWSALBTG\n * Duration\n7 days\n * Description\nNo description available.\n\n\n * Cookie\ncsrf_token\n * Duration\nsession\n * Description\nNo description available.\n\n\n * Cookie\ntoken_v2\n * Duration\n1 day\n * Description\nDescription is currently not available.\n\n\n * Cookie\nD\n * Duration\n1 year\n * Description\nDescription is currently not available.\n\n\n * Cookie\nPHPSESSID\n * Duration\nsession\n * Description\nThis cookie is native to PHP applications. The cookie stores and identifies a user's unique session ID to manage user sessions on the website. The cookie is a session cookie and will be deleted when all the browser windows are closed.\n\n\n * Cookie\nVISITOR_PRIVACY_METADATA\n * Duration\n6 months\n * Description\nYouTube sets this cookie to store the user's cookie consent state for the current domain. \n\n\n * Cookie\ncookietest\n * Duration\nsession\n * Description\nThe cookietest cookie is typically used to determine whether the user's browser accepts cookies, essential for website functionality and user experience.\n\n\n * Cookie\n__Host-airtable-session\n * Duration\n1 year\n * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n * Cookie\n__Host-airtable-session.sig\n * Duration\n1 year\n * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n * Cookie\nm\n * Duration\n1 year 1 month 4 days\n * Description\nStripe sets this cookie for fraud prevention purposes. It identifies the device used to access the website, allowing the website to be formatted accordingly.\n\n\n * Cookie\nBIGipServer*\n * Duration\nsession\n * Description\nMarketo sets this cookie to collect information about the user's online activity and build a profile about their interests to provide advertisements relevant to the user. \n\n\n * Cookie\n__cfruid\n * Duration\nsession\n * Description\nCloudflare sets this cookie to identify trusted web traffic.\n\n\n * Cookie\n_GRECAPTCHA\n * Duration\n6 months\n * Description\nGoogle Recaptcha service sets this cookie to identify bots to protect the website against malicious spam attacks.\n\n\n * Cookie\n__Secure-YNID\n * Duration\n6 months\n * Description\nGoogle cookie used to protect user security and prevent fraud, especially during the login process.\n\n\n * Cookie\ncookieyes-consent\n * Duration\n1 year\n * Description\nCookieYes sets this cookie to remember users' consent preferences so that their preferences are respected on subsequent visits to this site. It does not collect or store any personal information about the site visitors.\n\n\nFunctional\nFunctional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.\n * Cookie\nlidc\n * Duration\n1 day\n * Description\nLinkedIn sets the lidc cookie to facilitate data center selection.\n\n\n * Cookie\nbrw\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nbrwConsent\n * Duration\n5 minutes\n * Description\nDescription is currently not available.\n\n\n * Cookie\nWMF-Uniq\n * Duration\n1 year\n * Description\nDescription is currently not available.\n\n\n * Cookie\nloom_anon_comment\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nloom_referral_video\n * Duration\nsession\n * Description\nDescription is currently not available.\n\n\n * Cookie\nVISITOR_INFO1_LIVE\n * Duration\n6 months\n * Description\nA cookie set by YouTube to measure bandwidth that determines whether the user gets the new or old player interface.\n\n\n * Cookie\nyt-remote-connected-devices\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n * Cookie\nytidb::LAST_RESULT_ENTRY_KEY\n * Duration\nNever Expires\n * Description\nThe cookie ytidb::LAST_RESULT_ENTRY_KEY is used by YouTube to store the last search result entry that was clicked by the user. This information is used to improve the user experience by providing more relevant search results in the future.\n\n\n * Cookie\nyt-remote-device-id\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n * Cookie\nyt-remote-session-name\n * Duration\nsession\n * Description\nThe yt-remote-session-name cookie is used by YouTube to store the user's video player preferences using embedded YouTube video.\n\n\n * Cookie\nyt-remote-fast-check-period\n * Duration\nsession\n * Description\nThe yt-remote-fast-check-period cookie is used by YouTube to store the user's video player preferences for embedded YouTube videos.\n\n\n * Cookie\nyt-remote-session-app\n * Duration\nsession\n * Description\nThe yt-remote-session-app cookie is used by YouTube to store user preferences and information about the interface of the embedded YouTube video player.\n\n\n * Cookie\nyt-remote-cast-available\n * Duration\nsession\n * Description\nThe yt-remote-cast-available cookie is used to store the user's preferences regarding whether casting is available on their YouTube video player.\n\n\n * Cookie\nyt-remote-cast-installed\n * Duration\nsession\n * Description\nThe yt-remote-cast-installed cookie is used to store the user's video player preferences using embedded YouTube video.\n\n\n * Cookie\ncp_session\n * Duration\n3 months\n * Description\nCodepen sets this cookie for Help systems found in the website.\n\n\n * Cookie\nloid\n * Duration\n1 year 1 month 4 days\n * Description\nThis cookie is set by the Reddit. The cookie enables the sharing of content from the website onto the social media platform.\n\n\nAnalytics\nAnalytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.\n * Cookie\n__hstc\n * Duration\n6 months\n * Description\nHubspot set this main cookie for tracking visitors. It contains the domain, initial timestamp (first visit), last timestamp (last visit), current timestamp (this visit), and session number (increments for each subsequent session).\n\n\n * Cookie\nhubspotutk\n * Duration\n6 months\n * Description\nHubSpot sets this cookie to keep track of the visitors to the website. This cookie is passed to HubSpot on form submission and used when deduplicating contacts.\n\n\n * Cookie\n_ga\n * Duration\n1 year 1 month 4 days\n * Description\nGoogle Analytics sets this cookie to calculate visitor, session and campaign data and track site usage for the site's analytics report. The cookie stores information anonymously and assigns a randomly generated number to recognise unique visitors.\n\n\n * Cookie\n_ga_*\n * Duration\n1 year 1 month 4 days\n * Description\nGoogle Analytics sets this cookie to store and count page views.\n\n\n * Cookie\n__Host-psifi.analyticsTrace\n * Duration\n6 hours\n * Description\nDescription is currently not available.\n\n\n * Cookie\n__Host-psifi.analyticsTraceV2\n * Duration\n6 hours\n * Description\nDescription is currently not available.\n\n\n * Cookie\n_gh_sess\n * Duration\nsession\n * Description\nGitHub sets this cookie for temporary application and framework state between pages like what step the user is on in a multiple step form.\n\n\n * Cookie\nYSC\n * Duration\nsession\n * Description\nYSC cookie is set by Youtube and is used to track the views of embedded videos on Youtube pages.\n\n\n * Cookie\najs_anonymous_id\n * Duration\n1 year\n * Description\nThis cookie is set by Segment to count the number of people who visit a certain site by tracking if they have visited before.\n\n\n * Cookie\nvuid\n * Duration\n1 year 1 month 4 days\n * Description\nVimeo installs this cookie to collect tracking information by setting a unique ID to embed videos on the website. \n\n\nPerformance\nPerformance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors.\n * Cookie\nAWSALB\n * Duration\n7 days\n * Description\nAWSALB is an application load balancer cookie set by Amazon Web Services to map the session to the target.\n\n\n * Cookie\nacq\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nacq.sig\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nptc\n * Duration\n2 years\n * Description\nNo description available.\n\n\nAdvertisement\nAdvertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns.\n * Cookie\nmuc_ads\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to collect user behaviour and interaction data to optimize the website.\n\n\n * Cookie\nguest_id_marketing\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n * Cookie\nguest_id_ads\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n * Cookie\npersonalization_id\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to integrate and share features for social media and also store information about how the user uses the website, for tracking and targeting.\n\n\n * Cookie\nguest_id\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor. It registers if a user is signed in to the Twitter platform and collects information about ad preferences.\n\n\n * Cookie\nbcookie\n * Duration\n1 year\n * Description\nLinkedIn sets this cookie from LinkedIn share buttons and ad tags to recognize browser IDs.\n\n\n * Cookie\n__Secure-ROLLOUT_TOKEN\n * Duration\n6 months\n * Description\nYouTube sets this cookie to manage feature rollout and experimentation. It helps Google control which new features or interface changes are shown to users as part of testing and staged rollouts, ensuring consistent experience for a given user during an experiment.\n\n\n * Cookie\nyt.innertube::nextId\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n * Cookie\nyt.innertube::requests\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n * Cookie\nsession_tracker\n * Duration\nsession\n * Description\nThis cookie is set by the Reddit. This cookie is used to identify trusted web traffic. It also helps in adverstising on the website.\n\n\n * Cookie\nedgebucket\n * Duration\nsession\n * Description\nReddit sets this cookie to save the information about a log-on Reddit user, for the purpose of advertisement recommendations and updating the content.\n\n\n * Cookie\ndid\n * Duration\n1 year\n * Description\nArbor sets this cookie to show targeted ads to site visitors.This cookie expires after 2 months or 1 year.\n\n\nUncategorised\nOther uncategorised cookies are those that are being analysed and have not been classified into a category as yet.\nNo cookies to display.\nReject All Save My Preferences Accept All\nToggle Search[](https://towardsdatascience.com/)Toggle Mobile Navigation\n[](https://towardsdatascience.com/)\nPublish AI, ML & data-science insights to a global community of data professionals.\nSign in[Submit an Article](https://write.insightmediagroup.io)\n[Latest](https://towardsdatascience.com/latest/)[Editor's Picks](https://towardsdatascience.com/tag/editors-pick/)[Newsletter](https://towardsdatascience.com/tag/the-variable/)[ShipAI](https://towardsdatascience.com/shipai/)\n* * *\n[Write For TDS](https://towardsdatascience.com/submissions/)\n[LinkedIn](https://www.linkedin.com/company/towards-data-science/)[X](https://x.com/TDataScience)Toggle Search\n[Data Science](https://towardsdatascience.com/category/data-science/)\n# Understanding Buffer of Thoughts (BoT) - Reasoning with Large Language Models\nNew prompt engineering tool for complex reasoning, compared with Chain of thought (CoT) and Tree of Thought (ToT)\n[Hesam Sheikh](https://towardsdatascience.com/author/itshesamsheikh/)\nJune 14, 202410 min read\n### A new prompting tool for complex reasoning, compared with Chain of thought (CoT) and Tree of Thought (ToT)\n> If you're not a member, [**read for free!**](https://hesamsheikh.substack.com/)**✨**\n![Photo by Etienne Girardet on Unsplash](../images/000.jpg)\nPhoto by [Etienne Girardet](https://unsplash.com/@etiennegirardet?utm_source=medium&utm_medium=referral) on [Unsplash](https://unsplash.com?utm_source=medium&utm_medium=referral)\nSqueezing proficiency in complex reasoning tasks and avoiding hallucinations remains a major research topic in Large Language Models (LLMs). Despite the effort, LLMs need help with generalized reasoning capabilities. Traditional methods such as **Chain-of-Thought (CoT)** or **Tree-of-Thought (ToT)** often require multiple assumptions or numerous back-and-forth prompting which means intensive computation.\n![Buffer of Thoughts \\(BoT\\) vs other prompting methods. \\(source: Paper\\)](../images/001.png)\nBuffer of Thoughts (BoT) vs other prompting methods. (source: [Paper](https://arxiv.org/abs/2406.04271))\nThe new proposed method in the paper, [**Buffer of Thoughts: Thought-Augmented Reasoning with Large Language Models**](https://arxiv.org/abs/2406.04271)**[1]** , combats these limitations with a dynamic, adaptive repository of high-level thought templates **** called** meta-buffer.**In BoT, once the user presents a new problem, it is first simplified and analyzed to extract key elements, which then guide the retrieval of a relevant thought template from a dynamic dataset. This allows adaptive and efficient problem-solving through modified and complex reasoning patterns. According to the original paper, this is so effective that _** \"Llama3–8B+BoT**has the potential to surpass** Llama3–70B model.**_\"\n**BoT** achieves efficient reasoning across problems that are similar to its templates as it:\n * (1) leverages previous solutions on new challenges,\n * (2) boosts efficiency by eliminating the need for multiple query iterations (as we see in Graph-of-Thoughts (GoT) or ToT), and\n * (3) dynamically updates its template repository to ensure it evolves as it encounters new tasks.\n\n\nIn this article, we will first go through the general outline of how BoT works, understand the function of each key part, and test the procedure with an example.\n## How does BoT work?\nThe general thought-augmented reasoning process (as shown in the figure below) starts with **Problem Distillation** , which analyzes and condenses the incoming task into essential elements and constraints and then creates a simplified problem statement.\nThis distilled information is then used to query the **Meta-Buffer** , a dynamic repository that contains high-level thought templates. From the thought templates, one that is most similar to the distilled problem is retrieved. Then, during the **Instantiation Process** , it is instantiated with specific requirements and information about the distilled problem.\nThroughout this process, the **Buffer Manager** actively keeps an eye on the **Meta-Buffer.** Once it detects a new insight not included in the meta-buffer, **Buffer Manager** updates it to ensure a continual evolution of the thought template repository.\n![The BoT process. \\(source: Paper\\)](../images/002.png)\nThe BoT process. (source: [Paper](http://Paper))\nLet's go through each of these key parts to gain a more detailed look:\n### Problem Distiller\nProblem Distiller can be thought of as a **preprocess** on the input tasks in order to...\n * (1) extract essential information of the problem, and\n * (2) simplify complex tasks for a better search and retrieval of thought templates.\n\n\nProblem Distiller takes the burden off of LLM to identify and extract vital information and constraints of the problem. This is done by a meta prompt ϕ:\n![\\(source: Paper\\)](../images/003.png)\n(source: [Paper](http://Paper))\nThe prompt used by the authors to distill key information about a task is as follows:\nvbnet\n\n```\n[Problem Distiller]:As a highly professional and intelligent expert in information distillation, you excel atextracting essential information to solve problems from user input queries. You adeptlytransform this extracted information into a suitable format based on the respective type of theissue.Please categorize and extract the crucial information required to solve the problem from theuser's input query, the distilled information should include.1. Key information:Values and information of key variables extracted from user input, which will be handed overto the respective expert for task resolution, ensuring all essential information required to solvethe problem is provided.2. Restrictions:The objective of the problem and corresponding constraints.3. Distilled task:Extend the problem based on 1 and 2, summarize a meta problem that can address the userquery and handle more input and output variations. Incorporate the real-world scenario of theextended problem along with the types of key variables and information constraints from theoriginal problem to restrict the key variables in the extended problem. After that, use the userquery input key information as input to solve the problem as an example.\n```\n\n### Meta-Buffer\nThe meta-buffer is a central database that stores high-level thought templates. These templates are high-level abstractions representing various problem-solving processes. The idea is that LLM can leverage past problems and insights to solve current challenges. The best part is that the Meta-Buffer dynamically updates to ensure new unseen problems are also included. The Meta-Buffer doesn't enforce thought templates to follow a specific instruction.\n**Template Retrieval** : Once a task is distilled, BoT would go through the thought templates and grab the one most similar to the task. This is done by calculating the embedding similarity between the task and the thought templates.\n![The retriever would calculate the similarity between the embedding of the input task f\\(xd\\), and the embedding of templates f\\(DTi \\). \\(source: Paper\\)](../images/004.png)\nThe retriever would calculate the similarity between the embedding of the input task f(xd), and the embedding of templates f(DTi ). (source: [Paper](https://arxiv.org/abs/2406.04271))\nthe retriever would calculate the similarity between the embedding of the input task **f(xd)** , and the embedding of templates **f(D** _**Ti**_**).** This is only done if the similarity is above a certain threshold δ (0.5–0.7). If none of the thought templates have a similarity score with the task above the δ threshold, then the **xd** is identified as a new task. Depending on if the task is new or not, one of the two paths would be taken:\n * If the task is similar to one of the thought templates, the template would be instantiated with the distilled information using an instantiation prompt (which you can check in the paper). This instantiation process can be denoted as\n\n\n![Instantiated Reasoning. \\(source: Paper\\)](../images/005.png)\nInstantiated Reasoning. (source: [Paper](https://arxiv.org/abs/2406.04271))\n * If the task is new, a general thought template that is designed to address a broad set of problems is used. As the task is processed, the **Buffer Manager** observes and learns and potentially creates a new, more specific thought template and pushes it to the meta-buffer.\n\n\n### Buffer Manager\nThe Buffer Manager serves as a crucial player in maintaining and enhancing the meta-buffer. Based on the new insights and outcomes from the tasks that are solved, it updates the thought templates. Also, whenever a new or drastically different problem is solved, the buffer manager assesses whether or not to create a new thought template. This is to ensure thought templates remain to the point and are not redundant.\n![checking whether or not a newly generated template is similar to the existing ones. \\(source: Paper\\)](../images/006.png)\nchecking whether or not a newly generated template is similar to the existing ones. (source: Paper)\nBy employing the above formulation, the Buffer Manager checks whether or not the meta-buffer already has the necessary knowledge to tackle a problem.\n### BoT vs. Single-Query vs. Multi-Query\nHow does the BoT stand out compared to previous methods? The authors of the paper evaluate various methods on different datasets of various tasks, such as Data Understanding, Python Programming Puzzles, Multilingual Grade School Math **** (MGSM), etc. The results show a surprising advantage of** BoT** in almost all the tasks.\n![BoT vs previous methods. The best results \\(marked in blue\\) are all achieved by BoT. \\(source: Paper\\)](../images/007.png)\nBoT vs previous methods. The best results (marked in blue) are all achieved by BoT. (source: [Paper](https://arxiv.org/abs/2406.04271))\nOne of the key advantages of BoT is its efficiency - requiring **only 12%** of the computational cost on average when compared to multi-query prompting methods. This high computational cost and latency of multi-query methods such as ToT often renders them impractical in real-world use cases.\n> BoT+Llama3–8B has the potential to surpass single Llama3–70B model\n![comparing the effect of BoT on Llama3–8B and 70B. Annotated. \\(source: Paper\\)](../images/008.png)\ncomparing the effect of BoT on Llama3–8B and 70B. Annotated. (source: [Paper](https://arxiv.org/abs/2406.04271))\n## Buffer of Thoughts in Practice\nThe demo code for Buffer of Thoughts is published on [GitHub](https://github.com/YangLing0818/buffer-of-thought-llm) [2]. To test out the functionality in practice, I will use this method on a custom task: Word Reordering. In this task, the LLM must take a scrambled sentence, such as _\"Sam name is my\"_ , and return a permutation of these words that is semantically meaningful, which in this example would be _\"my name is Sam\",_ (this is not a benchmark with baseline performance). Some examples of the scrambled sentences and the correct ones are as follows:\njson\n\n```\n{\"input\": \" life plan and families to for social hospital workers outside with patients work the \", \n\"target\": \" social workers work with patients and families to plan for life outside the hospital \"}\n{\"input\": \" yield plant refers dry total to production biological matter \", \n\"target\": \" biological yield refers to total plant dry matter production \"}\n{\"input\": \" the bloodstream into alcohol from directly stomach goes the \", \n\"target\": \" alcohol goes directly from the stomach into the bloodstream \"}\n```\n\nI will also use the following user prompt to explain the task:\ntext\n\n```\nReorder a list of words to make them semantically meaningful, \nfor example \"Sam name is my\" -> \"my name is Sam\". \nPlease provide only the reordered sentence without any explanations. \nStart the ordered sentence with and end it with \n```\n\nProvided this task, the BoT framework first distills my task and extracts the goal, the restriction, and how I would like my output along with other key information. The full procedure of BoT in detail, is the following terminal output:\nbash\n\n```\nUser prompt:\nReorder a list of words to make them semantically meaningful, for example \"Sam name is my\" -> \"my name is Sam\". Please provide only the reordered sentence without any explanations. Start the ordered sentence with and end it with \nInput:\n the melting in solid to gold leaf metals is achieve made by desired gold and mixing color other \nDistilled information:Meta distiller Respond:\n\nDistilled Information:\n\n1. Key information:\n- Input: \" the melting in solid to gold leaf metals is achieve made by desired gold and mixing color other \"\n\n2. Restriction:\n- Reorder the list of words such that the sentence is semantically meaningful\n- The output should begin with \"\" and end with \"\"\n\n3. Distilled task:\n- The task is to reorder the words to create a meaningful sentence from the provided mixed words inside the tags \"\" and \"\"\n\n4. Python transformation:\n (Optional, skip when Python tag is Not for Python) Input parameters:\n input_sentence = \" the melting in solid to gold leaf metals is achieve made by desired gold and mixing color other \"\n\n5. Answer form: (Optional, skip when there is no specific answer form)\n The output sentence should be in the form of \" ... \"\n\n **Note: The generation ends here. Do not show this message in your answer !**\nInstantiated reasoning result: ```python\ndef ReorderWords(input_sentence):\n # Remove the start and end tags, and split the sentence into words\n words = input_sentence.replace(\"\", \"\").replace(\"\", \"\").strip().split()\n\n # Tentative reordering to form a meaningful sentence\n # Based on the context provided in the task, the following reordering seems logical\n reordered_words = [\"the\", \"melting\", \"in\", \"solid\", \"metals\", \"is\", \"achieved\", \"by\", \"mixing\", \"gold\", \"leaf\", \"and\", \"desired\", \"other\", \"color\", \"gold\"]\n\n # Join the words into a single sentence and add start and end tags\n reordered_sentence = \" \" + \" \".join(reordered_words) + \" \"\n\n return reordered_sentence\n\n# Input sentence provided in the task\ninput_sentence = \" the melting in solid to gold leaf metals is achieve made by desired gold and mixing color other \"\nprint(ReorderWords(input_sentence))\n```\n\nThe result of code execution: the melting in solid metals is achieved by mixing gold leaf and desired other color gold \ntext\n\n```\n\nSome of the examples of the reordered sentences using BoT:\n\n```bash\n{\"input\": \" life plan and families to for social hospital workers outside with patients work the \", \n\"result\": \" Hospital workers work outside with social patients and plan to the families for life n\"}\n{\"input\": \" yield plant refers dry total to production biological matter \", \n\"result\": \" Plant yield refers to total dry matter biological production n\"}\n{\"input\": \" the bloodstream into alcohol from directly stomach goes the \", \n\"result\": \" the alcohol goes directly from the stomach into the bloodstream n\"}\n```\n\nNote, that the BoT repository I used is a demo code and lacks some of the features mentioned in the original paper, such as a general thought template, the dynamic updating of the Meta-Buffer, or finding the closest template embedding to the user task. These are important aspects of the framework and without them, we cannot conclude the performance of Buffer of Thoughts in practice.\n## Final Words\nIn conclusion, BoT shows promising results in both accuracy and efficiency in various domains and tasks. It's an interesting approach to breaking down a reasoning problem into its fundamental restrictions and key information and building on top of previous solutions and templates to better formulate the task for an LLM to understand.\nBy addressing some of the limitations in other prompting techniques, Buffer of Thoughts allows LLM to have more complex thinking patterns, potentially making smaller lightweight models perform at the level of larger ones.\nAllowing small LLMs to achieve results close to large LLMs is a key topic addressed in many of the current research papers. The goal is to employ various prompting and fine-tuning techniques to extract accurate AI outputs with low computation and cost.\nBuffer of Thoughts is a novel and promising prompting framework that leverages a domain of techniques to guide LLM step by step, in a reasoning process. A complete practical implementation of the Buffer of Thoughts technique is yet to come, but in the meanwhile, test out the provided benchmarks in the demo GitHub repository [2].\n* * *\nIf you have made it this far, consider this for a **further read** :\n> [**Platonic Representation: Are AI Deep Network Models Converging?**](https://towardsdatascience.com/platonic-representation-hypothesis-c812813d7248)\n**🌟 Join +1000 people learning about** Python, ML / MLOps / AI, Data Science, and LLM. [**follow me**](https://medium.com/@itshesamsheikh/subscribe) and check out my **X/Twitter** , where I keep you updated Daily**.**\nThanks for reading,\n— Hesam\n* * *\n[1] Yang, L., Yu, Z., Zhang, T., Cao, S., Xu, M., Zhang, W., Gonzalez, J. E., & Cui, B. (2024). Buffer of Thoughts: Thought-Augmented Reasoning with Large Language Models. _arXiv_. Retrieved from \n[2] buffer-of-thought-llm, \n## Register Now to Continue Reading\n### Unlock Towards Data Science with a Free Account\nAll articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more.\nRegister for freeSign in to your account\n* * *\n![](../images/body-2e861118b1a56d435c93.png)\nWritten By\n## [Hesam Sheikh](https://towardsdatascience.com/author/itshesamsheikh/)\n[See all from Hesam Sheikh](https://towardsdatascience.com/author/itshesamsheikh/)\n[Chain Of Thought](https://towardsdatascience.com/tag/chain-of-thought/)[Data Science](https://towardsdatascience.com/tag/data-science/)[Large Language Models](https://towardsdatascience.com/tag/large-language-models/)[Machine Learning](https://towardsdatascience.com/tag/machine-learning/)[Prompt Engineering](https://towardsdatascience.com/tag/prompt-engineering/)\nShare This Article\n * [](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Ftowardsdatascience.com%2Funderstanding-buffer-of-thoughts-bot-reasoning-with-large-language-models-391919d2f76f%2F&title=Understanding%20Buffer%20of%20Thoughts%20\\(BoT\\)%20-%20Reasoning%20with%20Large%20Language%20Models)\n * [](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Ftowardsdatascience.com%2Funderstanding-buffer-of-thoughts-bot-reasoning-with-large-language-models-391919d2f76f%2F&title=Understanding%20Buffer%20of%20Thoughts%20\\(BoT\\)%20-%20Reasoning%20with%20Large%20Language%20Models)\n * [](https://x.com/share?url=https%3A%2F%2Ftowardsdatascience.com%2Funderstanding-buffer-of-thoughts-bot-reasoning-with-large-language-models-391919d2f76f%2F&text=Understanding%20Buffer%20of%20Thoughts%20\\(BoT\\)%20-%20Reasoning%20with%20Large%20Language%20Models)\n\n\nTowards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program.\n[Write for TDS](https://towardsdatascience.com/submissions/)\n## Related Articles\n * [![](../images/009.jpg)](https://towardsdatascience.com/graph-theory-basics-88a89863e3c1/)\n## [Graph Theory Basics](https://towardsdatascience.com/graph-theory-basics-88a89863e3c1/)\n[Programming](https://towardsdatascience.com/category/programming/)\nWhat you need to know as graph theory adoption continues to take off\n[Miguel Saldana](https://towardsdatascience.com/author/m-saldana/)\nMarch 22, 20218 min read\n * [![](../images/010.png)](https://towardsdatascience.com/implementing-ai-is-like-buying-and-driving-a-car-but-different-1e85f6afcc92/)\n## [Implementing AI is like buying and driving a car (but different)](https://towardsdatascience.com/implementing-ai-is-like-buying-and-driving-a-car-but-different-1e85f6afcc92/)\n[Data Science](https://towardsdatascience.com/category/data-science/)\nAn analogy to help explain common myths and pitfalls to management\n[Koen Peters](https://towardsdatascience.com/author/kpeters_/)\nJanuary 18, 20238 min read\n * [![](../images/011.png)](https://towardsdatascience.com/generating-knowledge-graphs-with-wikipedia-ec17030a40f6/)\n## [Generating Knowledge Graphs with Wikipedia](https://towardsdatascience.com/generating-knowledge-graphs-with-wikipedia-ec17030a40f6/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nA Python Guide to Quickly Generate Knowledge Graphs\n[Jye Sawtell-Rickson](https://towardsdatascience.com/author/jyesr/)\nJanuary 1, 20235 min read\n * [![](../images/012.jpg)](https://towardsdatascience.com/google-offered-10m-for-a-dying-airlines-data-how-can-you-value-yours/)\n## [Google Offered $10M for a Dying Airline's Data. How Can You Value Yours?](https://towardsdatascience.com/google-offered-10m-for-a-dying-airlines-data-how-can-you-value-yours/)\n[Data Governance](https://towardsdatascience.com/category/data-science/data-governance/)\nHow to put a dollar value on your organization's operational data\n[Thuwarakesh Murallie](https://towardsdatascience.com/author/thuwarakesh/)\nSeptember 21, 20268 min read\n * [![](../images/013.jpg)](https://towardsdatascience.com/the-insiders-guide-to-generative-and-discriminative-machine-learning-models-34d5791d53d3/)\n## [What are the differences between generative and discriminative machine learning models?](https://towardsdatascience.com/the-insiders-guide-to-generative-and-discriminative-machine-learning-models-34d5791d53d3/)\n[Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/)\nGenerative and discriminative are classes of machine learning models.\n[Saurav Singla](https://towardsdatascience.com/author/sauravsingla08/)\nSeptember 12, 20206 min read\n * [![](../images/014.jpg)](https://towardsdatascience.com/how-to-use-pandas-to-get-your-data-in-the-format-you-need-dc7fd63ab5e9/)\n## [How to Use Pandas to Get Your Data in the Format You Need](https://towardsdatascience.com/how-to-use-pandas-to-get-your-data-in-the-format-you-need-dc7fd63ab5e9/)\n[Programming](https://towardsdatascience.com/category/programming/)\nLearn the difference between long-form and wide-form data, and how to transition between them in Pandas.\n[Murtaza Ali](https://towardsdatascience.com/author/murtaza5152-ali/)\nOctober 13, 20225 min read\n * [![](../images/015.png)](https://towardsdatascience.com/the-price-of-power-part-2-ebbe22b0ce07/)\n## [The Price of Power, Part 2](https://towardsdatascience.com/the-price-of-power-part-2-ebbe22b0ce07/)\n[Data Science](https://towardsdatascience.com/category/data-science/)\nRetail Power Prices in Texas: Analyzing the Whole Market in Context This is the second...\n[Dale Furrow](https://towardsdatascience.com/author/dkfurrow/)\nMay 6, 20213 min read\n * [![](../images/016.jpeg)](https://towardsdatascience.com/introduction-to-knowledge-graph-based-recommender-systems-34254efd1960/)\n## [Introduction to Knowledge Graph-Based Recommender Systems](https://towardsdatascience.com/introduction-to-knowledge-graph-based-recommender-systems-34254efd1960/)\n[Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/)\nA brief presentation of KG Recommender Systems Families\n[Amine Dadoun](https://towardsdatascience.com/author/amine.dadoun/)\nApril 1, 20237 min read\n * [![](../images/017.jpeg)](https://towardsdatascience.com/how-i-turned-my-companys-docs-into-a-searchable-database-with-openai-4f2d34bd8736/)\n## [How I Turned My Company's Docs into a Searchable Database with OpenAI](https://towardsdatascience.com/how-i-turned-my-companys-docs-into-a-searchable-database-with-openai-4f2d34bd8736/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nAnd how you can do the same with your docs\n[Jacob Marks, Ph.D.](https://towardsdatascience.com/author/jacob_marks/)\nApril 25, 202315 min read\n\n\n * [YouTube](https://www.youtube.com/c/TowardsDataScience)\n * [X](https://x.com/TDataScience)\n * [LinkedIn](https://www.linkedin.com/company/towards-data-science/)\n * [Threads](https://www.threads.net/@towardsdatascience)\n * [Bluesky](https://bsky.app/profile/towardsdatascience.com)\n\n\n[](https://towardsdatascience.com/)\nYour home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals.\nSubscribe to Our Newsletter[Write For TDS](https://towardsdatascience.com/submissions/)•[About](https://towardsdatascience.com/about-towards-data-science-d691af11cc2f/)•[Advertise](https://contact.towardsdatascience.com/advertise-with-towards-data-science)•[Privacy Policy](https://towardsdatascience.com/privacy-policy/)•[Terms of Use](https://towardsdatascience.com/website-terms-of-use/)\n© Insight Media Group, LLC 2026•[The New Stack](https://thenewstack.io)•[roadmap.sh](https://roadmap.sh)•[Creator Network](https://insightmediagroup.io/creator-network)\n#### Recommended Articles \nClose\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607686/image?etag=1782161464448&width=500&height=500) How Relevance Models Foreshadowed Transformers for NLP ](https://towardsdatascience.com/how-relevance-models-foreshadowed-transformers-for-nlp/ \"How Relevance Models Foreshadowed Transformers for NLP\")\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500) Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1 ](https://towardsdatascience.com/javascript-fatigue-you-dont-need-js-to-build-chatgpt/ \"Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1\")\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500) The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs ](https://towardsdatascience.com/subset-sum-problem-solved-in-linear-time-for-dense-enough-inputs/ \"The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs\")\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/3660/image?etag=1780176246070&width=500&height=500) Does Your Company Have a Data Strategy? ](https://towardsdatascience.com/does-your-company-have-a-data-strategy-e8944dbec4ae/ \"Does Your Company Have a Data Strategy?\")\n\n\n", "metadata": { "title": "Understanding Buffer of Thoughts (BoT) - Reasoning with Large Language Models | Towards Data Science", "description": "New prompt engineering tool for complex reasoning, compared with Chain of thought (CoT) and Tree of Thought (ToT)", "keywords": null, "author": "Hesam Sheikh", "og:locale": "en_US", "og:type": "article", "og:title": "Understanding Buffer of Thoughts (BoT) - Reasoning with Large Language Models", "og:description": "New prompt engineering tool for complex reasoning, compared with Chain of thought (CoT) and Tree of Thought (ToT)", "og:url": "https://towardsdatascience.com/understanding-buffer-of-thoughts-bot-reasoning-with-large-language-models-391919d2f76f/", "og:site_name": "Towards Data Science", "og:image": "https://assets.insightmediagroup.io/media/wp-content/uploads/2024/06/0Sc-px7DsflTWJAPU-scaled.jpg", "og:image:width": "2560", "og:image:height": "1920", "og:image:type": "image/jpeg", "twitter:card": "summary_large_image", "twitter:site": "@TDataScience", "twitter:creator": "@TDataScience", "twitter:label1": "Written by", "twitter:data1": "Hesam Sheikh", "twitter:label2": "Est. reading time", "twitter:data2": "10 min read", "article:published_time": "2024-06-14T22:21:57.000Z", "article:modified_time": "2024-06-14T22:21:57.000Z", "article:author": "https://towardsdatascience.com/author/itshesamsheikh/" }, "paywall": { "listed_paid": false, "detected": false, "marker": "", "public_only": false, "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙" }, "crawl": { "success": true, "status_code": 202, "error": "" }, "code_blocks": [ { "language": "txt", "file": "attachments/093822e7cfa9/code/block_000.txt", "chars": 1216 }, { "language": "txt", "file": "attachments/093822e7cfa9/code/block_001.txt", "chars": 577 }, { "language": "txt", "file": "attachments/093822e7cfa9/code/block_002.txt", "chars": 246 }, { "language": "txt", "file": "attachments/093822e7cfa9/code/block_003.txt", "chars": 1384 }, { "language": "txt", "file": "attachments/093822e7cfa9/code/block_004.txt", "chars": 127 }, { "language": "bash", "file": "attachments/093822e7cfa9/code/block_005.sh", "chars": 584 } ], "images": [ { "status": "http_404", "url": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607686/image?etag=1782161464448&width=500&height=500", "src": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607686/image?etag=1782161464448&width=500&height=500", "alt": "", "downloaded": false, "discovery": "saved_markdown", "retried_at": "2026-10-20T17:47:34Z" }, { "status": "http_404", "url": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500", "src": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500", "alt": "", "downloaded": false, "discovery": "saved_markdown", "retried_at": "2026-10-20T17:47:34Z" }, { "status": "http_404", "url": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500", "src": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500", "alt": "", "downloaded": false, "discovery": "saved_markdown", "retried_at": "2026-10-20T17:47:34Z" }, { "status": "http_404", "url": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/3660/image?etag=1780176246070&width=500&height=500", "src": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/3660/image?etag=1780176246070&width=500&height=500", "alt": "", "downloaded": false, "discovery": "saved_markdown", "retried_at": "2026-10-20T17:47:34Z" } ], "audio": [], "videos": [ { "url": "https://www.youtube.com/c/TowardsDataScience", "kind": "channel_reference", "status": "reference_only" } ], "links": [ { "type": "internal", "href": "https://towardsdatascience.com/" }, { "type": "internal", "href": "https://towardsdatascience.com/latest/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/editors-pick/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/the-variable/" }, { "type": "internal", "href": "https://towardsdatascience.com/shipai/" }, { "type": "internal", "href": "https://towardsdatascience.com/submissions/" }, { "type": "internal", "href": "https://towardsdatascience.com/category/data-science/" }, { "type": "internal", "href": "https://towardsdatascience.com/author/itshesamsheikh/" }, { "type": "internal", "href": "https://towardsdatascience.com/platonic-representation-hypothesis-c812813d7248" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/chain-of-thought/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/data-science/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/large-language-models/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/machine-learning/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/prompt-engineering/" }, { "type": "internal", "href": "https://towardsdatascience.com/graph-theory-basics-88a89863e3c1/" }, { "type": "internal", "href": "https://towardsdatascience.com/category/programming/" }, { "type": "internal", "href": "https://towardsdatascience.com/author/m-saldana/" }, { "type": "internal", "href": "https://towardsdatascience.com/implementing-ai-is-like-buying-and-driving-a-car-but-different-1e85f6afcc92/" }, { "type": "internal", "href": "https://towardsdatascience.com/author/kpeters_/" }, { "type": "internal", "href": "https://towardsdatascience.com/generating-knowledge-graphs-with-wikipedia-ec17030a40f6/" }, { "type": "external", "href": "https://write.insightmediagroup.io" }, { "type": "external", "href": "https://www.linkedin.com/company/towards-data-science/" }, { "type": "external", "href": "https://x.com/TDataScience" }, { "type": "external", "href": "https://hesamsheikh.substack.com/" }, { "type": "external", "href": "https://unsplash.com/@etiennegirardet" }, { "type": "external", "href": "https://unsplash.com" }, { "type": "external", "href": "https://arxiv.org/abs/2406.04271" }, { "type": "external", "href": "http://paper" }, { "type": "external", "href": "https://github.com/YangLing0818/buffer-of-thought-llm" }, { "type": "external", "href": "https://medium.com/@itshesamsheikh/subscribe" }, { "type": "external", "href": "https://www.facebook.com/sharer/sharer.php?title=Understanding+Buffer+of+Thoughts+%28BoT%29+-+Reasoning+with+Large+Language+Models&u=https%3A%2F%2Ftowardsdatascience.com%2Funderstanding-buffer-of-thoughts-bot-reasoning-with-large-language-models-391919d2f76f%2F" }, { "type": "external", "href": "https://www.linkedin.com/shareArticle?mini=true&title=Understanding+Buffer+of+Thoughts+%28BoT%29+-+Reasoning+with+Large+Language+Models&url=https%3A%2F%2Ftowardsdatascience.com%2Funderstanding-buffer-of-thoughts-bot-reasoning-with-large-language-models-391919d2f76f%2F" }, { "type": "external", "href": "https://x.com/share?text=Understanding+Buffer+of+Thoughts+%28BoT%29+-+Reasoning+with+Large+Language+Models&url=https%3A%2F%2Ftowardsdatascience.com%2Funderstanding-buffer-of-thoughts-bot-reasoning-with-large-language-models-391919d2f76f%2F" }, { "type": "external", "href": "https://www.youtube.com/c/TowardsDataScience" }, { "type": "external", "href": "https://www.threads.net/@towardsdatascience" }, { "type": "external", "href": "https://bsky.app/profile/towardsdatascience.com" }, { "type": "external", "href": "https://thenewstack.io" }, { "type": "external", "href": "https://roadmap.sh" }, { "type": "external", "href": "https://insightmediagroup.io/creator-network" } ], "repair_version": 2, "repaired_at": "2026-10-20T17:27:00Z", "source_url": "https://towardsdatascience.com/understanding-buffer-of-thoughts-bot-reasoning-with-large-language-models-391919d2f76f/", "delivery_status": "unverified", "discovery_status": "http_403", "missing_reason": "源页面未重新核验:http_403；已按保留正文补采图片", "repair_error": "", "actual_video_files": [], "expected_saved_body_images": 4, "image_discovery": "saved_markdown", "source_body_markdown": "![Revisit consent button](https://cdn-cookieyes.com/assets/images/revisit.svg)\nWe value your privacy\nWe use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking \"Accept All\", you consent to our use of cookies.\nCustomise Reject All Accept All\nCustomise Consent Preferences ![](https://cdn-cookieyes.com/assets/images/close.svg)\nWe use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below.\nThe cookies that are categorised as \"Necessary\" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... Show more\nNecessaryAlways Active\nNecessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.\n * Cookie\nBCTempID\n * Duration\n10 minutes\n * Description\nNo description available.\n\n\n * Cookie\n__cf_bm\n * Duration\n1 hour\n * Description\nThis cookie, set by Cloudflare, is used to support Cloudflare Bot Management. \n\n\n * Cookie\nAWSALBCORS\n * Duration\n7 days\n * Description\nAmazon Web Services set this cookie for load balancing.\n\n\n * Cookie\n_cfuvid\n * Duration\nsession\n * Description\nCloudflare sets this cookie to track users across sessions to optimize user experience by maintaining session consistency and providing personalized services\n\n\n * Cookie\nli_gc\n * Duration\n6 months\n * Description\nLinkedin set this cookie for storing visitor's consent regarding using cookies for non-essential purposes.\n\n\n * Cookie\n__hssrc\n * Duration\nsession\n * Description\nThis cookie is set by Hubspot whenever it changes the session cookie. The __hssrc cookie set to 1 indicates that the user has restarted the browser, and if the cookie does not exist, it is assumed to be a new session.\n\n\n * Cookie\n__hssc\n * Duration\n1 hour\n * Description\nHubSpot sets this cookie to keep track of sessions and to determine if HubSpot should increment the session number and timestamps in the __hstc cookie.\n\n\n * Cookie\nwpEmojiSettingsSupports\n * Duration\nsession\n * Description\nWordPress sets this cookie when a user interacts with emojis on a WordPress site. It helps determine if the user's browser can display emojis properly.\n\n\n * Cookie\nBCSessionID\n * Duration\n1 year 1 month 4 days\n * Description\nBlueconic sets this cookie as a unique identifier for the BlueConic profile.\n\n\n * Cookie\n_octo\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nlogged_in\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\n__Secure-YEC\n * Duration\npast\n * Description\nYouTube sets this cookie to stores the user's video player preferences using embedded YouTube video\n\n\n * Cookie\n__eoi\n * Duration\n6 months\n * Description\nDescription is currently not available.\n\n\n * Cookie\nAWSALBTGCORS\n * Duration\n7 days\n * Description\nNo description available.\n\n\n * Cookie\nlogin-status-p\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nAWSALBTG\n * Duration\n7 days\n * Description\nNo description available.\n\n\n * Cookie\ncsrf_token\n * Duration\nsession\n * Description\nNo description available.\n\n\n * Cookie\ntoken_v2\n * Duration\n1 day\n * Description\nDescription is currently not available.\n\n\n * Cookie\nD\n * Duration\n1 year\n * Description\nDescription is currently not available.\n\n\n * Cookie\nPHPSESSID\n * Duration\nsession\n * Description\nThis cookie is native to PHP applications. The cookie stores and identifies a user's unique session ID to manage user sessions on the website. The cookie is a session cookie and will be deleted when all the browser windows are closed.\n\n\n * Cookie\nVISITOR_PRIVACY_METADATA\n * Duration\n6 months\n * Description\nYouTube sets this cookie to store the user's cookie consent state for the current domain. \n\n\n * Cookie\ncookietest\n * Duration\nsession\n * Description\nThe cookietest cookie is typically used to determine whether the user's browser accepts cookies, essential for website functionality and user experience.\n\n\n * Cookie\n__Host-airtable-session\n * Duration\n1 year\n * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n * Cookie\n__Host-airtable-session.sig\n * Duration\n1 year\n * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n * Cookie\nm\n * Duration\n1 year 1 month 4 days\n * Description\nStripe sets this cookie for fraud prevention purposes. It identifies the device used to access the website, allowing the website to be formatted accordingly.\n\n\n * Cookie\nBIGipServer*\n * Duration\nsession\n * Description\nMarketo sets this cookie to collect information about the user's online activity and build a profile about their interests to provide advertisements relevant to the user. \n\n\n * Cookie\n__cfruid\n * Duration\nsession\n * Description\nCloudflare sets this cookie to identify trusted web traffic.\n\n\n * Cookie\n_GRECAPTCHA\n * Duration\n6 months\n * Description\nGoogle Recaptcha service sets this cookie to identify bots to protect the website against malicious spam attacks.\n\n\n * Cookie\n__Secure-YNID\n * Duration\n6 months\n * Description\nGoogle cookie used to protect user security and prevent fraud, especially during the login process.\n\n\n * Cookie\ncookieyes-consent\n * Duration\n1 year\n * Description\nCookieYes sets this cookie to remember users' consent preferences so that their preferences are respected on subsequent visits to this site. It does not collect or store any personal information about the site visitors.\n\n\nFunctional\nFunctional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.\n * Cookie\nlidc\n * Duration\n1 day\n * Description\nLinkedIn sets the lidc cookie to facilitate data center selection.\n\n\n * Cookie\nbrw\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nbrwConsent\n * Duration\n5 minutes\n * Description\nDescription is currently not available.\n\n\n * Cookie\nWMF-Uniq\n * Duration\n1 year\n * Description\nDescription is currently not available.\n\n\n * Cookie\nloom_anon_comment\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nloom_referral_video\n * Duration\nsession\n * Description\nDescription is currently not available.\n\n\n * Cookie\nVISITOR_INFO1_LIVE\n * Duration\n6 months\n * Description\nA cookie set by YouTube to measure bandwidth that determines whether the user gets the new or old player interface.\n\n\n * Cookie\nyt-remote-connected-devices\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n * Cookie\nytidb::LAST_RESULT_ENTRY_KEY\n * Duration\nNever Expires\n * Description\nThe cookie ytidb::LAST_RESULT_ENTRY_KEY is used by YouTube to store the last search result entry that was clicked by the user. This information is used to improve the user experience by providing more relevant search results in the future.\n\n\n * Cookie\nyt-remote-device-id\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n * Cookie\nyt-remote-session-name\n * Duration\nsession\n * Description\nThe yt-remote-session-name cookie is used by YouTube to store the user's video player preferences using embedded YouTube video.\n\n\n * Cookie\nyt-remote-fast-check-period\n * Duration\nsession\n * Description\nThe yt-remote-fast-check-period cookie is used by YouTube to store the user's video player preferences for embedded YouTube videos.\n\n\n * Cookie\nyt-remote-session-app\n * Duration\nsession\n * Description\nThe yt-remote-session-app cookie is used by YouTube to store user preferences and information about the interface of the embedded YouTube video player.\n\n\n * Cookie\nyt-remote-cast-available\n * Duration\nsession\n * Description\nThe yt-remote-cast-available cookie is used to store the user's preferences regarding whether casting is available on their YouTube video player.\n\n\n * Cookie\nyt-remote-cast-installed\n * Duration\nsession\n * Description\nThe yt-remote-cast-installed cookie is used to store the user's video player preferences using embedded YouTube video.\n\n\n * Cookie\ncp_session\n * Duration\n3 months\n * Description\nCodepen sets this cookie for Help systems found in the website.\n\n\n * Cookie\nloid\n * Duration\n1 year 1 month 4 days\n * Description\nThis cookie is set by the Reddit. The cookie enables the sharing of content from the website onto the social media platform.\n\n\nAnalytics\nAnalytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.\n * Cookie\n__hstc\n * Duration\n6 months\n * Description\nHubspot set this main cookie for tracking visitors. It contains the domain, initial timestamp (first visit), last timestamp (last visit), current timestamp (this visit), and session number (increments for each subsequent session).\n\n\n * Cookie\nhubspotutk\n * Duration\n6 months\n * Description\nHubSpot sets this cookie to keep track of the visitors to the website. This cookie is passed to HubSpot on form submission and used when deduplicating contacts.\n\n\n * Cookie\n_ga\n * Duration\n1 year 1 month 4 days\n * Description\nGoogle Analytics sets this cookie to calculate visitor, session and campaign data and track site usage for the site's analytics report. The cookie stores information anonymously and assigns a randomly generated number to recognise unique visitors.\n\n\n * Cookie\n_ga_*\n * Duration\n1 year 1 month 4 days\n * Description\nGoogle Analytics sets this cookie to store and count page views.\n\n\n * Cookie\n__Host-psifi.analyticsTrace\n * Duration\n6 hours\n * Description\nDescription is currently not available.\n\n\n * Cookie\n__Host-psifi.analyticsTraceV2\n * Duration\n6 hours\n * Description\nDescription is currently not available.\n\n\n * Cookie\n_gh_sess\n * Duration\nsession\n * Description\nGitHub sets this cookie for temporary application and framework state between pages like what step the user is on in a multiple step form.\n\n\n * Cookie\nYSC\n * Duration\nsession\n * Description\nYSC cookie is set by Youtube and is used to track the views of embedded videos on Youtube pages.\n\n\n * Cookie\najs_anonymous_id\n * Duration\n1 year\n * Description\nThis cookie is set by Segment to count the number of people who visit a certain site by tracking if they have visited before.\n\n\n * Cookie\nvuid\n * Duration\n1 year 1 month 4 days\n * Description\nVimeo installs this cookie to collect tracking information by setting a unique ID to embed videos on the website. \n\n\nPerformance\nPerformance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors.\n * Cookie\nAWSALB\n * Duration\n7 days\n * Description\nAWSALB is an application load balancer cookie set by Amazon Web Services to map the session to the target.\n\n\n * Cookie\nacq\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nacq.sig\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nptc\n * Duration\n2 years\n * Description\nNo description available.\n\n\nAdvertisement\nAdvertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns.\n * Cookie\nmuc_ads\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to collect user behaviour and interaction data to optimize the website.\n\n\n * Cookie\nguest_id_marketing\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n * Cookie\nguest_id_ads\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n * Cookie\npersonalization_id\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to integrate and share features for social media and also store information about how the user uses the website, for tracking and targeting.\n\n\n * Cookie\nguest_id\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor. It registers if a user is signed in to the Twitter platform and collects information about ad preferences.\n\n\n * Cookie\nbcookie\n * Duration\n1 year\n * Description\nLinkedIn sets this cookie from LinkedIn share buttons and ad tags to recognize browser IDs.\n\n\n * Cookie\n__Secure-ROLLOUT_TOKEN\n * Duration\n6 months\n * Description\nYouTube sets this cookie to manage feature rollout and experimentation. It helps Google control which new features or interface changes are shown to users as part of testing and staged rollouts, ensuring consistent experience for a given user during an experiment.\n\n\n * Cookie\nyt.innertube::nextId\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n * Cookie\nyt.innertube::requests\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n * Cookie\nsession_tracker\n * Duration\nsession\n * Description\nThis cookie is set by the Reddit. This cookie is used to identify trusted web traffic. It also helps in adverstising on the website.\n\n\n * Cookie\nedgebucket\n * Duration\nsession\n * Description\nReddit sets this cookie to save the information about a log-on Reddit user, for the purpose of advertisement recommendations and updating the content.\n\n\n * Cookie\ndid\n * Duration\n1 year\n * Description\nArbor sets this cookie to show targeted ads to site visitors.This cookie expires after 2 months or 1 year.\n\n\nUncategorised\nOther uncategorised cookies are those that are being analysed and have not been classified into a category as yet.\nNo cookies to display.\nReject All Save My Preferences Accept All\nToggle Search[](https://towardsdatascience.com/)Toggle Mobile Navigation\n[](https://towardsdatascience.com/)\nPublish AI, ML & data-science insights to a global community of data professionals.\nSign in[Submit an Article](https://write.insightmediagroup.io)\n[Latest](https://towardsdatascience.com/latest/)[Editor's Picks](https://towardsdatascience.com/tag/editors-pick/)[Newsletter](https://towardsdatascience.com/tag/the-variable/)[ShipAI](https://towardsdatascience.com/shipai/)\n* * *\n[Write For TDS](https://towardsdatascience.com/submissions/)\n[LinkedIn](https://www.linkedin.com/company/towards-data-science/)[X](https://x.com/TDataScience)Toggle Search\n[Data Science](https://towardsdatascience.com/category/data-science/)\n# Understanding Buffer of Thoughts (BoT) - Reasoning with Large Language Models\nNew prompt engineering tool for complex reasoning, compared with Chain of thought (CoT) and Tree of Thought (ToT)\n[Hesam Sheikh](https://towardsdatascience.com/author/itshesamsheikh/)\nJune 14, 202410 min read\n### A new prompting tool for complex reasoning, compared with Chain of thought (CoT) and Tree of Thought (ToT)\n> If you're not a member, [**read for free!**](https://hesamsheikh.substack.com/)**✨**\n![Photo by Etienne Girardet on Unsplash](https://assets.insightmediagroup.io/media/wp-content/uploads/2024/06/0Sc-px7DsflTWJAPU-scaled.jpg)\nPhoto by [Etienne Girardet](https://unsplash.com/@etiennegirardet?utm_source=medium&utm_medium=referral) on [Unsplash](https://unsplash.com?utm_source=medium&utm_medium=referral)\nSqueezing proficiency in complex reasoning tasks and avoiding hallucinations remains a major research topic in Large Language Models (LLMs). Despite the effort, LLMs need help with generalized reasoning capabilities. Traditional methods such as **Chain-of-Thought (CoT)** or **Tree-of-Thought (ToT)** often require multiple assumptions or numerous back-and-forth prompting which means intensive computation.\n![Buffer of Thoughts \\(BoT\\) vs other prompting methods. \\(source: Paper\\)](https://assets.insightmediagroup.io/media/wp-content/uploads/2024/06/1GOBApb9eUG3QB3uGzYIQyA-1.png)\nBuffer of Thoughts (BoT) vs other prompting methods. (source: [Paper](https://arxiv.org/abs/2406.04271))\nThe new proposed method in the paper, [**Buffer of Thoughts: Thought-Augmented Reasoning with Large Language Models**](https://arxiv.org/abs/2406.04271)**[1]** , combats these limitations with a dynamic, adaptive repository of high-level thought templates **** called** meta-buffer.**In BoT, once the user presents a new problem, it is first simplified and analyzed to extract key elements, which then guide the retrieval of a relevant thought template from a dynamic dataset. This allows adaptive and efficient problem-solving through modified and complex reasoning patterns. According to the original paper, this is so effective that _** \"Llama3–8B+BoT**has the potential to surpass** Llama3–70B model.**_\"\n**BoT** achieves efficient reasoning across problems that are similar to its templates as it:\n * (1) leverages previous solutions on new challenges,\n * (2) boosts efficiency by eliminating the need for multiple query iterations (as we see in Graph-of-Thoughts (GoT) or ToT), and\n * (3) dynamically updates its template repository to ensure it evolves as it encounters new tasks.\n\n\nIn this article, we will first go through the general outline of how BoT works, understand the function of each key part, and test the procedure with an example.\n## How does BoT work?\nThe general thought-augmented reasoning process (as shown in the figure below) starts with **Problem Distillation** , which analyzes and condenses the incoming task into essential elements and constraints and then creates a simplified problem statement.\nThis distilled information is then used to query the **Meta-Buffer** , a dynamic repository that contains high-level thought templates. From the thought templates, one that is most similar to the distilled problem is retrieved. Then, during the **Instantiation Process** , it is instantiated with specific requirements and information about the distilled problem.\nThroughout this process, the **Buffer Manager** actively keeps an eye on the **Meta-Buffer.** Once it detects a new insight not included in the meta-buffer, **Buffer Manager** updates it to ensure a continual evolution of the thought template repository.\n![The BoT process. \\(source: Paper\\)](https://assets.insightmediagroup.io/media/wp-content/uploads/2024/06/1uMSmql9X-whVPYKijsTNVQ-1.png)\nThe BoT process. (source: [Paper](http://Paper))\nLet's go through each of these key parts to gain a more detailed look:\n### Problem Distiller\nProblem Distiller can be thought of as a **preprocess** on the input tasks in order to...\n * (1) extract essential information of the problem, and\n * (2) simplify complex tasks for a better search and retrieval of thought templates.\n\n\nProblem Distiller takes the burden off of LLM to identify and extract vital information and constraints of the problem. This is done by a meta prompt ϕ:\n![\\(source: Paper\\)](https://assets.insightmediagroup.io/media/wp-content/uploads/2024/06/1MC3YflNqZSP3q4lMmgqVEA-1.png)\n(source: [Paper](http://Paper))\nThe prompt used by the authors to distill key information about a task is as follows:\nvbnet\n\n```\n[Problem Distiller]:As a highly professional and intelligent expert in information distillation, you excel atextracting essential information to solve problems from user input queries. You adeptlytransform this extracted information into a suitable format based on the respective type of theissue.Please categorize and extract the crucial information required to solve the problem from theuser's input query, the distilled information should include.1. Key information:Values and information of key variables extracted from user input, which will be handed overto the respective expert for task resolution, ensuring all essential information required to solvethe problem is provided.2. Restrictions:The objective of the problem and corresponding constraints.3. Distilled task:Extend the problem based on 1 and 2, summarize a meta problem that can address the userquery and handle more input and output variations. Incorporate the real-world scenario of theextended problem along with the types of key variables and information constraints from theoriginal problem to restrict the key variables in the extended problem. After that, use the userquery input key information as input to solve the problem as an example.\n```\n\n### Meta-Buffer\nThe meta-buffer is a central database that stores high-level thought templates. These templates are high-level abstractions representing various problem-solving processes. The idea is that LLM can leverage past problems and insights to solve current challenges. The best part is that the Meta-Buffer dynamically updates to ensure new unseen problems are also included. The Meta-Buffer doesn't enforce thought templates to follow a specific instruction.\n**Template Retrieval** : Once a task is distilled, BoT would go through the thought templates and grab the one most similar to the task. This is done by calculating the embedding similarity between the task and the thought templates.\n![The retriever would calculate the similarity between the embedding of the input task f\\(xd\\), and the embedding of templates f\\(DTi \\). \\(source: Paper\\)](https://assets.insightmediagroup.io/media/wp-content/uploads/2024/06/1_F34rCJ41VnsLqlBBvkG4A-1.png)\nThe retriever would calculate the similarity between the embedding of the input task f(xd), and the embedding of templates f(DTi ). (source: [Paper](https://arxiv.org/abs/2406.04271))\nthe retriever would calculate the similarity between the embedding of the input task **f(xd)** , and the embedding of templates **f(D** _**Ti**_**).** This is only done if the similarity is above a certain threshold δ (0.5–0.7). If none of the thought templates have a similarity score with the task above the δ threshold, then the **xd** is identified as a new task. Depending on if the task is new or not, one of the two paths would be taken:\n * If the task is similar to one of the thought templates, the template would be instantiated with the distilled information using an instantiation prompt (which you can check in the paper). This instantiation process can be denoted as\n\n\n![Instantiated Reasoning. \\(source: Paper\\)](https://assets.insightmediagroup.io/media/wp-content/uploads/2024/06/1vsrdxG7bXs5HAx1YdApGMA-1.png)\nInstantiated Reasoning. (source: [Paper](https://arxiv.org/abs/2406.04271))\n * If the task is new, a general thought template that is designed to address a broad set of problems is used. As the task is processed, the **Buffer Manager** observes and learns and potentially creates a new, more specific thought template and pushes it to the meta-buffer.\n\n\n### Buffer Manager\nThe Buffer Manager serves as a crucial player in maintaining and enhancing the meta-buffer. Based on the new insights and outcomes from the tasks that are solved, it updates the thought templates. Also, whenever a new or drastically different problem is solved, the buffer manager assesses whether or not to create a new thought template. This is to ensure thought templates remain to the point and are not redundant.\n![checking whether or not a newly generated template is similar to the existing ones. \\(source: Paper\\)](https://assets.insightmediagroup.io/media/wp-content/uploads/2024/06/14A8XsjDczL31_1DXOXpZSA-1.png)\nchecking whether or not a newly generated template is similar to the existing ones. (source: Paper)\nBy employing the above formulation, the Buffer Manager checks whether or not the meta-buffer already has the necessary knowledge to tackle a problem.\n### BoT vs. Single-Query vs. Multi-Query\nHow does the BoT stand out compared to previous methods? The authors of the paper evaluate various methods on different datasets of various tasks, such as Data Understanding, Python Programming Puzzles, Multilingual Grade School Math **** (MGSM), etc. The results show a surprising advantage of** BoT** in almost all the tasks.\n![BoT vs previous methods. The best results \\(marked in blue\\) are all achieved by BoT. \\(source: Paper\\)](https://assets.insightmediagroup.io/media/wp-content/uploads/2024/06/1Ht4i0Hf9zMEHkfFKkGysXQ-1.png)\nBoT vs previous methods. The best results (marked in blue) are all achieved by BoT. (source: [Paper](https://arxiv.org/abs/2406.04271))\nOne of the key advantages of BoT is its efficiency - requiring **only 12%** of the computational cost on average when compared to multi-query prompting methods. This high computational cost and latency of multi-query methods such as ToT often renders them impractical in real-world use cases.\n> BoT+Llama3–8B has the potential to surpass single Llama3–70B model\n![comparing the effect of BoT on Llama3–8B and 70B. Annotated. \\(source: Paper\\)](https://assets.insightmediagroup.io/media/wp-content/uploads/2024/06/10rgW6uKTyvHkf9AK-Gh9Mw-1.png)\ncomparing the effect of BoT on Llama3–8B and 70B. Annotated. (source: [Paper](https://arxiv.org/abs/2406.04271))\n## Buffer of Thoughts in Practice\nThe demo code for Buffer of Thoughts is published on [GitHub](https://github.com/YangLing0818/buffer-of-thought-llm) [2]. To test out the functionality in practice, I will use this method on a custom task: Word Reordering. In this task, the LLM must take a scrambled sentence, such as _\"Sam name is my\"_ , and return a permutation of these words that is semantically meaningful, which in this example would be _\"my name is Sam\",_ (this is not a benchmark with baseline performance). Some examples of the scrambled sentences and the correct ones are as follows:\njson\n\n```\n{\"input\": \" life plan and families to for social hospital workers outside with patients work the \", \n\"target\": \" social workers work with patients and families to plan for life outside the hospital \"}\n{\"input\": \" yield plant refers dry total to production biological matter \", \n\"target\": \" biological yield refers to total plant dry matter production \"}\n{\"input\": \" the bloodstream into alcohol from directly stomach goes the \", \n\"target\": \" alcohol goes directly from the stomach into the bloodstream \"}\n```\n\nI will also use the following user prompt to explain the task:\ntext\n\n```\nReorder a list of words to make them semantically meaningful, \nfor example \"Sam name is my\" -> \"my name is Sam\". \nPlease provide only the reordered sentence without any explanations. \nStart the ordered sentence with and end it with \n```\n\nProvided this task, the BoT framework first distills my task and extracts the goal, the restriction, and how I would like my output along with other key information. The full procedure of BoT in detail, is the following terminal output:\nbash\n\n```\nUser prompt:\nReorder a list of words to make them semantically meaningful, for example \"Sam name is my\" -> \"my name is Sam\". Please provide only the reordered sentence without any explanations. Start the ordered sentence with and end it with \nInput:\n the melting in solid to gold leaf metals is achieve made by desired gold and mixing color other \nDistilled information:Meta distiller Respond:\n\nDistilled Information:\n\n1. Key information:\n- Input: \" the melting in solid to gold leaf metals is achieve made by desired gold and mixing color other \"\n\n2. Restriction:\n- Reorder the list of words such that the sentence is semantically meaningful\n- The output should begin with \"\" and end with \"\"\n\n3. Distilled task:\n- The task is to reorder the words to create a meaningful sentence from the provided mixed words inside the tags \"\" and \"\"\n\n4. Python transformation:\n (Optional, skip when Python tag is Not for Python) Input parameters:\n input_sentence = \" the melting in solid to gold leaf metals is achieve made by desired gold and mixing color other \"\n\n5. Answer form: (Optional, skip when there is no specific answer form)\n The output sentence should be in the form of \" ... \"\n\n **Note: The generation ends here. Do not show this message in your answer !**\nInstantiated reasoning result: ```python\ndef ReorderWords(input_sentence):\n # Remove the start and end tags, and split the sentence into words\n words = input_sentence.replace(\"\", \"\").replace(\"\", \"\").strip().split()\n\n # Tentative reordering to form a meaningful sentence\n # Based on the context provided in the task, the following reordering seems logical\n reordered_words = [\"the\", \"melting\", \"in\", \"solid\", \"metals\", \"is\", \"achieved\", \"by\", \"mixing\", \"gold\", \"leaf\", \"and\", \"desired\", \"other\", \"color\", \"gold\"]\n\n # Join the words into a single sentence and add start and end tags\n reordered_sentence = \" \" + \" \".join(reordered_words) + \" \"\n\n return reordered_sentence\n\n# Input sentence provided in the task\ninput_sentence = \" the melting in solid to gold leaf metals is achieve made by desired gold and mixing color other \"\nprint(ReorderWords(input_sentence))\n```\n\nThe result of code execution: the melting in solid metals is achieved by mixing gold leaf and desired other color gold \ntext\n\n```\n\nSome of the examples of the reordered sentences using BoT:\n\n```bash\n{\"input\": \" life plan and families to for social hospital workers outside with patients work the \", \n\"result\": \" Hospital workers work outside with social patients and plan to the families for life n\"}\n{\"input\": \" yield plant refers dry total to production biological matter \", \n\"result\": \" Plant yield refers to total dry matter biological production n\"}\n{\"input\": \" the bloodstream into alcohol from directly stomach goes the \", \n\"result\": \" the alcohol goes directly from the stomach into the bloodstream n\"}\n```\n\nNote, that the BoT repository I used is a demo code and lacks some of the features mentioned in the original paper, such as a general thought template, the dynamic updating of the Meta-Buffer, or finding the closest template embedding to the user task. These are important aspects of the framework and without them, we cannot conclude the performance of Buffer of Thoughts in practice.\n## Final Words\nIn conclusion, BoT shows promising results in both accuracy and efficiency in various domains and tasks. It's an interesting approach to breaking down a reasoning problem into its fundamental restrictions and key information and building on top of previous solutions and templates to better formulate the task for an LLM to understand.\nBy addressing some of the limitations in other prompting techniques, Buffer of Thoughts allows LLM to have more complex thinking patterns, potentially making smaller lightweight models perform at the level of larger ones.\nAllowing small LLMs to achieve results close to large LLMs is a key topic addressed in many of the current research papers. The goal is to employ various prompting and fine-tuning techniques to extract accurate AI outputs with low computation and cost.\nBuffer of Thoughts is a novel and promising prompting framework that leverages a domain of techniques to guide LLM step by step, in a reasoning process. A complete practical implementation of the Buffer of Thoughts technique is yet to come, but in the meanwhile, test out the provided benchmarks in the demo GitHub repository [2].\n* * *\nIf you have made it this far, consider this for a **further read** :\n> [**Platonic Representation: Are AI Deep Network Models Converging?**](https://towardsdatascience.com/platonic-representation-hypothesis-c812813d7248)\n**🌟 Join +1000 people learning about** Python, ML / MLOps / AI, Data Science, and LLM. [**follow me**](https://medium.com/@itshesamsheikh/subscribe) and check out my **X/Twitter** , where I keep you updated Daily**.**\nThanks for reading,\n— Hesam\n* * *\n[1] Yang, L., Yu, Z., Zhang, T., Cao, S., Xu, M., Zhang, W., Gonzalez, J. E., & Cui, B. (2024). Buffer of Thoughts: Thought-Augmented Reasoning with Large Language Models. _arXiv_. Retrieved from \n[2] buffer-of-thought-llm, \n## Register Now to Continue Reading\n### Unlock Towards Data Science with a Free Account\nAll articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more.\nRegister for freeSign in to your account\n* * *\n![](https://assets.insightmediagroup.io/media/1787657497520_1vlj6x.png)\nWritten By\n## [Hesam Sheikh](https://towardsdatascience.com/author/itshesamsheikh/)\n[See all from Hesam Sheikh](https://towardsdatascience.com/author/itshesamsheikh/)\n[Chain Of Thought](https://towardsdatascience.com/tag/chain-of-thought/)[Data Science](https://towardsdatascience.com/tag/data-science/)[Large Language Models](https://towardsdatascience.com/tag/large-language-models/)[Machine Learning](https://towardsdatascience.com/tag/machine-learning/)[Prompt Engineering](https://towardsdatascience.com/tag/prompt-engineering/)\nShare This Article\n * [](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Ftowardsdatascience.com%2Funderstanding-buffer-of-thoughts-bot-reasoning-with-large-language-models-391919d2f76f%2F&title=Understanding%20Buffer%20of%20Thoughts%20\\(BoT\\)%20-%20Reasoning%20with%20Large%20Language%20Models)\n * [](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Ftowardsdatascience.com%2Funderstanding-buffer-of-thoughts-bot-reasoning-with-large-language-models-391919d2f76f%2F&title=Understanding%20Buffer%20of%20Thoughts%20\\(BoT\\)%20-%20Reasoning%20with%20Large%20Language%20Models)\n * [](https://x.com/share?url=https%3A%2F%2Ftowardsdatascience.com%2Funderstanding-buffer-of-thoughts-bot-reasoning-with-large-language-models-391919d2f76f%2F&text=Understanding%20Buffer%20of%20Thoughts%20\\(BoT\\)%20-%20Reasoning%20with%20Large%20Language%20Models)\n\n\nTowards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program.\n[Write for TDS](https://towardsdatascience.com/submissions/)\n## Related Articles\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/0aFDSzfrgmujhEgY3-scaled.jpg)](https://towardsdatascience.com/graph-theory-basics-88a89863e3c1/)\n## [Graph Theory Basics](https://towardsdatascience.com/graph-theory-basics-88a89863e3c1/)\n[Programming](https://towardsdatascience.com/category/programming/)\nWhat you need to know as graph theory adoption continues to take off\n[Miguel Saldana](https://towardsdatascience.com/author/m-saldana/)\nMarch 22, 20218 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2023/01/1KRgO_CpskIPXzJtL6J6xbQ.png)](https://towardsdatascience.com/implementing-ai-is-like-buying-and-driving-a-car-but-different-1e85f6afcc92/)\n## [Implementing AI is like buying and driving a car (but different)](https://towardsdatascience.com/implementing-ai-is-like-buying-and-driving-a-car-but-different-1e85f6afcc92/)\n[Data Science](https://towardsdatascience.com/category/data-science/)\nAn analogy to help explain common myths and pitfalls to management\n[Koen Peters](https://towardsdatascience.com/author/kpeters_/)\nJanuary 18, 20238 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2023/01/0vd2LCu4kzycAz-R6.png)](https://towardsdatascience.com/generating-knowledge-graphs-with-wikipedia-ec17030a40f6/)\n## [Generating Knowledge Graphs with Wikipedia](https://towardsdatascience.com/generating-knowledge-graphs-with-wikipedia-ec17030a40f6/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nA Python Guide to Quickly Generate Knowledge Graphs\n[Jye Sawtell-Rickson](https://towardsdatascience.com/author/jyesr/)\nJanuary 1, 20235 min read\n * [![](https://assets.insightmediagroup.io/media/1789309276738_iqe21i.jpg)](https://towardsdatascience.com/google-offered-10m-for-a-dying-airlines-data-how-can-you-value-yours/)\n## [Google Offered $10M for a Dying Airline's Data. How Can You Value Yours?](https://towardsdatascience.com/google-offered-10m-for-a-dying-airlines-data-how-can-you-value-yours/)\n[Data Governance](https://towardsdatascience.com/category/data-science/data-governance/)\nHow to put a dollar value on your organization's operational data\n[Thuwarakesh Murallie](https://towardsdatascience.com/author/thuwarakesh/)\nSeptember 21, 20268 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2020/09/0h6xNbUWZfiZQfQvs-scaled.jpg)](https://towardsdatascience.com/the-insiders-guide-to-generative-and-discriminative-machine-learning-models-34d5791d53d3/)\n## [What are the differences between generative and discriminative machine learning models?](https://towardsdatascience.com/the-insiders-guide-to-generative-and-discriminative-machine-learning-models-34d5791d53d3/)\n[Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/)\nGenerative and discriminative are classes of machine learning models.\n[Saurav Singla](https://towardsdatascience.com/author/sauravsingla08/)\nSeptember 12, 20206 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2022/10/0GpMI4iPGbmE1eS3C-scaled.jpg)](https://towardsdatascience.com/how-to-use-pandas-to-get-your-data-in-the-format-you-need-dc7fd63ab5e9/)\n## [How to Use Pandas to Get Your Data in the Format You Need](https://towardsdatascience.com/how-to-use-pandas-to-get-your-data-in-the-format-you-need-dc7fd63ab5e9/)\n[Programming](https://towardsdatascience.com/category/programming/)\nLearn the difference between long-form and wide-form data, and how to transition between them in Pandas.\n[Murtaza Ali](https://towardsdatascience.com/author/murtaza5152-ali/)\nOctober 13, 20225 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/05/0CgZh_-BHckEcC8Xk.png)](https://towardsdatascience.com/the-price-of-power-part-2-ebbe22b0ce07/)\n## [The Price of Power, Part 2](https://towardsdatascience.com/the-price-of-power-part-2-ebbe22b0ce07/)\n[Data Science](https://towardsdatascience.com/category/data-science/)\nRetail Power Prices in Texas: Analyzing the Whole Market in Context This is the second...\n[Dale Furrow](https://towardsdatascience.com/author/dkfurrow/)\nMay 6, 20213 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2023/04/1sd7-6RMKjgi_Z413Si-szw.jpeg)](https://towardsdatascience.com/introduction-to-knowledge-graph-based-recommender-systems-34254efd1960/)\n## [Introduction to Knowledge Graph-Based Recommender Systems](https://towardsdatascience.com/introduction-to-knowledge-graph-based-recommender-systems-34254efd1960/)\n[Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/)\nA brief presentation of KG Recommender Systems Families\n[Amine Dadoun](https://towardsdatascience.com/author/amine.dadoun/)\nApril 1, 20237 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2023/04/1rsp22rKwFDjiwwCcUly56Q-scaled.jpeg)](https://towardsdatascience.com/how-i-turned-my-companys-docs-into-a-searchable-database-with-openai-4f2d34bd8736/)\n## [How I Turned My Company's Docs into a Searchable Database with OpenAI](https://towardsdatascience.com/how-i-turned-my-companys-docs-into-a-searchable-database-with-openai-4f2d34bd8736/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nAnd how you can do the same with your docs\n[Jacob Marks, Ph.D.](https://towardsdatascience.com/author/jacob_marks/)\nApril 25, 202315 min read\n\n\n * [YouTube](https://www.youtube.com/c/TowardsDataScience)\n * [X](https://x.com/TDataScience)\n * [LinkedIn](https://www.linkedin.com/company/towards-data-science/)\n * [Threads](https://www.threads.net/@towardsdatascience)\n * [Bluesky](https://bsky.app/profile/towardsdatascience.com)\n\n\n[](https://towardsdatascience.com/)\nYour home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals.\nSubscribe to Our Newsletter[Write For TDS](https://towardsdatascience.com/submissions/)•[About](https://towardsdatascience.com/about-towards-data-science-d691af11cc2f/)•[Advertise](https://contact.towardsdatascience.com/advertise-with-towards-data-science)•[Privacy Policy](https://towardsdatascience.com/privacy-policy/)•[Terms of Use](https://towardsdatascience.com/website-terms-of-use/)\n© Insight Media Group, LLC 2026•[The New Stack](https://thenewstack.io)•[roadmap.sh](https://roadmap.sh)•[Creator Network](https://insightmediagroup.io/creator-network)\n#### Recommended Articles \nClose\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607686/image?etag=1782161464448&width=500&height=500) How Relevance Models Foreshadowed Transformers for NLP ](https://towardsdatascience.com/how-relevance-models-foreshadowed-transformers-for-nlp/ \"How Relevance Models Foreshadowed Transformers for NLP\")\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500) Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1 ](https://towardsdatascience.com/javascript-fatigue-you-dont-need-js-to-build-chatgpt/ \"Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1\")\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500) The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs ](https://towardsdatascience.com/subset-sum-problem-solved-in-linear-time-for-dense-enough-inputs/ \"The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs\")\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/3660/image?etag=1780176246070&width=500&height=500) Does Your Company Have a Data Strategy? ](https://towardsdatascience.com/does-your-company-have-a-data-strategy-e8944dbec4ae/ \"Does Your Company Have a Data Strategy?\")\n\n\n", "body_base_path": "attachments/093822e7cfa9/pages/", "body_markdown_path": "attachments/093822e7cfa9/pages/archived-body.md", "local_html": "attachments/093822e7cfa9/pages/archived-body.html", "body_source": "prior_capture" }