{
  "source_file": "0930",
  "source_platform": "Substack",
  "author": "Deep (Learning) Focus\nCameron R. Wolfe",
  "title_listed": "AI Agents from First Principles",
  "keywords": "AI Agent, 第一性原理, LLM作为策略, 工具使用, 自主规划, Agent架构",
  "domain": "大模型算法",
  "content_type": "深度技术解析",
  "is_paid_flag": "免费",
  "url": "https://cameronrwolfe.substack.com/p/ai-agents",
  "publish_date": "2025-06",
  "corpus_stage": "Agent",
  "notes": "从最基本概念出发构建AI Agent理解：LLM→工具→推理→自主规划的完整技术链条",
  "id": "321882e27e41",
  "collected_at": "2026-10-01T20:28:20",
  "title": "AI Agents from First Principles",
  "body_markdown": "(from [1] and\nsource\n)\nThe capabilities of large language models (LLMs) are advancing rapidly. As LLMs become more capable, we can use them to create higher-level systems that solve increasingly complex problems, interact with external environments and operate over longer time horizons—\nthese are referred to as AI agent systems\n. AI agents are a popular topic, but there is considerable confusion regarding the definition and capabilities of these agents. In this overview, we will build an understanding of AI agents from first principles. Starting with a standard text-to-text LLM, we will explore how functionalities like tool usage, reasoning and more can enhance a standard LLM, leading to the creation of complex, autonomous systems.\nLLMs and their Capabilities\nThe input-output signature of a standard LLM\nThe functionality of an LLM is depicted above. Given a textual prompt, the LLM generates a textual response. This functionality is easy to understand and can be generalized to solve nearly any problem. In many ways, the generality of an LLM is one of its biggest strengths. In this section, we will outline how new capabilities—\nsuch as reasoning or interacting with external APIs—\ncan be integrated into an LLM by taking advantage of this text-to-text structure. As we will soon learn, advanced capabilities of modern AI agents are largely built upon this basic functionality.\nTool Usage\nAs LLMs started to become more capable, teaching them how to integrate with and use external tools quickly became a popular topic in AI research. Examples of useful tools that can be integrated with an LLM include calculators, calendars, search engines, code interpreters and more.\nWhy is this approach so popular?\nPut simply, LLMs are (obviously) not the best tool for solving all tasks. In many cases, simpler and more reliable tools are available; e.g., calculators for performing basic arithmetic or search engines for getting up-to-date factual info on a certain topic. Given that LLMs excel in planning and orchestration, however, we can easily teach them how to use these tools as part of their problem solving process!\nThe fundamental idea behind tool-use LLMs is endowing an LLM with the ability to delegate sub-tasks or components of a problem to a more specialized or robust tool. The LLM serves as the “brain” that orchestrates various specialized tools together.\nTool usage examples (from [2, 3])\nFinetuning for tool usage.\nEarly work on tool use—\ne.g., LaMDA [2] or the Toolformer [3] (depicted above)\n—used targeted finetuning to teach an LLM how to leverage a fixed set of tools. We simply curate training examples where a function call to some tool is directly inserted into the LLM’s token stream; see below.\nStructure of a tool call\nDuring training, these tool calls are treated similarly to any other token—\nthey are just part of the textual sequence\n! When a call to a tool is generated by the LLM at inference time, we handle it as follows:\nStop generating tokens.\nParse the tool call (i.e., determine the tool being used and its parameters).\nMake a call to the tool with these parameters.\nAdd the response from the tool to the LLM’s token stream.\nContinue generating tokens.\nThe tool call can be handled in real-time as the LLM generates its output, and the information returned by the tool is added directly into the model’s context!\nPrompt-based tool usage.\nTeaching LLMs to call tools via finetuning requires curating—\nusually with human annotation\n—a large training dataset. As LLM capabilities improved, later work instead emphasized in-context learning-based approaches for tool usage.\nWhy would we finetune a language model when we can simply explain the tools that are available in the model’s prompt?\n(from [4, 5])\nPrompt-based tool usage requires less human effort, allowing us to drastically increase the number of tools to which LLMs have access. For example, later work in this space integrates LLMs with hundreds [4] or even thousands [5] of tools; see above. To do this, we treat each tool as a generic API and provide the schema for relevant APIs as context in the model’s prompt. This approach enables LLMs to be integrated with arbitrary APIs on the internet using a standardized structure, which makes countless applications possible; e.g., finding information, calling other ML models, booking a vacation, handling your calendar and much more.\n“Today, we're open-sourcing the\nModel Context Protocol\n(MCP), a new standard for connecting AI assistants to the systems where data lives, including content repositories, business tools, and development environments. Its aim is to help frontier models produce better, more relevant responses.”\n- from [15]\nModel context protocol (MCP)\n—\nproposed by Anthropic\n—is a popular framework that extends upon the idea of allowing LLMs to interact with arbitrary tools. Put simply, MCP standardizes the format used by external systems to provide context into the prompt of an LLM. To solve complex problems,\nLLMs will need to integrate with a progressively larger set of external tools over time\n. To streamline this process, MCP proposes a standard format for these integrations and allows developers to create pre-built integrations, called MCP servers, that can be used by any LLM to connect with a variety of custom data sources; see below.\nDepiction of the general architecture for MCP (\nsource\n)\nFor those who are interested in digging deeper into tool usage, please see the following series of overview on this topic:\nFinetuning LLMs to use tools [\nlink\n]\nPrompt-based tool usage [\nlink\n]\nIntegrating LLMs with code interpreters [\nlink\n]\nAllowing LLMs to create their own tools [\nlink\n]\nLimitations of tool usage.\nDespite the power of the tool usage paradigm, the capabilities of tool-use LLMs are limited by their reasoning capabilities. To effectively leverage tools, our LLM must be able to:\nDecompose complex problems into smaller sub-tasks.\nDetermine what tools should be used to solve a problem.\nReliably craft calls to relevant tools with the correct format.\nComplex tool usage requires the LLM to be an effective orchestrator, which is very dependent upon the model’s reasoning capabilities and overall reliability.\nReasoning Models\nGiven the relationship between agency and reasoning, reasoning capabilities have been a core focus of LLM research for several years. For a more in-depth overview of current reasoning research, please see the overview below. However, we will briefly cover the key ideas behind reasoning models here for completeness.\nDemystifying Reasoning Models\nCameron R. Wolfe, Ph.D.\n·\nFebruary 18, 2025\nRead full story\nCoT prompting.\nWhen LLMs first became popular, one of the most common criticisms of these models was that they could not perform complex reasoning. However, research on\nChain of Thought (CoT) prompting\n[6, 7] revealed that vanilla LLMs are better at reasoning than we initially realized. The idea behind CoT prompting is simple. Instead of directly prompting an LLM for output, we ask it to generate a rationale or explanation prior to its final output; see below.\n(from [7])\nInterestingly, this approach drastically improves the performance of vanilla LLMs on reasoning tasks, indicating that LLMs are capable of complex reasoning—\nto a reasonable extent\n—if we can find the correct approach to elicit these capabilities.\nReasoning models.\nCoT prompting is incredibly effective and is a core part of all modern LLMs; e.g., ChatGPT usually outputs a CoT with its answers by default. However, this approach to reasoning is also somewhat naive. The entire reasoning process revolves around the CoT generated by the LLM and there is no dynamic adaptation based on the complexity of the problem being solved.\n(\nsource\n)\nTo solve these issues, recent research has introduced new training strategies to create LLMs that specialize in reasoning (i.e., reasoning models). These models approach problem solving differently compared to standard LLMs—\nthey spend a variable amount of time “thinking” prior to providing an answer to a question\n.\n(from [8])\nThe thoughts of a reasoning model are just standard chains of thought, but a CoT from a reasoning model is much longer than that of a standard LLM (i.e., can be several thousands of tokens), tends to exhibit complex reasoning behavior (e.g., backtracking and self-refinement) and can dynamically adapt based on the difficulty of the problem being solved—\nharder problems warrant a longer CoT\n.\nThe key advancement that made reasoning models possible was large-scale post-training with\nreinforcement learning from verifiable rewards (RLVR)\n; see above. If we have a dataset of ground truth solutions to verifiable problems (e.g., Math or coding), we can simply check whether the answer generated by the LLM is correct and use this signal to train a model with RL. During this training process, reasoning models naturally learn how to generate long chains of thought to solve verifiable reasoning problems via RL-powered self-evolution.\n“We explore the potential of LLMs to develop reasoning capabilities without any supervised data, focusing on their self-evolution through a pure reinforcement learning process.”\n- from [8]\nReasoning trajectories.\nIn summary, reasoning models, which are trained via large-scale post-training with RLVR, change the behavior of a standard LLM as shown below. Instead of directly generating output, the reasoning model first generates an arbitrarily long CoT\n1\nthat decomposes and solves the reasoning task—\nthis is the “thinking” process\n. We can change how much the model thinks by controlling the length of this reasoning trace; e.g., the\no-series\nof reasoning models from OpenAI provide low, medium and high levels of reasoning effort.\nThe input-output signature of a reasoning model\nAlthough the model still generates a single output given a prompt, the reasoning trajectory implicitly demonstrates a variety of advanced behaviors; e.g., planning, backtracking, monitoring, evaluation and more. For examples of these reasoning trajectories and their properties, see the\nSynthetic-1 dataset\n, which contains over 2M examples of reasoning traces generated by\nDeepSeek-R1\n.\nReasoning + agents.\nGiven recent advancements in reasoning,\na sufficiently capable LLM that can plan and effectively reason over its instructions should be able to decompose a problem, solve each component of the problem and arrive at a final solution itself. Providing LLMs with more autonomy and relying on their capabilities—\nrather than human intervention\n—to solve complex problems is a key idea behind agent systems. To make the idea of an agent more clear, let’s now discuss a framework that can be used to design these types of systems.\nThe ReAct Framework\n[1]\n“It is becoming more evident that with the help of LLMs, language as a fundamental cognitive mechanism will play a critical role in interaction and decision making.”\n- from [1]\nReAct [1]—\nshort for\nRE\nasoning and\nACT\nion\n—is one of the first general frameworks to be proposed for autonomously decomposing and solving complex problems with an LLM agent. We can think of ReAct as a sequential, multi-step problem-solving process powered by an LLM at its core. At each time step\nt\n, the LLM incorporates any feedback that is available and considers the current state of the problem it is trying to solve, allowing it to effectively reason over and select the best possible course of action for the future. Given that (nearly) any LLM system can be modeled sequentially, ReAct is a generic and powerful framework.\nCreating a Framework for Agents\nAt a particular time step\nt\n, our agent is given an observation from its environment\no_t\n. Based upon this observation, our agent will decide to take some action\na_t\n, which may be intermediate—\nsuch as searching the web to find data that is needed to solve a problem\n—or terminal (i.e., the final action that “solves” the problem of interest). We define the function that our agent uses to produce this action as a policy\nπ\n2\n. The policy takes the context—\na concatenated list of prior actions and observations from the agent\n—as input and predicts the next action\na_t\nas output, either deterministically or stochastically\n3\n. As depicted below, this loop of observations and actions continues until our agent outputs a terminal action.\nThe observation-action loop for agents\nReAct [1] makes one key modification to the observation-action loop shown above. The space of potential actions that can be outputted by the policy\nA\ntypically includes the set of intermediate and terminal actions that can be taken by the agent; e.g., searching for data on the web or outputting a final solution to a problem. However, ReAct expands the action space to include language, allowing the agent to produce a textual output as an action instead of taking a traditional action. In other words,\nthe agent can choose to “think”\n; see below.\nThe ReAct framework\nFormally, we can define a thought as a special kind of action as shown above. As one might infer from the name of the framework, the primary motivation behind ReAct is finding a balance between reasoning and action. Similarly to a human, the agent should be able to think and plan the actions that it takes in an environment—\nreasoning and action have a symbiotic relationship\n.\n“Reasoning traces help the model induce, track, and update action plans, while actions allow it to interface with and gather additional information from external sources such as knowledge bases or environments.”\n- from [1]\nHow do agents think?\nExample action space for an agent\nThe traditional action space for an agent is discrete and—\nin most cases\n—relatively small. For example, an agent specialized in question-answering could have the following options for actions (depicted above):\nPerform a Google search to retrieve relevant webpages.\nGrab relevant information from a particular webpage.\nReturn a final answer.\nThere are only so many actions that this agent can take while working towards a solution. In contrast, the space of language is virtually unlimited. As a result, the ReAct framework requires the use of a strong language model as its policy. In order to produce useful thoughts that benefit performance, the LLM backend of our agent system must possess advanced reasoning and planning capabilities!\n“Learning in this augmented action space is difficult and requires strong language priors… we mainly focus on the setup where a frozen large language model… is prompted with few-shot in-context examples to generate both domain-specific actions and free-form language thoughts for task solving.”\n- from [1]\nThought patterns.\nCommon examples of useful thought patterns that can be produced by an agent include decomposing tasks, creating itemized action plans, tracking progress toward a final solution, or simply outputting information—\nfrom the implicit knowledge base of the LLM\n—that may be relevant to solving a problem.\nAgents use their thinking ability to explicitly describe how a problem should be solved and then execute—\nand monitor the execution of\n—this plan. In both of the examples above, the agent explicitly writes out the next steps that it needs to perform when solving a problem; e.g.,\n“Next, I need to…”\nor\n“I need to search…”\n.\nA few-shot example given to a ReAct agent (from [1])\nIn most cases, the thoughts produced by an agent—\ncommonly referred to as a problem or task-solving trajectory\n—mimic that of a human trying to solve a problem. In fact, experiments with ReAct in [1] guide the agent’s approach to a problem by providing\nin-context examples\nof task-solving trajectories (i.e., actions, thoughts and observations) used by humans to solve similar problems. Agents prompted in this fashion are likely adopt a human-like reasoning process.\n“We let the language model decide the asynchronous occurrence of thoughts and actions for itself.”\n- from [1]\nWhen should the agent think?\nDepending on the problem we are solving, the ReAct framework can be setup differently. For reasoning heavy tasks, thoughts are typically interleaved with actions—\nwe can hard-code the agent such that it produces a single thought before every action\n. However, the agent can also be given the ability to determine for itself whether thinking is necessary. For tasks that require a lot of actions (i.e., decision-making tasks), the agent may choose to use thoughts more sparsely within its problem-solving trajectory.\nConcrete Use Cases\n(from [1])\nTwo use cases are considered for applications of the ReAct framework in [1]:\nKnowledge-intensive reasoning\n: using ReAct for question answering and fact verification tasks (e.g.,\nHotpotQA\nand\nFEVER\n).\nDecision making\n: applying ReAct to interactive (language-based) decision-making tasks; e.g.,\nALFWorld\nfor navigating simulated households or\nWebShop\nfor completing autonomous shopping tasks.\nExamples of ReAct being applied in each use case are provided above. The ReAct framework is implemented with an LLM—\nPaLM-540B\nin particular\n—that is prompted with several in-context examples that outline the problem solving process. The LLM’s prompt provides human-crafted thought-action-observation trajectories that are followed to arrive at a final solution to a question.\n“By interacting with a Wikipedia API, ReAct is able to retrieve information to support reasoning, while also use reasoning to target what to retrieve next, demonstrating a synergy of reasoning and acting.”\n- from [1]\nKnowledge-intensive reasoning.\nIn this domain, the LLM agent is provided only a question (and optionally a claim) as input. To answer a question or evaluate the correctness of a claim, the LLM must either rely upon its internal knowledge base or retrieve necessary information from an external environment. Specifically, the agent’s action space is outlined below. Here, we see that authors in [1] expose basic information retrieval functionality—\nreflective of how a typical human would lookup information on Wikipedia\n—to the LLM agent via its action space.\nReAct action space for knowledge-intensive reasoning\nTo teach the LLM how to properly think and act in this environment, humans manually craft ReAct-style problem-solving trajectories for each application dataset, including six trajectories for HotpotQA and three trajectories for FEVER. These trajectories are then provided as in-context examples in the prompt provided to the LLM agent, allowing the trajectories to be used as a demonstration of how to solve a problem. A full example of a prompt—\nwith three in-context examples\n—used for a ReAct agent on HotpotQA is provided below.\nQuestion:\nMusician and satirist Allie Goertz wrote a song about the \"The Simpsons\" character Milhouse, who Matt Groening named after who?\nThought 1:\nThe question simplifies to \"The Simpsons\" character Milhouse is named after who. I only need to search Milhouse and find who it is named after.\nAction 1:\nSearch[Milhouse]\nObservation 1:\nMilhouse Mussolini Van Houten is a recurring character in the Fox animated television series The Simpsons voiced by Pamela Hayden and created by Matt Groening.\nThought 2:\nThe paragraph does not tell who Milhouse is named after, maybe I can look up \"named after\".\nAction 2:\nLookup[named after]\nObservation 2:\n(Result 1 / 1) Milhouse was named after U.S. president Richard Nixon, whose middle name was Milhous.\nThought 3:\nMilhouse was named after U.S. president Richard Nixon, so the answer is Richard Nixon.\nAction 3:\nFinish[Richard Nixon]\nQuestion:\nWhich documentary is about Finnish rock groups, Adam Clayton Powell or The Saimaa Gesture?\nThought 1:\nI need to search Adam Clayton Powell and The Saimaa Gesture, and find which documentary is about Finnish rock groups.\nAction 1:\nSearch[Adam Clayton Powell]\nObservation 1:\nCould not find [Adam Clayton Powell]. Similar: [’Adam Clayton Powell III’, ’Seventh Avenue (Manhattan)’, ’Adam Clayton Powell Jr. State Office Building’, ’Isabel Washington Powell’, ’Adam Powell’, ’Adam Clayton Powell (film)’, ’Giancarlo Esposito’].\nThought 2:\nTo find the documentary, I can search Adam Clayton Powell (film).\nAction 2:\nSearch[Adam Clayton Powell (film)]\nObservation 2:\nAdam Clayton Powell is a 1989 American documentary film directed by Richard Kilberg. The film is about the rise and fall of influential African-American politician Adam Clayton Powell Jr.[3][4] It was later aired as part of the PBS series The American Experience.\nThought 3:\nAdam Clayton Powell (film) is a documentary about an African-American politician, not Finnish rock groups. So the documentary about Finnish rock groups must instead be The Saimaa Gesture.\nAction 3:\nFinish[The Saimaa Gesture]\nQuestion:\nWhat profession does Nicholas Ray and Elia Kazan have in common?\nThought 1:\nI need to search Nicholas Ray and Elia Kazan, find their professions, then find the profession they have in common.\nAction 1:\nSearch[Nicholas Ray]\nObservation 1:\nNicholas Ray (born Raymond Nicholas Kienzle Jr., August 7, 1911 - June 16, 1979) was an American film director, screenwriter, and actor best known for the 1955 film Rebel Without a Cause.\nThought 2:\nProfessions of Nicholas Ray are director, screenwriter, and actor. I need to search Elia Kazan next and find his professions.\nAction 2:\nSearch[Elia Kazan]\nObservation 2:\nElia Kazan was an American film and theatre director, producer, screenwriter and actor.\nThought 3:\nProfessions of Elia Kazan are director, producer, screenwriter, and actor. So profession Nicholas Ray and Elia Kazan have in common is director, screenwriter, and actor.\nAction 3:\nFinish[director, screenwriter, actor]\nQuestion:\nWhich magazine was started first Arthur’s Magazine or First for Women?\nIn this example, the ReAct agent is explicitly prompted to output a thought prior to every concrete action that it takes. Unlike a traditional LLM, the ReAct agent does not produce a single output per prompt. Rather, the agent generates output sequentially as follows:\nSelects an action to perform (either a concrete action or a thought).\nGets feedback from the environment based on this action (e.g., the information retrieved from a search query).\nContinues on to the next action with this new context.\nEventually, the terminal action is reached, triggering the end of the problem solving process; see below. This stateful, sequential problem solving approach is characteristic of agents and helps to distinguish them from standard LLMs.\nSequentially solving a problem with ReAct\nDecision making.\nThe setup for ReAct on decision making tasks in very similar to that of knowledge-intensive reasoning tasks. For both decision making tasks, humans manually annotate several reasoning trajectories that are used as in-context examples for the ReAct agent. Unlike knowledge-intensive reasoning tasks, however, the thought patterns used by ReAct for decision making tasks are sparse—\nthe model is prompted to use discretion in determining when and how it should think\n. Additionally, the ReAct agent is provided with a wider variety of tools and actions to use for the WebShop dataset; e.g., search, filter, choose a product, choose product attributes, buy a product and more. This application serves as a good test of ReAct when interacting with a more complex environment.\nDoes ReAct perform well?\nThe ReAct agents described above are compared to several baselines:\nPrompting\n: few-shot prompt that removes thoughts, actions and observations from example trajectories, leaving only questions and answers.\nCoT prompting\n: same as above, but the model is prompted to produce a chain of thought before outputting a final solution\n4\n.\nAct (action-only)\n: removes thoughts from ReAct trajectories, leaving only observations and actions.\nImitation\n: agents trained via imitation and / or reinforcement learning to mimic human reasoning trajectories (e.g.,\nBUTLER\n).\nAs shown below, the ReAct framework consistently outperforms the Act setup,\nrevealing that the ability of an agent to think as it acts is incredibly important\n. Going further, we see that CoT prompting is a strong baseline that outperforms ReAct in some cases but struggles in scenarios where the LLM is prone to hallucination—\nReAct is able to leverage external sources of information to avoid hallucinating in these cases\n.  Finally, we see that there is much room to improve the performance of ReAct agents. In fact, the agents explored in [1] are quite brittle; e.g., authors note that simply retrieving non-informative information can lead to failure.\n(from [1])\nReAct + CoT.\nReAct is factual and grounded in its approach to solving problems. Although CoT prompting may suffer from hallucinated facts due to not being grounded in external knowledge, this approach still excels at formulating a structure for solving complex reasoning tasks. ReAct imposes a strict structure of observations, thoughts and actions onto the agent’s reasoning trajectory, while CoT has more flexibility in formulating the reasoning process.\n(from [1])\nTo reap the benefits of both approaches\n5\n,\nwe can toggle between them\n! For example, we can default to CoT prompting if ReAct fails to return an answer after\nN\nsteps (i.e., ReAct → CoT) or take several CoT samples and use ReAct if disagreement exists among the answers (i.e., CoT → ReAct). As shown above, such a backoff approach—\nin either direction\n—boosts the agent’s problem solving capabilities.\nPrior Attempts at Agents\nAlthough ReAct was (arguably) the first lasting framework to be proposed for AI agents, there were a variety of impactful papers and ideas previously proposed within the agents space. Here, we will quickly outline some of these key proposals and how they compare, allowing us to understand how the ReAct framework builds upon prior work to create a more useful and popular framework.\n(from [10])\nInner monologue (IM) [10]\nwas one of the most comparable works to ReAct and is applied to planning in a robotics setting. As shown above, IM integrates an LLM with several domain-specific feedback mechanisms; e.g., scene descriptors or success detectors. Somewhat similarly to ReAct, the LLM is used to generate a plan and monitor the solution of a task—\nlike picking up an object\n—by iteratively acting, thinking and receiving feedback from the external environment.\n“We investigate to what extent LLMs used in embodied contexts can reason over sources of feedback provided through natural language… We propose that by leveraging environment feedback, LLMs are able to form an inner monologue that allows them to more richly process and plan in robotic control scenarios.”\n- from [10]\nIM demonstrates the feasibility of leveraging LLMs as a general tool for problem solving in domains beyond natural language. Relative to ReAct, however, the ability of the LLM to “think” within IM is limited—\nthe model can only observe feedback from the environment and decide what needs to be done next\n. ReAct solves this problem by empowering the agent to output extensive, free-form thoughts.\n(from [14])\nLLMs for interactive decision making (LID) [14]\nuses language as a general medium for planning and action by proposing a language-based framework for solving sequential problems. We can formulate the context and action space for a wide variety of tasks as a sequence of tokens, thus converting arbitrary tasks into a standardized format that is LLM-compatible. Then, this data can be ingested by an LLM, allowing powerful foundation models to incorporate feedback from the environment and make decisions; see above. In [14], authors finetune LID using imitation learning to correctly predict actions across a variety of domains.\n(from [11])\nWebGPT [11]\nexplores integrating an LLM (\nGPT-3\n) with a text-based web browser to more effectively answer questions. This work is any early pioneer of open-ended tool use and teaches the LLM how to openly search and navigate the web. However, WebGPT is explicitly finetuned over a large dataset of task solutions from humans (i.e., behavior cloning or imitation learning). Therefore, this system—\ndespite being very forward-looking and effective (i.e., produces answers preferred to those of a human in >50% of cases)\n—requires a large amount of human intervention. Nonetheless, finetuning LLM agents with human feedback is a hot research topic even today, and WebGPT is a foundational work in this space.\n(from [12])\nInspired by the broad capabilities of LLMs,\nGato [12]\nis a single “generalist” agent that is capable of acting across many modalities, tasks and domains. For example, Gato is used for playing Atari, captioning images, manipulating robotic arms and more. As described in the report, Gato is capable of\n“deciding based on its context whether to output text, joint torques, button presses, or other tokens.”\nThis model truly works towards the goal of creating an autonomous system that can solve almost any problem. Similarly to WebGPT, however, Gato is trained via an imitation learning approach that collects a massive dataset of context and actions—\nall represented as flat sequences of tokens\n—across many problem scenarios.\n(from [13])\nReasoning via Planning (RAP) [13]\naims to endow LLMs with a better world model—\nor an understanding of the environment in which they act and the rewards that come from it\n—with the goal of improving the LLM’s ability to plan solutions to complex, multi-step problems. In particular, the LLM is used to build a reasoning tree that can be explored via\nMonte Carlo Tree Search (MCTS)\nto find a solution that achieves high reward. Here, the LLM itself is also used to evaluate solutions—\nthe LLM serves as both an agent and a world model in RAP\n!\n“The LLM (as agent) incrementally builds a reasoning tree under the guidance of the LLM (as world model) and rewards, and efficiently obtains a high-reward reasoning path with a proper balance between exploration vs. exploitation.”\n- from [13]\nRAP is a useful and effective framework, but it is applied purely to text-based reasoning problems in [13]—\nit is not a general problem-solving framework like ReAct\n. There are many such works that bear a high level of resemblance to agent systems but are applied mostly to improving LLM reasoning capabilities:\nSelection-Inference\nimproves LLM reasoning capabilities by separating the problem solving process into alternating steps of selection (or planning) and solving. A similar approach is pioneered by\nCreswell et al\n.\nRe2\nis a prompting strategy that improves LLM reasoning capabilities by asking the LLM to re-read the question prior to deriving an answer.\nLLM-Augmenter\ncombines an LLM with databases or sources of domain-specific information that provide useful external knowledge to the LLM, thus improving groundedness in question-answering tasks.\nFor a more complete survey of research on the intersection of agents and reasoning for LLMs (and much more), see\nthis incredible writeup\n.\nWhat is an “agent”?\n“The simplest way to view the starting points for language model-based agents is any tool-use language model. The spectrum of agents increases in complexity from here.”\n-\nNathan Lambert\nDespite their popularity in the industry, agents do not have a clear definition—\nthere is a lot of discussion about what qualifies as an “agent”\n. Lack of clarity on the definition of agents arises from the fact that we encounter a variety of agents in today’s world that lie on a wide spectrum of complexity. At a high level, the functionality of an agent may appear similar to that of an LLM in some cases, but an agent typically has a wider scope of strategies and tools available for solving a problem. Using the information we have learned so far, we will now create a framework for understanding the spectrum of capabilities an AI agent may possess, as well as how these capabilities differ from a standard LLM.\nFrom LLMs to Agents\nWe have learned about a variety of concepts in this overview, including\ni)\nstandard LLMs,\nii)\ntool usage,\niii)\nreasoning models and\niv)\nautonomous systems for problem solving. Starting with the standard definition of an LLM, we will now explain how each of these ideas can be used to build upon the standard LLM’s capabilities, creating a system that is more agentic in nature.\n[Level 0] Standard LLMs.\nAs a starting point, we can consider the standard setup for an LLM (depicted above), which receives a textual prompt as input and generates a textual response as output. To solve problems, this system purely relies upon the internal knowledge base of the LLM without introducing external systems or imposing any structure upon the problem-solving process. To solve more complex reasoning problems, we may also use a reasoning-style LLM or a CoT prompting approach to elicit a reasoning trajectory; see below.\n[Level 1] Tool usage.\nRelying upon an LLM’s internal knowledge base is risky—\nLLMs have a fixed knowledge cutoff date and a tendency to hallucinate\n. To mitigate this problem, we can teach an LLM how to make API calls for the purpose of retrieving useful information and solving sub-tasks with specialized tools. Using this approach, the LLM can more robustly solve problems by delegating the solution of sub-tasks to more specialized systems; see below.\n[Level 2] Decomposing problems.\nExpecting an LLM to solve a complex problem in a single step may be unreasonable. Instead, we can create a framework that plans how a problem should be solved and iteratively derives a solution. Such an LLM system can be handcrafted; e.g., by chaining multiple prompts or executing several prompts in parallel and aggregating their results. Alternatively, we can avoid this manual effort by using a framework like ReAct that relies upon an LLM to sequentially derive and execute a problem-solving strategy; see below.\nOf course, the problem of decomposing and solving complex problems with an LLM is intricately related to tool usage and reasoning. The LLM may rely upon various tools throughout the problem solving process, and reasoning capabilities are essential for formulating detailed and correct plans for solving a problem. Going further, this LLM-centric approach to problem solving introduces the notion of control flow to inference with an LLM—\nthe agent’s output is sequentially built as it statefully moves through a sequence of problem-solving steps\n.\n[Level 3] Increasing autonomy.\nThe above framework outlines most key functionalities of AI agents today. However, we can also make such a system more capable by providing it with a greater level of autonomy. For example, we can include within the agent’s action space the ability to take concrete actions (e.g., buying an item, sending an email or opening a pull request) on our behalf.\n“An agent is anything that can perceive its environment and act upon that environment… This means that an agent is characterized by the environment it operates in and the set of actions it can perform.”\n-\nChip Huyen\nSo far, the agents we have outlined always take a prompt from a human user as input. When given this prompt, they begin the process of thinking, acting and formulating an appropriate response. In other words,\nthese agents only take action when triggered by a prompt from a human user\n. However, this does not have to be the case! We can build agents that continuously operate in the background. For example, a lot of research has been done on\nopen-ended computer use agents\n, and OpenAI recently announced\nCodex\n—\na cloud-based software engineering agent that can work on many tasks in parallel and even make PRs to codebases on its own\n.\nAI agent spectrum.\nCombining all of the concepts we have discussed throughout this overview, we could create an agent system that:\nRuns asynchronously without any human input.\nUses reasoning LLMs to formulate plans for solving complex tasks.\nUses a standard LLM to produce basic thoughts or synthesize information.\nTakes actions in the external world (e.g., booking a plane ticket or adding an event to our calendar) on our behalf.\nRetrieves up-to-date info via the Google search API (or any other tool).\nEach style of LLM—\nas well as any other tool or model\n—has both strengths and weaknesses. These components provide agent systems with many capabilities that are useful for various aspects of problem solving.\nThe crux of agent systems is orchestrating these components in a way that is seamless and reliable\n. However, agents\nlie on a spectrum\nand may or may not use all of these functionalities; e.g., the system described above, a basic tool-use LLM and a chain of prompts for solving a particular class of problems all fall under the umbrella of an agent system.\nThe Future of AI Agents\nAlthough AI agents are incredibly popular, work in this space—\nboth from a research and application perspective\n—is nascent. As we have learned, agents operate via a sequential problem solving process. If any step in this process goes wrong, then the agent is likely to fail. As such,\nreliability is a prerequisite for building effective agents in complex environments\n. In other words, building robust agent systems will require creating LLMs with more\nnines of reliability\n; see below.\n“Last year, you said the thing that was holding [agents] back was the extra nines of reliability… that's the way you would still describe the way in which these software agents aren't able to do a full day of work, but are able to help you out with a couple minutes.”\n-\nDwarkesh Podcast\nMany agents today\nare (arguably) brittle\ndue to a lack of reliability. However, progress is being made quickly, both on LLMs in general (i.e., better reasoning and new generations of models) and agents in particular. Recent research has focused especially on\neffectively evaluating agents\n,\ncreating multi-agent systems\nand\nfinetuning agent systems\nto improve reliability in specialized domains. Given the pace of research in this area,\nwe are likely to see a significant increase in the capabilities and generality of these agent systems in the near future\n.\nNew to the newsletter?\nHi! I’m\nCameron R. Wolfe\n, Deep Learning Ph.D. and Senior Research Scientist at\nNetflix\n. This is the Deep (Learning) Focus newsletter, where I help readers better understand important topics in AI research. The newsletter will always be free and open to read. If you like the newsletter, please subscribe, consider a paid subscription, share it, or follow me on\nX\nand\nLinkedIn\n!\nSubscribe\nBibliography\n[1] Yao, Shunyu, et al. \"React: Synergizing reasoning and acting in language models.\"\nInternational Conference on Learning Representations (ICLR)\n. 2023.\n[2] Schick, Timo, et al. \"Toolformer: Language models can teach themselves to use tools.\"\nAdvances in Neural Information Processing Systems\n36 (2023): 68539-68551.\n[3] Thoppilan, Romal, et al. \"Lamda: Language models for dialog applications.\"\narXiv preprint arXiv:2201.08239\n(2022).\n[4] Shen, Yongliang, et al. \"Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face.\"\nAdvances in Neural Information Processing Systems\n36 (2023): 38154-38180.\n[5] Patil, Shishir G., et al. \"Gorilla: Large language model connected with massive apis.\"\nAdvances in Neural Information Processing Systems\n37 (2024): 126544-126565.\n[6] Wei, Jason, et al. \"Chain-of-thought prompting elicits reasoning in large language models.\"\nAdvances in neural information processing systems\n35 (2022): 24824-24837.\n[7] Kojima, Takeshi, et al. \"Large language models are zero-shot reasoners.\"\nAdvances in neural information processing systems\n35 (2022): 22199-22213.\n[8] Guo, Daya, et al. \"Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.\"\narXiv preprint arXiv:2501.12948\n(2025).\n[9] Lambert, Nathan, et al. \"T\\\" ulu 3: Pushing frontiers in open language model post-training.\"\narXiv preprint arXiv:2411.15124\n(2024).\n[10] Huang, Wenlong, et al. \"Inner monologue: Embodied reasoning through planning with language models.\"\narXiv preprint arXiv:2207.05608\n(2022).\n[11] Nakano, Reiichiro, et al. \"Webgpt: Browser-assisted question-answering with human feedback.\"\narXiv preprint arXiv:2112.09332\n(2021).\n[12] Reed, Scott, et al. \"A generalist agent.\"\narXiv preprint arXiv:2205.06175\n(2022).\n[13] Hao, Shibo, et al. \"Reasoning with language model is planning with world model.\"\narXiv preprint arXiv:2305.14992\n(2023).\n[14] Li, Shuang, et al. \"Pre-trained language models for interactive decision-making.\"\nAdvances in Neural Information Processing Systems\n35 (2022): 31199-31212.\n[15] Anthropic. “Introducing the Model Context Protocol”\nhttps://www.anthropic.com/news/model-context-protocol\n(2024).\n1\nIn the context of reasoning models, these chains of thought are also referred to as reasoning trajectories or traces.\n2\nThis is quite similar to the definition of a policy in reinforcement learning (RL); see\nhere\nfor details. In both cases, the policy is implemented as a language model and produces an action as output. The main difference between the agent and RL definition of a policy is the policy’s input. For agents, the input is the current observation. For RL, the the policy’s input is the current state of the environment.\n3\nSee\nhere\nfor details on the difference between a deterministic and stochastic policy.\n4\nCoT prompting can also be extended with\nself-consistency\nwith a majority vote to further improve performance.\n5\nNotably, ReAct (or any other agentic framework) is not guaranteed to outperform standard CoT prompting! The relative performance of these techniques is highly related to the complexity of problems being solved—\nCoT prompting performs very well in cases where hallucination is unlikely to be a problem for the LLM being used\n.",
  "metadata": {
    "title": "AI Agents from First Principles",
    "description": "Understanding AI agents by building upon the most basic concepts of LLMs...",
    "keywords": null,
    "author": "Cameron R. Wolfe, Ph.D.",
    "og:type": "article",
    "og:title": "AI Agents from First Principles",
    "og:description": "Understanding AI agents by building upon the most basic concepts of LLMs...",
    "og:image": "https://substackcdn.com/image/fetch/$s_!IitU!,w_1200,h_675,c_fill,f_jpg,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcee4a772-78a7-41b7-8cf1-4da233376ea6_2002x1122.png",
    "og:url": "https://cameronrwolfe.substack.com/p/ai-agents",
    "twitter:title": "AI Agents from First Principles",
    "twitter:description": "Understanding AI agents by building upon the most basic concepts of LLMs...",
    "twitter:image": "https://substackcdn.com/image/fetch/$s_!5lFg!,f_auto,q_auto:best,fl_progressive:steep/https%3A%2F%2Fcameronrwolfe.substack.com%2Fapi%2Fv1%2Fpost_preview%2F164903679%2Ftwitter.jpg%3Fversion%3D4",
    "twitter:card": "summary_large_image",
    "article:modified_time": "2025-06-09T09:35:51.283Z"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [
    {
      "file": "attachments/321882e27e41/code/body-000.txt",
      "chars": 3061,
      "language": "text"
    }
  ],
  "images": [
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!HVW3!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa1cd321e-1b6e-45db-89d5-4d1eaffa039b_2000x1122.png",
      "bytes": 155765,
      "reused": true,
      "local_path": "attachments/321882e27e41/images/039.png",
      "src": "https://substackcdn.com/image/fetch/$s_!HVW3!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa1cd321e-1b6e-45db-89d5-4d1eaffa039b_2000x1122.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!dPSO!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F8b5bd8d4-a75a-4a61-a95c-f2e3363fef79_2216x700.png",
      "bytes": 46507,
      "reused": false,
      "local_path": "attachments/321882e27e41/images/body-fd378c2ecb46e03276f7.png",
      "src": "https://substackcdn.com/image/fetch/$s_!dPSO!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F8b5bd8d4-a75a-4a61-a95c-f2e3363fef79_2216x700.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!kSln!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdeb1e059-821d-4e77-98af-2129d4a8766a_1742x782.png",
      "bytes": 153882,
      "reused": false,
      "local_path": "attachments/321882e27e41/images/body-4aa0dcb8778692b0c1b2.png",
      "src": "https://substackcdn.com/image/fetch/$s_!kSln!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdeb1e059-821d-4e77-98af-2129d4a8766a_1742x782.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!N4MY!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F59b33434-e0d9-4211-847a-ff89508dfa37_2382x350.png",
      "bytes": 33449,
      "reused": false,
      "local_path": "attachments/321882e27e41/images/body-40831042652ad691b945.png",
      "src": "https://substackcdn.com/image/fetch/$s_!N4MY!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F59b33434-e0d9-4211-847a-ff89508dfa37_2382x350.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!bcHR!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fef039977-ac5d-4a83-94dd-944ccae42847_1698x1090.png",
      "bytes": 324768,
      "reused": false,
      "local_path": "attachments/321882e27e41/images/body-b87b4f5f21c609a71ef2.png",
      "src": "https://substackcdn.com/image/fetch/$s_!bcHR!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fef039977-ac5d-4a83-94dd-944ccae42847_1698x1090.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!EpEu!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6cc4f982-4113-4f06-ad44-7d41235e6c4e_1610x1108.png",
      "bytes": 67385,
      "reused": false,
      "local_path": "attachments/321882e27e41/images/body-f20467fe27c227824bb1.png",
      "src": "https://substackcdn.com/image/fetch/$s_!EpEu!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6cc4f982-4113-4f06-ad44-7d41235e6c4e_1610x1108.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!mk9r!,w_140,h_140,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F23d9c87e-b238-4fdd-996e-4ed4465b9931_2334x1282.png",
      "bytes": 3184,
      "reused": false,
      "local_path": "attachments/321882e27e41/images/body-bbfe827cc78ebd515397.png",
      "src": "https://substackcdn.com/image/fetch/$s_!mk9r!,w_140,h_140,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F23d9c87e-b238-4fdd-996e-4ed4465b9931_2334x1282.png",
      "alt": "Demystifying Reasoning Models",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!NPw_!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F599a636e-b0b2-4de3-84c8-3edf906bfa82_1616x882.png",
      "bytes": 173790,
      "reused": false,
      "local_path": "attachments/321882e27e41/images/body-79ad2f7a309d8a6f27f6.png",
      "src": "https://substackcdn.com/image/fetch/$s_!NPw_!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F599a636e-b0b2-4de3-84c8-3edf906bfa82_1616x882.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!JJH6!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F8c08cfd9-85a6-4079-b510-59857ae05c3e_1970x1174.png",
      "bytes": 182689,
      "reused": false,
      "local_path": "attachments/321882e27e41/images/body-8f282268b9871b515f9e.png",
      "src": "https://substackcdn.com/image/fetch/$s_!JJH6!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F8c08cfd9-85a6-4079-b510-59857ae05c3e_1970x1174.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!lZD6!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9bdc9fc1-4032-41ba-9d7a-946f4826f826_1840x454.png",
      "bytes": 90204,
      "reused": false,
      "local_path": "attachments/321882e27e41/images/body-583f5fef0aa55088c662.png",
      "src": "https://substackcdn.com/image/fetch/$s_!lZD6!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9bdc9fc1-4032-41ba-9d7a-946f4826f826_1840x454.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!mzxO!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7334cdb5-5398-47d2-98bb-01ca41a58879_1854x726.png",
      "bytes": 80426,
      "reused": false,
      "local_path": "attachments/321882e27e41/images/body-ece08bcabe7c08b33f36.png",
      "src": "https://substackcdn.com/image/fetch/$s_!mzxO!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7334cdb5-5398-47d2-98bb-01ca41a58879_1854x726.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!iThv!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fff8c2a7d-e62b-4ed7-bb2c-99de79b0ad96_2390x688.png",
      "bytes": 47802,
      "reused": false,
      "local_path": "attachments/321882e27e41/images/body-2f6dc341e53b91d50d82.png",
      "src": "https://substackcdn.com/image/fetch/$s_!iThv!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fff8c2a7d-e62b-4ed7-bb2c-99de79b0ad96_2390x688.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!XNLJ!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9c731b2f-9a36-45a8-b016-89691850dc88_2134x760.png",
      "bytes": 51841,
      "reused": false,
      "local_path": "attachments/321882e27e41/images/body-c6a458848b2899289564.png",
      "src": "https://substackcdn.com/image/fetch/$s_!XNLJ!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9c731b2f-9a36-45a8-b016-89691850dc88_2134x760.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!7PpC!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe159d88e-5792-4106-bff8-044361ced6fb_2136x934.png",
      "bytes": 66027,
      "reused": false,
      "local_path": "attachments/321882e27e41/images/body-6592b295913f9336fe6c.png",
      "src": "https://substackcdn.com/image/fetch/$s_!7PpC!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe159d88e-5792-4106-bff8-044361ced6fb_2136x934.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!CAb3!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa8a68b2e-7283-4b8d-90ab-399d58bee163_1920x998.png",
      "bytes": 105010,
      "reused": false,
      "local_path": "attachments/321882e27e41/images/body-fdbaff4543b838070b90.png",
      "src": "https://substackcdn.com/image/fetch/$s_!CAb3!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa8a68b2e-7283-4b8d-90ab-399d58bee163_1920x998.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!Bj0c!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F527e2e86-7b4f-4919-b1f7-cbdf92e99148_1576x796.png",
      "bytes": 178155,
      "reused": false,
      "local_path": "attachments/321882e27e41/images/body-39c15c51189da62e9f23.png",
      "src": "https://substackcdn.com/image/fetch/$s_!Bj0c!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F527e2e86-7b4f-4919-b1f7-cbdf92e99148_1576x796.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!vp5a!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc6c6a76b-e84c-4a9c-b5a8-79cdfe7d2a4d_1394x1164.png",
      "bytes": 180579,
      "reused": false,
      "local_path": "attachments/321882e27e41/images/body-4e96aa417432c25ebad8.png",
      "src": "https://substackcdn.com/image/fetch/$s_!vp5a!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc6c6a76b-e84c-4a9c-b5a8-79cdfe7d2a4d_1394x1164.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!5QLQ!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F999ee665-a74a-47d3-8498-b3c7db099edc_1606x1566.png",
      "bytes": 385566,
      "reused": false,
      "local_path": "attachments/321882e27e41/images/body-7080c4bd1e29345b905b.png",
      "src": "https://substackcdn.com/image/fetch/$s_!5QLQ!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F999ee665-a74a-47d3-8498-b3c7db099edc_1606x1566.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!8CG-!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F803f9b98-970a-4145-b03b-2855a904c047_1868x738.png",
      "bytes": 91649,
      "reused": false,
      "local_path": "attachments/321882e27e41/images/body-43e2fa40fe5ba675d3ed.png",
      "src": "https://substackcdn.com/image/fetch/$s_!8CG-!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F803f9b98-970a-4145-b03b-2855a904c047_1868x738.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!Hg2e!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F75aadeb9-3ce5-489d-8e0c-8490981161e5_2320x1244.png",
      "bytes": 47132,
      "reused": false,
      "local_path": "attachments/321882e27e41/images/body-1eec255e677c41592781.png",
      "src": "https://substackcdn.com/image/fetch/$s_!Hg2e!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F75aadeb9-3ce5-489d-8e0c-8490981161e5_2320x1244.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!t7P-!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6e6e7533-bd8d-4787-821d-c92b081c3e16_1932x678.png",
      "bytes": 127182,
      "reused": false,
      "local_path": "attachments/321882e27e41/images/body-f3e27a064e00f9df2ad8.png",
      "src": "https://substackcdn.com/image/fetch/$s_!t7P-!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6e6e7533-bd8d-4787-821d-c92b081c3e16_1932x678.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!ainZ!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdb14d0a2-8b3c-43a9-b31d-e34b3a4b5c47_1986x1206.png",
      "bytes": 117153,
      "reused": false,
      "local_path": "attachments/321882e27e41/images/body-9612c9aa4ea31e197295.png",
      "src": "https://substackcdn.com/image/fetch/$s_!ainZ!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdb14d0a2-8b3c-43a9-b31d-e34b3a4b5c47_1986x1206.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!u-Fr!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fae13e457-28b8-40bf-b3ee-c1497e99c8a0_2146x1176.png",
      "bytes": 178600,
      "reused": false,
      "local_path": "attachments/321882e27e41/images/body-06387f84afa045fb260d.png",
      "src": "https://substackcdn.com/image/fetch/$s_!u-Fr!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fae13e457-28b8-40bf-b3ee-c1497e99c8a0_2146x1176.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!hmBA!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fabdedf5c-f94b-41e4-8628-8f5bf7ca7df5_1690x700.png",
      "bytes": 125969,
      "reused": false,
      "local_path": "attachments/321882e27e41/images/body-d93725639781eac16760.png",
      "src": "https://substackcdn.com/image/fetch/$s_!hmBA!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fabdedf5c-f94b-41e4-8628-8f5bf7ca7df5_1690x700.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!GqwP!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6cb9686e-bbd4-472b-9d3f-94c36adf91ef_2134x1032.png",
      "bytes": 342214,
      "reused": false,
      "local_path": "attachments/321882e27e41/images/body-076469d08d1fffc77f0b.png",
      "src": "https://substackcdn.com/image/fetch/$s_!GqwP!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6cb9686e-bbd4-472b-9d3f-94c36adf91ef_2134x1032.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!cMs-!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb280ff9b-b2c5-4c67-b96a-e45d7b341ff0_1252x1002.png",
      "bytes": 170025,
      "reused": false,
      "local_path": "attachments/321882e27e41/images/body-b5136ad51791c2841414.png",
      "src": "https://substackcdn.com/image/fetch/$s_!cMs-!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb280ff9b-b2c5-4c67-b96a-e45d7b341ff0_1252x1002.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!aZGa!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd7a99d91-2746-43bf-b5be-b05ce8a6e26a_1834x990.png",
      "bytes": 185774,
      "reused": false,
      "local_path": "attachments/321882e27e41/images/body-8fcf5b064c4c231324d8.png",
      "src": "https://substackcdn.com/image/fetch/$s_!aZGa!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd7a99d91-2746-43bf-b5be-b05ce8a6e26a_1834x990.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!GApJ!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F8a7db9c5-fdc1-4ac5-be2d-85a56cda348d_1348x552.png",
      "bytes": 36885,
      "reused": false,
      "local_path": "attachments/321882e27e41/images/body-6fc77d7592731fcaba51.png",
      "src": "https://substackcdn.com/image/fetch/$s_!GApJ!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F8a7db9c5-fdc1-4ac5-be2d-85a56cda348d_1348x552.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!lHTC!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F20d63a60-9328-4e1e-bfce-3234b7cdfc6a_1846x548.png",
      "bytes": 49845,
      "reused": false,
      "local_path": "attachments/321882e27e41/images/body-33ae61e65759513b9891.png",
      "src": "https://substackcdn.com/image/fetch/$s_!lHTC!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F20d63a60-9328-4e1e-bfce-3234b7cdfc6a_1846x548.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!l6Hv!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcfd4ac4a-6040-40b0-852d-80daeac6bf00_1360x932.png",
      "bytes": 55512,
      "reused": false,
      "local_path": "attachments/321882e27e41/images/body-ddffba667e2bf0c20d69.png",
      "src": "https://substackcdn.com/image/fetch/$s_!l6Hv!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcfd4ac4a-6040-40b0-852d-80daeac6bf00_1360x932.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!NrQa!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa3765945-9ea8-47de-a7f7-802a009e9c67_1856x990.png",
      "bytes": 71907,
      "reused": false,
      "local_path": "attachments/321882e27e41/images/body-73d5e4470b6c52d2f385.png",
      "src": "https://substackcdn.com/image/fetch/$s_!NrQa!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa3765945-9ea8-47de-a7f7-802a009e9c67_1856x990.png",
      "alt": "",
      "downloaded": true
    }
  ],
  "audio": [],
  "videos": [],
  "links": [
    {
      "href": "https://substackcdn.com/image/fetch/$s_!HVW3!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa1cd321e-1b6e-45db-89d5-4d1eaffa039b_2000x1122.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://modelcontextprotocol.io/introduction",
      "label": "source",
      "type": "a"
    },
    {
      "href": "https://substackcdn.com/image/fetch/$s_!kSln!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdeb1e059-821d-4e77-98af-2129d4a8766a_1742x782.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://substackcdn.com/image/fetch/$s_!bcHR!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fef039977-ac5d-4a83-94dd-944ccae42847_1698x1090.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://modelcontextprotocol.io/",
      "label": "Model Context Protocol",
      "type": "a"
    },
    {
      "href": "https://www.anthropic.com/news/model-context-protocol",
      "label": "proposed by Anthropic",
      "type": "a"
    },
    {
      "href": "https://substackcdn.com/image/fetch/$s_!EpEu!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6cc4f982-4113-4f06-ad44-7d41235e6c4e_1610x1108.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://cameronrwolfe.substack.com/p/teaching-language-models-to-use-tools",
      "label": "link",
      "type": "a"
    },
    {
      "href": "https://cameronrwolfe.substack.com/p/language-models-and-friends-gorilla",
      "label": "link",
      "type": "a"
    },
    {
      "href": "https://cameronrwolfe.substack.com/p/program-aided-language-models",
      "label": "link",
      "type": "a"
    },
    {
      "href": "https://cameronrwolfe.substack.com/p/can-language-models-make-their-own",
      "label": "link",
      "type": "a"
    },
    {
      "href": "https://cameronrwolfe.substack.com/p/demystifying-reasoning-models",
      "label": "Demystifying Reasoning Models Cameron R. Wolfe, Ph.D. · February 18, 2025 Read full story",
      "type": "a"
    },
    {
      "href": "https://substack.com/profile/29736521-cameron-r-wolfe-phd",
      "label": "Cameron R. Wolfe, Ph.D.",
      "type": "a"
    },
    {
      "href": "https://cameronrwolfe.substack.com/p/chain-of-thought-prompting-for-llms",
      "label": "Chain of Thought (CoT) prompting",
      "type": "a"
    },
    {
      "href": "https://substackcdn.com/image/fetch/$s_!NPw_!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F599a636e-b0b2-4de3-84c8-3edf906bfa82_1616x882.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://substackcdn.com/image/fetch/$s_!JJH6!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F8c08cfd9-85a6-4079-b510-59857ae05c3e_1970x1174.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://openai.com/index/learning-to-reason-with-llms/",
      "label": "source",
      "type": "a"
    },
    {
      "href": "https://substackcdn.com/image/fetch/$s_!mzxO!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7334cdb5-5398-47d2-98bb-01ca41a58879_1854x726.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://cameronrwolfe.substack.com/i/153722335/reinforcement-learning-with-verifiable-rewards",
      "label": "reinforcement learning from verifiable rewards (RLVR)",
      "type": "a"
    },
    {
      "href": "https://openai.com/index/introducing-o3-and-o4-mini/",
      "label": "o-series",
      "type": "a"
    },
    {
      "href": "https://substackcdn.com/image/fetch/$s_!iThv!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fff8c2a7d-e62b-4ed7-bb2c-99de79b0ad96_2390x688.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://www.primeintellect.ai/blog/synthetic-1-release",
      "label": "Synthetic-1 dataset",
      "type": "a"
    },
    {
      "href": "https://arxiv.org/abs/2501.12948",
      "label": "DeepSeek-R1",
      "type": "a"
    },
    {
      "href": "https://arxiv.org/abs/2210.03629",
      "label": "The ReAct Framework",
      "type": "a"
    },
    {
      "href": "https://substackcdn.com/image/fetch/$s_!XNLJ!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9c731b2f-9a36-45a8-b016-89691850dc88_2134x760.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://substackcdn.com/image/fetch/$s_!7PpC!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe159d88e-5792-4106-bff8-044361ced6fb_2136x934.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://substackcdn.com/image/fetch/$s_!CAb3!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa8a68b2e-7283-4b8d-90ab-399d58bee163_1920x998.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://substackcdn.com/image/fetch/$s_!Bj0c!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F527e2e86-7b4f-4919-b1f7-cbdf92e99148_1576x796.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://substackcdn.com/image/fetch/$s_!vp5a!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc6c6a76b-e84c-4a9c-b5a8-79cdfe7d2a4d_1394x1164.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://cameronrwolfe.substack.com/i/117151147/few-shot-learning",
      "label": "in-context examples",
      "type": "a"
    },
    {
      "href": "https://substackcdn.com/image/fetch/$s_!5QLQ!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F999ee665-a74a-47d3-8498-b3c7db099edc_1606x1566.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://huggingface.co/datasets/hotpotqa/hotpot_qa",
      "label": "HotpotQA",
      "type": "a"
    },
    {
      "href": "https://huggingface.co/datasets/fever/fever",
      "label": "FEVER",
      "type": "a"
    },
    {
      "href": "https://alfworld.github.io/",
      "label": "ALFWorld",
      "type": "a"
    },
    {
      "href": "https://webshop-pnlp.github.io/",
      "label": "WebShop",
      "type": "a"
    },
    {
      "href": "https://cameronrwolfe.substack.com/p/palm-efficiently-training-massive",
      "label": "PaLM-540B",
      "type": "a"
    },
    {
      "href": "https://substackcdn.com/image/fetch/$s_!8CG-!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F803f9b98-970a-4145-b03b-2855a904c047_1868x738.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://substackcdn.com/image/fetch/$s_!Hg2e!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F75aadeb9-3ce5-489d-8e0c-8490981161e5_2320x1244.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://arxiv.org/abs/2010.03768",
      "label": "BUTLER",
      "type": "a"
    },
    {
      "href": "https://substackcdn.com/image/fetch/$s_!t7P-!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6e6e7533-bd8d-4787-821d-c92b081c3e16_1932x678.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://substackcdn.com/image/fetch/$s_!ainZ!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdb14d0a2-8b3c-43a9-b31d-e34b3a4b5c47_1986x1206.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://substackcdn.com/image/fetch/$s_!u-Fr!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fae13e457-28b8-40bf-b3ee-c1497e99c8a0_2146x1176.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://substackcdn.com/image/fetch/$s_!hmBA!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fabdedf5c-f94b-41e4-8628-8f5bf7ca7df5_1690x700.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://substackcdn.com/image/fetch/$s_!GqwP!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6cb9686e-bbd4-472b-9d3f-94c36adf91ef_2134x1032.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://arxiv.org/abs/2005.14165",
      "label": "GPT-3",
      "type": "a"
    },
    {
      "href": "https://substackcdn.com/image/fetch/$s_!cMs-!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb280ff9b-b2c5-4c67-b96a-e45d7b341ff0_1252x1002.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://substackcdn.com/image/fetch/$s_!aZGa!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd7a99d91-2746-43bf-b5be-b05ce8a6e26a_1834x990.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://www.geeksforgeeks.org/ml-monte-carlo-tree-search-mcts/",
      "label": "Monte Carlo Tree Search (MCTS)",
      "type": "a"
    },
    {
      "href": "https://arxiv.org/abs/2205.09712",
      "label": "Selection-Inference",
      "type": "a"
    },
    {
      "href": "https://arxiv.org/abs/2208.14271",
      "label": "Creswell et al",
      "type": "a"
    },
    {
      "href": "https://arxiv.org/abs/2309.06275",
      "label": "Re2",
      "type": "a"
    },
    {
      "href": "https://arxiv.org/abs/2302.12813",
      "label": "LLM-Augmenter",
      "type": "a"
    },
    {
      "href": "https://arxiv.org/abs/2504.09037",
      "label": "this incredible writeup",
      "type": "a"
    },
    {
      "href": "https://www.interconnects.ai/p/the-ai-agent-spectrum",
      "label": "Nathan Lambert",
      "type": "a"
    },
    {
      "href": "https://substackcdn.com/image/fetch/$s_!l6Hv!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcfd4ac4a-6040-40b0-852d-80daeac6bf00_1360x932.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://substackcdn.com/image/fetch/$s_!NrQa!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa3765945-9ea8-47de-a7f7-802a009e9c67_1856x990.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://huyenchip.com/2025/01/07/agents.html",
      "label": "Chip Huyen",
      "type": "a"
    },
    {
      "href": "https://openai.com/index/universe/",
      "label": "open-ended computer use agents",
      "type": "a"
    },
    {
      "href": "https://openai.com/index/introducing-codex/",
      "label": "Codex",
      "type": "a"
    },
    {
      "href": "https://en.wikipedia.org/wiki/High_availability",
      "label": "nines of reliability",
      "type": "a"
    },
    {
      "href": "https://www.dwarkesh.com/p/sholto-trenton-2",
      "label": "Dwarkesh Podcast",
      "type": "a"
    },
    {
      "href": "https://arxiv.org/abs/2405.13966",
      "label": "are (arguably) brittle",
      "type": "a"
    },
    {
      "href": "https://arxiv.org/abs/2410.10934",
      "label": "effectively evaluating agents",
      "type": "a"
    },
    {
      "href": "https://arxiv.org/abs/2402.03578",
      "label": "creating multi-agent systems",
      "type": "a"
    },
    {
      "href": "https://arxiv.org/abs/2410.07706",
      "label": "finetuning agent systems",
      "type": "a"
    },
    {
      "href": "https://cameronrwolfe.me/",
      "label": "Cameron R. Wolfe",
      "type": "a"
    },
    {
      "href": "https://research.netflix.com/research-area/nlp-and-conversations",
      "label": "Netflix",
      "type": "a"
    },
    {
      "href": "https://twitter.com/cwolferesearch",
      "label": "X",
      "type": "a"
    },
    {
      "href": "https://www.linkedin.com/in/cameron-r-wolfe-ph-d-04744a238/",
      "label": "LinkedIn",
      "type": "a"
    },
    {
      "href": "https://cameronrwolfe.substack.com/i/137266538/markov-decision-process-mdp",
      "label": "here",
      "type": "a"
    },
    {
      "href": "https://spinningup.openai.com/en/latest/spinningup/rl_intro.html",
      "label": "here",
      "type": "a"
    },
    {
      "href": "https://arxiv.org/abs/2203.11171",
      "label": "self-consistency",
      "type": "a"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:47:20Z",
  "source_url": "https://cameronrwolfe.substack.com/p/ai-agents",
  "papers": [
    {
      "status": "ok",
      "url": "https://arxiv.org/pdf/2501.12948",
      "bytes": 5034998,
      "reused": true,
      "local_path": "papers/repair/d0ece4102f0cdb6b3ebb.pdf"
    },
    {
      "status": "ok",
      "url": "https://arxiv.org/pdf/2210.03629",
      "bytes": 633805,
      "reused": false,
      "local_path": "papers/repair/c08ee62f780204fcfff6.pdf"
    },
    {
      "status": "ok",
      "url": "https://arxiv.org/pdf/2010.03768",
      "bytes": 1783014,
      "reused": false,
      "local_path": "papers/repair/1ad4b52e31874db36aa3.pdf"
    },
    {
      "status": "ok",
      "url": "https://arxiv.org/pdf/2005.14165",
      "bytes": 6768044,
      "reused": true,
      "local_path": "papers/repair/0c1629d8d07d676c2aae.pdf"
    },
    {
      "status": "ok",
      "url": "https://arxiv.org/pdf/2205.09712",
      "bytes": 1823144,
      "reused": false,
      "local_path": "papers/repair/723659e9c3161c148d11.pdf"
    },
    {
      "status": "ok",
      "url": "https://arxiv.org/pdf/2208.14271",
      "bytes": 1931632,
      "reused": false,
      "local_path": "papers/repair/d3705cde9007a4b837f3.pdf"
    },
    {
      "status": "ok",
      "url": "https://arxiv.org/pdf/2309.06275",
      "bytes": 2607946,
      "reused": false,
      "local_path": "papers/repair/1df73b5c969900163b0d.pdf"
    },
    {
      "status": "ok",
      "url": "https://arxiv.org/pdf/2302.12813",
      "bytes": 998465,
      "reused": false,
      "local_path": "papers/repair/eb70849e21ffc2bc0abe.pdf"
    },
    {
      "status": "ok",
      "url": "https://arxiv.org/pdf/2504.09037",
      "bytes": 2338416,
      "reused": false,
      "local_path": "papers/repair/246ae46b51ccf17de77a.pdf"
    },
    {
      "status": "ok",
      "url": "https://arxiv.org/pdf/2405.13966",
      "bytes": 8514610,
      "reused": false,
      "local_path": "papers/repair/1a7a74416e48f4a0136a.pdf"
    },
    {
      "status": "ok",
      "url": "https://arxiv.org/pdf/2410.10934",
      "bytes": 5534279,
      "reused": false,
      "local_path": "papers/repair/81fdae82eedb616392f3.pdf"
    },
    {
      "status": "ok",
      "url": "https://arxiv.org/pdf/2402.03578",
      "bytes": 577963,
      "reused": false,
      "local_path": "papers/repair/d8b662cf1705a67a74c2.pdf"
    },
    {
      "status": "ok",
      "url": "https://arxiv.org/pdf/2410.07706",
      "bytes": 1834585,
      "reused": false,
      "local_path": "papers/repair/22bced55b75cc16ad77d.pdf"
    },
    {
      "status": "ok",
      "url": "https://arxiv.org/pdf/2203.11171",
      "bytes": 10505941,
      "reused": true,
      "local_path": "papers/repair/0a415c0e8756d0879469.pdf"
    }
  ],
  "actual_video_files": [],
  "body_chars": 41620,
  "expected_body_images": 31,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/321882e27e41/pages/source.html",
  "local_html": "attachments/321882e27e41/pages/article.html",
  "body_base_path": "attachments/321882e27e41/pages/",
  "body_markdown_path": "attachments/321882e27e41/pages/article.md"
}