保留正文与补采附件。源页面当前未重新核验；具体限制见交付清单。

Revisit consent button We value your privacy We use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking "Accept All", you consent to our use of cookies. Customise Reject All Accept All Customise Consent Preferences We use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below. The cookies that are categorised as "Necessary" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... Show more NecessaryAlways Active Necessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.

Functional Functional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.

Analytics Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.

Performance Performance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors.

Advertisement Advertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns.

Uncategorised Other uncategorised cookies are those that are being analysed and have not been classified into a category as yet. No cookies to display. Reject All Save My Preferences Accept All Toggle SearchToggle Mobile Navigation Publish AI, ML & data-science insights to a global community of data professionals. Sign inSubmit an Article LatestEditor's PicksNewsletterShipAI

Write For TDS LinkedInXToggle Search Large Language Models

Context Engineering — A Comprehensive Hands-On Tutorial with DSPy

Let's dissect the art and science of context engineering, one module at a time! Avishek Biswas August 5, 202515 min read You may have heard about Context Engineering by now. This article will cover the key ideas behind creating LLM applications using Context Engineering principles, visually explain these workflows, and share code snippets that apply these concepts practically. Don't worry about copy-pasting the code from this article into your editor. At the end of this article, I will share the GitHub link to the open-source code repository and a link to my 1-hour 20-minute YouTube course that explains the concepts presented here in greater detail. Unless otherwise mentioned, all images used in this article are produced by the author and are free to use. Let's begin! ···

What is Context Engineering?

There is a significant gap between writing simple prompts and building production-ready applications. Context Engineering is an umbrella term that refers to the delicate art and science of fitting information into the context window of an LLM as it works on a task. The exact scope of where the definition of Context Engineering begins and ends is debatable, but according to this tweet from Andrej Karpathy, we can identify the following key points:

Important: How this article is structured

Throughout this article, I will be referring to the points above while providing examples of how they are applied in building real applications. Whenever I do so, I will use a block quote like this:

It is a holistic approach that breaks up a larger problem into multiple subproblems When you see a quote in this format, the example that follows will apply the quoted concept programmatically. But before that, we must ask ourselves one question...

Why not pass everything into the LLM?

Research has shown that cramming every piece of information into the context of an LLM is far from ideal. Even though many frontier models do claim to support "long-context" windows, they still suffer from issues like context poisoning or context rot. A recent report from Chroma describes how increasing tokens can negatively impact LLM performance
(Source: Chroma) Too much unnecessary information in an LLM's context can pollute the model's understanding, lead to hallucinations, and result in poor performance. This is why simply having a large context window isn't enough. We need systematic approaches to context engineering.

Why DSPY

Just a logo For this tutorial, I have chosen the DSPy framework. I will explain the reasoning for this choice shortly, but let me assure you that the concepts presented here apply to almost any prompting framework, including writing prompts in pure English. DSPy is a declarative framework for building modular AI software. They have neatly separated the two key aspects of any LLM task —
(a) the input and output contracts passed into a module,
and (b) the logic that governs how information flows.

Let's see an example!

Imagine we want to use an LLM to write a joke. Specifically, we want it to generate a setup, a punchline, and the full delivery in a comedian's voice. Oh, and we also want the output in JSON format so that we can post-process individual fields of the dictionary after generation. For example, perhaps we want to print the punchline on a T-shirt (assume someone has already written a convenient function for that). python

system_prompt = """
You are a comedian who tells jokes, you are always funny. 
Generate the setup, punchline, and full delivery in the comedian's voice.

Output in the following JSON format:
{
"setup": <str>,
"punchline": <str>,
"delivery": <str>
}

Your response should be parsable withou errors in Python using json.loads().
"""

client = openai.Client()
response = client.chat.completions.create( 
    model="gpt-4o-mini", 
    temperature = 1,
    messages=[
      {"role": "system", "content": system_prompt,
      {"role": "user", "content": "Write a joke about AI"}
    ] 
)

joke = json.loads(response.choices[0].message.content) # Hope for the best

print_on_a_tshirt(joke["punchline"])

Notice how we post-process the LLM's response to extract the dictionary? What if something "bad" happened, like the LLM failing to generate the response in the desired format? Our entire code would fail and there will be no printing on any T-shirts! The above code is also quite difficult to extend. For example, if we wanted the LLM to do chain of thought reasoning before generating the answer, we would need to write additional logic to parse that reasoning text correctly. Furthermore, it can be difficult to look at plain English prompts like these and understand what the inputs and outputs of these systems are. DSPy solves all of the above. Let's write the above example using DSPy. python

class JokeGenerator(dspy.Signature): 
    """You're a comedian who tells jokes. You're always funny.""" 
    query: str = dspy.InputField()

    setup: str = dspy.OutputField()
    punchline: str = dspy.OutputField() 
    delivery: str = dspy.OutputField()

joke_gen = dspy.Predict(JokeGenerator) 
joke_gen.set_lm(lm=dspy.LM("openai/gpt-4.1-mini", temperature=1))

result = joke_gen(query="Write a joke about AI")
print(result)
print_on_a_tshirt(result.punchline)

This approach gives you structured, predictable outputs that you can work with programmatically, eliminating the need for regex parsing or error-prone string manipulation. Dspy Signatures explicitly makes you define what the inputs to the system are ("query" in the above example), and the outputs to the system (setup, punchline, and delivery) as well as their data-types. It also tells the LLM the order in which you want them to be generated. The output of the previous code block (minus the t-shirt stuff) The dspy.Predict thing is an example of a DSPy Module. With modules, you define how the LLM converts from inputs to outputs. dspy.Predict is the most basic one - you can pass the query to it, as in joke_gen(query="Write a joke about AI") and it will create a basic prompt to send to the LLM. Internally, DSPy just creates a prompt as you can see below. Once the LLM responds, DSPy will create Pydantic BaseModel objects that perform automatic schema validation and send back the output. If errors occur during this validation process, DSPy automatically attempts to fix them by re-prompting the LLM—thereby significantly reducing the risk of a program crash. Dspy.Predict vs Dspy.ChainOfThought In chain of thought, we ask the LLM to generate reasoning text before generating the answer (Source: Author) Another common theme in context engineering is Chain of Thought. Here, we want the LLM to generate reasoning text before providing its final answer. This allows the LLM's context to be populated with its self-generated reasoning before it generates the final output tokens. To do that, you can simply replace dspy.Predict with dspy.ChainOfThought in the example above. The rest of the code remains the same. Now you can see that the LLM generates reasoning before the defined output fields.

Multi-Step Interactions and Agentic Workflows

The best part of DSPy's approach is how it decouples system dependencies (Signatures) from control flows (Modules), which makes writing code for multi-step interactions trivial (and fun!). In this section, let's see how we can build some simple agentic flows.

Sequential Processing

Let's remind ourselves about one of the key components of Context Engineering.

It is a holistic approach that breaks up a larger problem into multiple subproblems

Let's continue with our joke generation example. We can easily separate out two subproblems from it. Generating the idea is one, creating a joke is another. Sequential flows Sequential flows allow us to design LLM systems in a modular way where each agent can be of appropriate strength/size and is given context and tools that are appropriate for its task (Illustrated by author) Let's have two agents then — the first Agent generates a joke idea (setup and punchline) from a query. A second agent then generates the joke from this idea.

Each agent can be of appropriate capability and size depending on the complexity of the task We are also running the first agent with gpt-4.1-mini and the second agent with the more powerful gpt-4.1. Notice how we wrote our own dspy.Module called JokeGenerator. Here we use two separate dspy modules - the query_to_idea and the idea_to_joke to convert our original query to a JokeIdea and subsequently into a joke (as pictured above). python

class JokeIdea(BaseModel):
    setup: str
    contradiction: str
    punchline: str

class QueryToIdea(dspy.Signature):
    """Generate a joke idea with setup, contradiction, and punchline."""
    query = dspy.InputField()
    joke_idea: JokeIdea = dspy.OutputField()

class IdeaToJoke(dspy.Signature):
    """Convert a joke idea into a full comedian delivery."""
    joke_idea: JokeIdea = dspy.InputField()
    joke = dspy.OutputField()

class JokeGenerator(dspy.Module):
    def __init__(self):
        self.query_to_idea = dspy.Predict(QueryToIdea)
        self.idea_to_joke = dspy.Predict(IdeaToJoke)
        
        self.query_to_idea.set_lm(lm=dspy.LM("openai/gpt-4.1-mini"))
        self.idea_to_joke.set_lm(lm=dspy.LM("openai/gpt-4.1"))

    
    def forward(self, query):
        idea = self.query_to_idea(query=query)
        joke = self.idea_to_joke(joke_idea=idea.joke_idea)
        return joke

Iterative Refinement

You can also implement iterative improvement where the LLM reflects on and refines its outputs. For example, we can write a refinement module whose context is the output of a previous LM, and it must act as a feedback provider. The first LM can input this feedback and iteratively improve its response. Iterative refinement An illustration of Iterative refinement. The Idea LM produces a "Setup", "Contradiction", and "Punchline" for a joke. The Joke LM generates a joke out of it. The Refinement LM provides feedback to the Joke LM to
iteratively improve the final joke. (Source: Author)

Conditional Branching and Multi-Output Systems

The agents are connected with control flows, and we orchestrate exactly how information flows through our system Sometimes you want your agent to output multiple variations, and then select the best among them. Let's look at an example of that. Here we have first defined a joke judge - it inputs several joke ideas, and then picks the index of the best joke. This joke is then passed into the next section. python

num_samples = 5

class JokeJudge(dspy.Signature):
    """Given a list of joke ideas, you must pick the best joke"""
    joke_ideas: list[JokeIdeas] = dspy.InputField()
    best_idx: int = dspy.OutputField(
        le=num_samples,
        ge=1,
        description="The index of the funniest joke")

class ConditionalJokeGenerator(dspy.Module):
    def __init__(self):
        self.query_to_idea = dspy.ChainOfThought(QueryToIdea)
        self.judge = dspy.ChainOfThought(JokeJudge)
        self.idea_to_joke = dspy.ChainOfThought(IdeaToJoke)
    
    async def forward(self, query):
        # Generate multiple ideas in parallel
        ideas = await asyncio.gather(*[
            self.query_to_idea.acall(query=query) 
            for _ in range(num_samples)
        ])
        
        # Judge and rank ideas
        best_idx = (await self.judge.acall(joke_ideas=ideas)).best_idx
        
        # Select best idea and generate final joke
        best_idea = ideas[best_idx]

        # Convert from idea to joke
        return await self.idea_to_joke.acall(joke_idea=best_idea)

Tool Calling

LLM applications often need to interact with external systems. This is where tool-calling steps in. You can imagine a tool to be any Python function. You just need two things to define a Python function as an LLM tool:

Tool Calling An example of a tool: Web Search. Given a query, the LLM decides if a web search is necessary, generates a query for the web if so, and then incorporates its search results to generate the final answer (Illustration by Author) Let's see an example of fetching news. We first write a simple Python function, where we use Tavily. The function inputs a search query and fetches recent news articles from the last 7 days. python

client = TavilyClient(api_key=os.getenv("TAVILY_API_KEY"))

def fetch_recent_news(query: str) -> str:
    """Inputs a query string, searches for news and returns top results."""
    response = tavily_client.search(query, search_depth="advanced", 
                                    topic="news", days=7, max_results=3)
    return [x["content"] for x in response["results"]]

Now let's usedspy.ReAct (or the REasoning and ACTing). The module automatically reasons about the user's query, decides when to call which tools, and incorporates the tool results into the final response. Doing this is pretty easy: python

class HaikuGenerator(dspy.Signature):
    """
Generates a haiku about the latest news on the query.
Also create a simple file where you save the final summary.
    """
    query = dspy.InputField()
    summary = dspy.OutputField(desc="A summary of the latest news")
    haiku = dspy.OutputField()

program = dspy.ReAct(signature=HaikuGenerator,
                     tools=[fetch_recent_news],
                     max_iters=2)

program.set_lm(lm=dspy.LM("openai/gpt-4.1", temperature=0.7))
pred = program(query="OpenAI")

When the above code runs, the LLM first reasons about what the user wants and which tool to call (if any). Then it generates the name of the function and the arguments to call the function. We call the news function with the generated args, execute the function to generate the news data. This information is passed back into the LLM. The LLM makes a decision whether to call more tools, or "finish". If the LLM reasons that it has enough information to answer the user's original request, it chooses to finish, and generate the answer.

Agents can take actions while generating responses. Each action the agent can take should be well defined so the LLM can interact with it through reasoning and acting.

Advanced Tool Usage — Scratchpad and File I/O

An evolving standard for modern applications is to allow LLMs access to the file system, allowing them to read and write files, move between directories (with appropriate restrictions), grep and search text within files, and even run terminal commands! This pattern opens a ton of possibilities. It transforms the LLM from a passive text generator into an active agent capable of performing complex, multi-step tasks directly within a user's environment. For example, just displaying the list of tools available to Gemini CLI will reveal a short but incredibly powerful collection of tools. Gemini CLI Tools A screenshot of the default tools available via Gemini CLI

A quick word on MCP Servers

Another new paradigm in the space of agentic systems are MCP servers. MCPs need their own dedicated article, so I won't go over them in detail in this one. This has quickly become the industry-standard way to serve specialized tools to LLMs. It follows the classic Client-Server architecture where the LLM (a client) sends a request to the MCP server, and the MCP server carries out the requested action, and returns a result back to the LLM for downstream processing. MCPs are perfect for context engineering specific examples since you can declare system prompt formats, resources, restricted database access, etc, to your application. This repository has a great list of MCP servers that you can study to make your LLM applications connect with a wide variety of applications.

Retrieval-Augmented Generation (RAG)

Retrieval Augmented Generation has become a cornerstone of modern AI application development. It is an architectural approach that injects external, relevant, and up-to-date information into the Large Language Models (LLMs) that is contextually relevant to the user's query. RAG pipelines consist of a preprocessing and an inference-time phase. During pre-processing, we process the reference data corpus and save it in a queryable format. In the inference phase, we process the user query, retrieve relevant documents from our database, and pass them into the LLM to generate a response.

The information available to the agents can come from multiple sources - external database with Retrieval-Augmented Generation (RAG), tool calls (like web search), memory systems, or classic few-shot examples. Building RAGs is complicated, and there has been a lot of great research and engineering optimizations that have made life easier. I made a 17-minute video that covers all the aspects of building a reliable RAG pipeline. Please accept cookies to access this content

Some practical tips for Good RAG

python

class ChunkAnnotator(dspy.Signature):
    chunk: str = dspy.InputField()
    possible_questions: list[str] = dspy.OutputField(
           description="list of questions that this chunk answers"
           )

python

class QueryRewriting(dspy.Signature):
    user_query: str = dspy.InputField()
    conversation: str = dspy.InputField(
           description="The conversation so far")
    modified_query: str = dspy.OutputField(
           description="a query contextualizing the user query with the conversation's context and optimized for retrieval search"
           )

Direct retrieval vs Query rewriting vs HYDE Direct Retrieval vs Query Rewriting vs HYDE (Source: Author)

Multi-Hop Retrieval Multi-Hop Retrieval and Hybrid HyDE Search (Illustrated by Author)

python

class MultiHopHyDESearch(dspy.Module):
    def __init__(self, retriever):
        self.generate_queries = dspy.ChainOfThought(QueryGeneration)
        self.retriever = retriever
    
    def forward(self, query, n_hops=3):
        results = []
        
        for hop in range(n_hops): # Notice we loop multiple times

            # Generate optimized search queries
            search_queries = self.generate_queries(
                query=query, 
                previous_jokes=retrieved_jokes
            )
            
            # Retrieve using both semantic and keyword search
            semantic_results = self.retriever.semantic_search(
                search_queries.semantic_query
            )
            bm25_results = self.retriever.bm25_search(
                search_queries.bm25_query
            )
            
            # Fuse results
            hop_results = reciprocal_rank_fusion([
                semantic_results, bm25_results
            ])
            results.extend(hop_results)
        
        return results

The Mem0 architecture (Source: The Mem0 paper)

Best Practices and Production Considerations

This section is not directly about Context Engineering, but more about best practices to build LLM apps for production.

Additionally, systems need to be evaluated with metrics and maintained with observability. Monitoring token usage, latency, and cost to output quality is a key consideration.

1. Design Evaluation First

Before building features, decide how you'll measure success. This helps scope your application and guides optimization decisions.

Hyperparameters impacting LLM outputs A lot of parameters impact the quality of LLM's outputs (Illustrated by the author)

Evaluation of LLM apps A simple flowchart about evaluating LLM apps (Illustration by author)

3. Use Structured Outputs Almost Everywhere

Always prefer structured outputs over free-form text. It makes your system more reliable and easier to debug. You can add validation and retries as well!

4. Design for failure

When designing prompts or dspy modules, make sure you always consider "what happens if things go wrong?" Like any good software, cutting down error states and failing with swagger is the ideal scenario.

5. Monitor Everything

DSpy integrates with MLflow to track:

Langfuse, Logfire are equally great alternatives.

Outro

Context engineering represents a paradigm shift from simple prompt engineering to building comprehensive and modular LLM applications. The DSPy framework provides the tools and abstractions needed to implement these patterns systematically. As LLM capabilities continue to evolve, context engineering will become increasingly crucial for building applications that effectively leverage the power of large language models. To watch the full video course on which this article is based, please visit this YouTube link. Please accept cookies to access this content To access the full GitHub repo, visit: https://github.com/avbiswas/context-engineering-dspy Visit the Context Engineering repo for code access!

References

Author's YouTube channel : https://www.youtube.com/@avb_fj Author's Patreon : www.patreon.com/NeuralBreakdownwithAVB Author's Twitter (X) account : https://x.com/neural_avb Full Context Engineering video course:https://youtu.be/5Bym0ffALaU Github Link:https://github.com/avbiswas/context-engineering-dspy

Register Now to Continue Reading

Unlock Towards Data Science with a Free Account

All articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more. Register for freeSign in to your account

Written By

Avishek Biswas

See all from Avishek Biswas Deep DivesContext EngineeringDspyLlm ApplicationsMachine Learning Share This Article

Towards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program. Write for TDS

Related Articles

Pandas Indexes And Headers, Have You Ever Been Confused?

Artificial Intelligence From single-level index and headers to multi-level, why and how? Christopher Tao June 9, 20248 min read

Mathematical Optimization Heuristics Every Data Scientist Should Know

Machine Learning Local search, genetic algorithms and more. Hennie de Harder December 9, 20229 min read

Increase Llama 2's Latency and Throughput Performance by Up to 4X

Artificial Intelligence Real-world benchmarks for Llama-2 13B Het Trivedi August 8, 20237 min read

Step-by-step Approach of Building Data Pipelines as a Data Scientist or a Machine Learning Engineer

Deep Learning Learn how to build interesting pipelines in the field of machine learning to develop a system that can perform AI capabilities Suhas Maddali November 2, 20228 min read

What is Universality in LLMs? How to Find Universal Neurons

Large Language Models How independently trained transformers form same the neurons Shuyang Xiang September 2, 20255 min read

fit() vs predict() vs fit_predict() in Python scikit-learn

Machine Learning What's the difference between fit, predict and fit_predict methods in sklearn Giorgos Myrianthous March 9, 20215 min read

Simulated Data, Real Learnings : Power Analysis

Machine Learning Part 2 - Experimental Power Analysis Jarom Hulet March 26, 202411 min read

How to Develop Powerful Internal LLM Benchmarks

Large Language Models Learn how to compare LLMs using your own internal benchmark Eivind Kjosbakken August 26, 20256 min read

How to use Causal Inference when A/B testing is not available

Machine Learning Evaluating ad targeting product using causal inference: propensity score matching! Harry Lu January 8, 20247 min read

Your home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals. Subscribe to Our NewsletterWrite For TDS•About•Advertise•Privacy Policy•Terms of Use © Insight Media Group, LLC 2026•The New Stack•roadmap.sh•Creator Network