![Revisit consent button](../images/body-df29beacd7496220cc94.svg) We value your privacy We use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking "Accept All", you consent to our use of cookies. Customise Reject All Accept All Customise Consent Preferences ![](../images/body-2159bb67f3de8ad870eb.svg) We use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below. The cookies that are categorised as "Necessary" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... Show more NecessaryAlways Active Necessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data. * Cookie BCTempID * Duration 10 minutes * Description No description available. * Cookie __cf_bm * Duration 1 hour * Description This cookie, set by Cloudflare, is used to support Cloudflare Bot Management. * Cookie AWSALBCORS * Duration 7 days * Description Amazon Web Services set this cookie for load balancing. * Cookie _cfuvid * Duration session * Description Cloudflare sets this cookie to track users across sessions to optimize user experience by maintaining session consistency and providing personalized services * Cookie li_gc * Duration 6 months * Description Linkedin set this cookie for storing visitor's consent regarding using cookies for non-essential purposes. * Cookie __hssrc * Duration session * Description This cookie is set by Hubspot whenever it changes the session cookie. The __hssrc cookie set to 1 indicates that the user has restarted the browser, and if the cookie does not exist, it is assumed to be a new session. * Cookie __hssc * Duration 1 hour * Description HubSpot sets this cookie to keep track of sessions and to determine if HubSpot should increment the session number and timestamps in the __hstc cookie. * Cookie wpEmojiSettingsSupports * Duration session * Description WordPress sets this cookie when a user interacts with emojis on a WordPress site. It helps determine if the user's browser can display emojis properly. * Cookie BCSessionID * Duration 1 year 1 month 4 days * Description Blueconic sets this cookie as a unique identifier for the BlueConic profile. * Cookie _octo * Duration 1 year * Description No description available. * Cookie logged_in * Duration 1 year * Description No description available. * Cookie __Secure-YEC * Duration past * Description YouTube sets this cookie to stores the user's video player preferences using embedded YouTube video * Cookie __eoi * Duration 6 months * Description Description is currently not available. * Cookie AWSALBTGCORS * Duration 7 days * Description No description available. * Cookie login-status-p * Duration past * Description Description is currently not available. * Cookie AWSALBTG * Duration 7 days * Description No description available. * Cookie csrf_token * Duration session * Description No description available. * Cookie token_v2 * Duration 1 day * Description Description is currently not available. * Cookie D * Duration 1 year * Description Description is currently not available. * Cookie PHPSESSID * Duration session * Description This cookie is native to PHP applications. The cookie stores and identifies a user's unique session ID to manage user sessions on the website. The cookie is a session cookie and will be deleted when all the browser windows are closed. * Cookie VISITOR_PRIVACY_METADATA * Duration 6 months * Description YouTube sets this cookie to store the user's cookie consent state for the current domain. * Cookie cookietest * Duration session * Description The cookietest cookie is typically used to determine whether the user's browser accepts cookies, essential for website functionality and user experience. * Cookie __Host-airtable-session * Duration 1 year * Description This cookie is used to enable us to integrate the services of Airtable. * Cookie __Host-airtable-session.sig * Duration 1 year * Description This cookie is used to enable us to integrate the services of Airtable. * Cookie m * Duration 1 year 1 month 4 days * Description Stripe sets this cookie for fraud prevention purposes. It identifies the device used to access the website, allowing the website to be formatted accordingly. * Cookie BIGipServer* * Duration session * Description Marketo sets this cookie to collect information about the user's online activity and build a profile about their interests to provide advertisements relevant to the user. * Cookie __cfruid * Duration session * Description Cloudflare sets this cookie to identify trusted web traffic. * Cookie _GRECAPTCHA * Duration 6 months * Description Google Recaptcha service sets this cookie to identify bots to protect the website against malicious spam attacks. * Cookie __Secure-YNID * Duration 6 months * Description Google cookie used to protect user security and prevent fraud, especially during the login process. * Cookie cookieyes-consent * Duration 1 year * Description CookieYes sets this cookie to remember users' consent preferences so that their preferences are respected on subsequent visits to this site. It does not collect or store any personal information about the site visitors. Functional Functional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features. * Cookie lidc * Duration 1 day * Description LinkedIn sets the lidc cookie to facilitate data center selection. * Cookie brw * Duration 1 year * Description No description available. * Cookie brwConsent * Duration 5 minutes * Description Description is currently not available. * Cookie WMF-Uniq * Duration 1 year * Description Description is currently not available. * Cookie loom_anon_comment * Duration 1 year * Description No description available. * Cookie loom_referral_video * Duration session * Description Description is currently not available. * Cookie VISITOR_INFO1_LIVE * Duration 6 months * Description A cookie set by YouTube to measure bandwidth that determines whether the user gets the new or old player interface. * Cookie yt-remote-connected-devices * Duration Never Expires * Description YouTube sets this cookie to store the user's video preferences using embedded YouTube videos. * Cookie ytidb::LAST_RESULT_ENTRY_KEY * Duration Never Expires * Description The cookie ytidb::LAST_RESULT_ENTRY_KEY is used by YouTube to store the last search result entry that was clicked by the user. This information is used to improve the user experience by providing more relevant search results in the future. * Cookie yt-remote-device-id * Duration Never Expires * Description YouTube sets this cookie to store the user's video preferences using embedded YouTube videos. * Cookie yt-remote-session-name * Duration session * Description The yt-remote-session-name cookie is used by YouTube to store the user's video player preferences using embedded YouTube video. * Cookie yt-remote-fast-check-period * Duration session * Description The yt-remote-fast-check-period cookie is used by YouTube to store the user's video player preferences for embedded YouTube videos. * Cookie yt-remote-session-app * Duration session * Description The yt-remote-session-app cookie is used by YouTube to store user preferences and information about the interface of the embedded YouTube video player. * Cookie yt-remote-cast-available * Duration session * Description The yt-remote-cast-available cookie is used to store the user's preferences regarding whether casting is available on their YouTube video player. * Cookie yt-remote-cast-installed * Duration session * Description The yt-remote-cast-installed cookie is used to store the user's video player preferences using embedded YouTube video. * Cookie cp_session * Duration 3 months * Description Codepen sets this cookie for Help systems found in the website. * Cookie loid * Duration 1 year 1 month 4 days * Description This cookie is set by the Reddit. The cookie enables the sharing of content from the website onto the social media platform. Analytics Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc. * Cookie __hstc * Duration 6 months * Description Hubspot set this main cookie for tracking visitors. It contains the domain, initial timestamp (first visit), last timestamp (last visit), current timestamp (this visit), and session number (increments for each subsequent session). * Cookie hubspotutk * Duration 6 months * Description HubSpot sets this cookie to keep track of the visitors to the website. This cookie is passed to HubSpot on form submission and used when deduplicating contacts. * Cookie _ga * Duration 1 year 1 month 4 days * Description Google Analytics sets this cookie to calculate visitor, session and campaign data and track site usage for the site's analytics report. The cookie stores information anonymously and assigns a randomly generated number to recognise unique visitors. * Cookie _ga_* * Duration 1 year 1 month 4 days * Description Google Analytics sets this cookie to store and count page views. * Cookie __Host-psifi.analyticsTrace * Duration 6 hours * Description Description is currently not available. * Cookie __Host-psifi.analyticsTraceV2 * Duration 6 hours * Description Description is currently not available. * Cookie _gh_sess * Duration session * Description GitHub sets this cookie for temporary application and framework state between pages like what step the user is on in a multiple step form. * Cookie YSC * Duration session * Description YSC cookie is set by Youtube and is used to track the views of embedded videos on Youtube pages. * Cookie ajs_anonymous_id * Duration 1 year * Description This cookie is set by Segment to count the number of people who visit a certain site by tracking if they have visited before. * Cookie vuid * Duration 1 year 1 month 4 days * Description Vimeo installs this cookie to collect tracking information by setting a unique ID to embed videos on the website. Performance Performance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors. * Cookie AWSALB * Duration 7 days * Description AWSALB is an application load balancer cookie set by Amazon Web Services to map the session to the target. * Cookie acq * Duration past * Description Description is currently not available. * Cookie acq.sig * Duration past * Description Description is currently not available. * Cookie ptc * Duration 2 years * Description No description available. Advertisement Advertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns. * Cookie muc_ads * Duration 1 year 1 month 4 days * Description Twitter sets this cookie to collect user behaviour and interaction data to optimize the website. * Cookie guest_id_marketing * Duration 1 year 1 month 4 days * Description Twitter sets this cookie to identify and track the website visitor. * Cookie guest_id_ads * Duration 1 year 1 month 4 days * Description Twitter sets this cookie to identify and track the website visitor. * Cookie personalization_id * Duration 1 year 1 month 4 days * Description Twitter sets this cookie to integrate and share features for social media and also store information about how the user uses the website, for tracking and targeting. * Cookie guest_id * Duration 1 year 1 month 4 days * Description Twitter sets this cookie to identify and track the website visitor. It registers if a user is signed in to the Twitter platform and collects information about ad preferences. * Cookie bcookie * Duration 1 year * Description LinkedIn sets this cookie from LinkedIn share buttons and ad tags to recognize browser IDs. * Cookie __Secure-ROLLOUT_TOKEN * Duration 6 months * Description YouTube sets this cookie to manage feature rollout and experimentation. It helps Google control which new features or interface changes are shown to users as part of testing and staged rollouts, ensuring consistent experience for a given user during an experiment. * Cookie yt.innertube::nextId * Duration Never Expires * Description YouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen. * Cookie yt.innertube::requests * Duration Never Expires * Description YouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen. * Cookie session_tracker * Duration session * Description This cookie is set by the Reddit. This cookie is used to identify trusted web traffic. It also helps in adverstising on the website. * Cookie edgebucket * Duration session * Description Reddit sets this cookie to save the information about a log-on Reddit user, for the purpose of advertisement recommendations and updating the content. * Cookie did * Duration 1 year * Description Arbor sets this cookie to show targeted ads to site visitors.This cookie expires after 2 months or 1 year. Uncategorised Other uncategorised cookies are those that are being analysed and have not been classified into a category as yet. No cookies to display. Reject All Save My Preferences Accept All Toggle Search[](https://towardsdatascience.com/)Toggle Mobile Navigation [](https://towardsdatascience.com/) Publish AI, ML & data-science insights to a global community of data professionals. Sign in[Submit an Article](https://write.insightmediagroup.io) [Latest](https://towardsdatascience.com/latest/)[Editor's Picks](https://towardsdatascience.com/tag/editors-pick/)[Newsletter](https://towardsdatascience.com/tag/the-variable/)[ShipAI](https://towardsdatascience.com/shipai/) * * * [Write For TDS](https://towardsdatascience.com/submissions/) [LinkedIn](https://www.linkedin.com/company/towards-data-science/)[X](https://x.com/TDataScience)Toggle Search [Large Language Models](https://towardsdatascience.com/category/artificial-intelligence/large-language-models/) # Context Engineering — A Comprehensive Hands-On Tutorial with DSPy Let's dissect the art and science of context engineering, one module at a time! [Avishek Biswas](https://towardsdatascience.com/author/neural.avb/) August 5, 202515 min read ![](../images/000.png) You may have heard about **Context Engineering** by now. This article will cover the key ideas behind creating LLM applications using Context Engineering principles, visually explain these workflows, and share code snippets that apply these concepts practically. Don't worry about copy-pasting the code from this article into your editor. **At the end of this article, I will share the GitHub link to the open-source code repository and a link to my 1-hour 20-minute YouTube course that explains the concepts presented here in greater detail.** _Unless otherwise mentioned, all images used in this article are produced by the author and are free to use._ Let's begin! ··· ## What is Context Engineering? There is a significant gap between writing simple prompts and building production-ready applications. Context Engineering is an umbrella term that refers to the delicate art and science of fitting information into the context window of an LLM as it works on a task. The exact scope of where the definition of Context Engineering begins and ends is debatable, but according to [this tweet from Andrej Karpathy](https://x.com/karpathy/status/1937902205765607626), we can identify the following key points: * It is not _just_ atomic prompt engineering, where you ask one question to the LLM and get a response * It is a holistic approach that breaks up a larger problem into multiple subproblems * These subproblems can be solved by multiple LLMs (or agents) in isolation. Each agent is provided with the appropriate context to carry out its task * Each agent can be of appropriate capability and size depending on the complexity of the task. * Intermediate steps that each agent can take to complete the task - the context is _not just_ information we input - it also includes _intermediate tokens_ that the LLMs see during generation (eg. reasoning steps, tool results, etc) * The agents are connected with control flows, and we orchestrate exactly how information flows through our system * The information available to the agents can come from multiple sources - external databases with Retrieval-Augmented Generation (RAG), tool calls (like web search), memory systems, or classic few-shot examples. * Agents can take actions while generating responses. Each action the agent can take should be well-defined so the LLM can interact with it through reasoning and acting. * Additionally, systems need to be evaluated with metrics and maintained with observability. Monitoring token usage, latency, and cost to output quality is a key consideration. ### Important: How this article is structured _Throughout this article, I will be referring to the points above while providing examples of how they are applied in building real applications. Whenever I do so, I will use a block quote like this:_ > _It is a holistic approach that breaks up a larger problem into multiple subproblems_ _When you see a quote in this format, the example that follows will apply the quoted concept programmatically_. But before that, we must ask ourselves one question... ### Why not pass _everything_ into the LLM? Research has shown that cramming every piece of information into the context of an LLM is far from ideal. Even though many frontier models do claim to support "long-context" windows, they still suffer from issues like _context poisoning_ or [_context rot_](https://research.trychroma.com/context-rot). ![](../images/001.png) A recent report from Chroma describes how increasing tokens can negatively impact LLM performance (Source: Chroma) Too much unnecessary information in an LLM's context can pollute the model's understanding, lead to hallucinations, and result in poor performance. This is why simply having a large context window isn't enough. We need systematic approaches to context engineering. ### Why DSPY ![](../images/body-b5044f45cb8bbfc59b27.png) Just a logo For this tutorial, I have chosen the DSPy framework. I will explain the reasoning for this choice shortly, but let me assure you that the concepts presented here apply to almost any prompting framework, including writing prompts in pure English. DSPy is a declarative framework for building modular AI software. They have neatly separated the two key aspects of any LLM task — (a) the input and output contracts passed into a module, and (b) the logic that governs how information flows. ### Let's see an example! Imagine we want to use an LLM to write a joke. Specifically, we want it to generate a setup, a punchline, and the full delivery in a comedian's voice. Oh, and we also want the output in JSON format so that we can post-process individual fields of the dictionary after generation. For example, perhaps we want to print the punchline on a T-shirt (assume someone has already written a convenient function for that). python ``` system_prompt = """ You are a comedian who tells jokes, you are always funny. Generate the setup, punchline, and full delivery in the comedian's voice. Output in the following JSON format: { "setup": , "punchline": , "delivery": } Your response should be parsable withou errors in Python using json.loads(). """ client = openai.Client() response = client.chat.completions.create( model="gpt-4o-mini", temperature = 1, messages=[ {"role": "system", "content": system_prompt, {"role": "user", "content": "Write a joke about AI"} ] ) joke = json.loads(response.choices[0].message.content) # Hope for the best print_on_a_tshirt(joke["punchline"]) ``` Notice how we post-process the LLM's response to extract the dictionary? What if something "bad" happened, like the LLM failing to generate the response in the desired format? Our entire code would fail and there will be no printing on any T-shirts! The above code is also quite difficult to extend. For example, if we wanted the LLM to do chain of thought reasoning before generating the answer, we would need to write additional logic to parse that reasoning text correctly. Furthermore, it can be difficult to look at plain English prompts like these and understand what the inputs and outputs of these systems are. DSPy solves all of the above. Let's write the above example using DSPy. python ``` class JokeGenerator(dspy.Signature): """You're a comedian who tells jokes. You're always funny.""" query: str = dspy.InputField() setup: str = dspy.OutputField() punchline: str = dspy.OutputField() delivery: str = dspy.OutputField() joke_gen = dspy.Predict(JokeGenerator) joke_gen.set_lm(lm=dspy.LM("openai/gpt-4.1-mini", temperature=1)) result = joke_gen(query="Write a joke about AI") print(result) print_on_a_tshirt(result.punchline) ``` This approach gives you structured, predictable outputs that you can work with programmatically, eliminating the need for regex parsing or error-prone string manipulation. Dspy Signatures explicitly makes you define what the inputs to the system are ("query" in the above example), and the outputs to the system (setup, punchline, and delivery) as well as their data-types. It also tells the LLM the order in which you want them to be generated. ![](../images/body-5dfae2251dd20b0701d0.png) The output of the previous code block (minus the t-shirt stuff) The `dspy.Predict` thing is an example of a DSPy Module. With modules, you define how the LLM converts from inputs to outputs. `dspy.Predict` is the most basic one - you can pass the query to it, as in `joke_gen(query="Write a joke about AI")` and it will create a basic prompt to send to the LLM. Internally, DSPy just creates a prompt as you can see below. Once the LLM responds, DSPy will create Pydantic `BaseModel` objects that perform automatic schema validation and send back the output. If errors occur during this validation process, DSPy automatically attempts to fix them by re-prompting the LLM—thereby significantly reducing the risk of a program crash. ![Dspy.Predict vs Dspy.ChainOfThought](../images/002.jpeg) In chain of thought, we ask the LLM to generate reasoning text before generating the answer (Source: Author) Another common theme in context engineering is Chain of Thought. Here, we want the LLM to generate reasoning text before providing its final answer. This allows the LLM's context to be populated with its self-generated reasoning before it generates the final output tokens. To do that, you can simply replace `dspy.Predict` with `dspy.ChainOfThought` in the example above. The rest of the code remains the same. Now you can see that the LLM generates reasoning before the defined output fields. ## Multi-Step Interactions and Agentic Workflows The best part of DSPy's approach is how it decouples system dependencies (`Signatures`) from control flows (`Modules`), which makes writing code for multi-step interactions trivial (and fun!). In this section, let's see how we can build some simple agentic flows. ### Sequential Processing Let's remind ourselves about one of the key components of Context Engineering. > _It is a holistic approach that breaks up a larger problem into multiple subproblems_ Let's continue with our joke generation example. We can easily separate out two subproblems from it. Generating the idea is one, creating a joke is another. ![Sequential flows](../images/003.jpeg) Sequential flows allow us to design LLM systems in a modular way where each agent can be of appropriate strength/size and is given context and tools that are appropriate for its task (Illustrated by author) Let's have two agents then — the first Agent generates a joke idea (setup and punchline) from a query. A second agent then generates the joke from this idea. > _Each agent can be of appropriate capability and size depending on the complexity of the task_ We are also running the first agent with `gpt-4.1-mini` and the second agent with the more powerful `gpt-4.1`. Notice how we wrote our own `dspy.Module` called `JokeGenerator`. Here we use two separate dspy modules - the `query_to_idea` and the `idea_to_joke` to convert our original query to a `JokeIdea` and subsequently into a joke (as pictured above). python ``` class JokeIdea(BaseModel): setup: str contradiction: str punchline: str class QueryToIdea(dspy.Signature): """Generate a joke idea with setup, contradiction, and punchline.""" query = dspy.InputField() joke_idea: JokeIdea = dspy.OutputField() class IdeaToJoke(dspy.Signature): """Convert a joke idea into a full comedian delivery.""" joke_idea: JokeIdea = dspy.InputField() joke = dspy.OutputField() class JokeGenerator(dspy.Module): def __init__(self): self.query_to_idea = dspy.Predict(QueryToIdea) self.idea_to_joke = dspy.Predict(IdeaToJoke) self.query_to_idea.set_lm(lm=dspy.LM("openai/gpt-4.1-mini")) self.idea_to_joke.set_lm(lm=dspy.LM("openai/gpt-4.1")) def forward(self, query): idea = self.query_to_idea(query=query) joke = self.idea_to_joke(joke_idea=idea.joke_idea) return joke ``` ### Iterative Refinement You can also implement iterative improvement where the LLM reflects on and refines its outputs. For example, we can write a refinement module whose context is the output of a previous LM, and it must act as a feedback provider. The first LM can input this feedback and iteratively improve its response. ![Iterative refinement](../images/004.jpeg) An illustration of Iterative refinement. The Idea LM produces a "Setup", "Contradiction", and "Punchline" for a joke. The Joke LM generates a joke out of it. The Refinement LM provides feedback to the Joke LM to iteratively improve the final joke. (Source: Author) ### Conditional Branching and Multi-Output Systems > _The agents are connected with control flows, and we orchestrate exactly how information flows through our system_ Sometimes you want your agent to output multiple variations, and then select the best among them. Let's look at an example of that. Here we have first defined a joke judge - it inputs several joke ideas, and then picks the index of the best joke. This joke is then passed into the next section. python ``` num_samples = 5 class JokeJudge(dspy.Signature): """Given a list of joke ideas, you must pick the best joke""" joke_ideas: list[JokeIdeas] = dspy.InputField() best_idx: int = dspy.OutputField( le=num_samples, ge=1, description="The index of the funniest joke") class ConditionalJokeGenerator(dspy.Module): def __init__(self): self.query_to_idea = dspy.ChainOfThought(QueryToIdea) self.judge = dspy.ChainOfThought(JokeJudge) self.idea_to_joke = dspy.ChainOfThought(IdeaToJoke) async def forward(self, query): # Generate multiple ideas in parallel ideas = await asyncio.gather(*[ self.query_to_idea.acall(query=query) for _ in range(num_samples) ]) # Judge and rank ideas best_idx = (await self.judge.acall(joke_ideas=ideas)).best_idx # Select best idea and generate final joke best_idea = ideas[best_idx] # Convert from idea to joke return await self.idea_to_joke.acall(joke_idea=best_idea) ``` ## Tool Calling LLM applications often need to interact with external systems. This is where tool-calling steps in. You can imagine a tool to be any Python function. You just need two things to define a Python function as an LLM tool: * A description of what the function does * A list of inputs and their data types ![Tool Calling](../images/005.jpeg) An example of a tool: Web Search. Given a query, the LLM decides if a web search is necessary, generates a query for the web if so, and then incorporates its search results to generate the final answer (Illustration by Author) Let's see an example of fetching news. We first write a simple Python function, where we use [Tavily](https://tavily.com/). The function inputs a search query and fetches recent news articles from the last 7 days. python ``` client = TavilyClient(api_key=os.getenv("TAVILY_API_KEY")) def fetch_recent_news(query: str) -> str: """Inputs a query string, searches for news and returns top results.""" response = tavily_client.search(query, search_depth="advanced", topic="news", days=7, max_results=3) return [x["content"] for x in response["results"]] ``` Now let's use`dspy.ReAct` (or the REasoning and ACTing). The module automatically reasons about the user's query, decides when to call which tools, and incorporates the tool results into the final response. Doing this is pretty easy: python ``` class HaikuGenerator(dspy.Signature): """ Generates a haiku about the latest news on the query. Also create a simple file where you save the final summary. """ query = dspy.InputField() summary = dspy.OutputField(desc="A summary of the latest news") haiku = dspy.OutputField() program = dspy.ReAct(signature=HaikuGenerator, tools=[fetch_recent_news], max_iters=2) program.set_lm(lm=dspy.LM("openai/gpt-4.1", temperature=0.7)) pred = program(query="OpenAI") ``` When the above code runs, the LLM first reasons about what the user wants and which tool to call (if any). Then it generates the name of the function and the arguments to call the function. We call the news function with the generated args, execute the function to generate the news data. This information is passed back into the LLM. The LLM makes a decision whether to call more tools, or "finish". If the LLM reasons that it has enough information to answer the user's original request, it chooses to finish, and generate the answer. > _Agents can take actions while generating responses. Each action the agent can take should be well defined so the LLM can interact with it through reasoning and acting._ #### Advanced Tool Usage — Scratchpad and File I/O An evolving standard for modern applications is to allow LLMs access to the file system, allowing them to read and write files, move between directories (with appropriate restrictions), grep and search text within files, and even run terminal commands! This pattern opens a ton of possibilities. It transforms the LLM from a passive text generator into an active agent capable of performing complex, multi-step tasks directly within a user's environment. For example, just displaying the list of tools available to Gemini CLI will reveal a short but incredibly powerful collection of tools. ![Gemini CLI Tools](../images/006.png) A screenshot of the default tools available via Gemini CLI ### A quick word on MCP Servers Another new paradigm in the space of agentic systems are MCP servers. MCPs need their own dedicated article, so I won't go over them in detail in this one. This has quickly become the industry-standard way to serve specialized tools to LLMs. It follows the classic Client-Server architecture where the LLM (a client) sends a request to the MCP server, and the MCP server carries out the requested action, and returns a result back to the LLM for downstream processing. MCPs are perfect for context engineering specific examples since you can declare system prompt formats, resources, restricted database access, etc, to your application. [This repository has a great list of MCP servers](https://github.com/punkpeye/awesome-mcp-servers) that you can study to make your LLM applications connect with a wide variety of applications. ## Retrieval-Augmented Generation (RAG) Retrieval Augmented Generation has become a cornerstone of modern AI application development. It is an architectural approach that injects external, relevant, and up-to-date information into the Large Language Models (LLMs) that is contextually relevant to the user's query. RAG pipelines consist of a preprocessing and an inference-time phase. During pre-processing, we process the reference data corpus and save it in a queryable format. In the inference phase, we process the user query, retrieve relevant documents from our database, and pass them into the LLM to generate a response. > _The information available to the agents can come from multiple sources - external database with Retrieval-Augmented Generation (RAG), tool calls (like web search), memory systems, or classic few-shot examples._ Building RAGs is complicated, and there has been a lot of great research and engineering optimizations that have made life easier. I made a 17-minute video that covers all the aspects of building a reliable RAG pipeline. Please accept cookies to access this content ### Some practical tips for Good RAG * When preprocessing, generate additional metadata per chunk. This can be as simple as "questions this chunk answers". When saving the chunks to your database, also save the generated metadata! python ``` class ChunkAnnotator(dspy.Signature): chunk: str = dspy.InputField() possible_questions: list[str] = dspy.OutputField( description="list of questions that this chunk answers" ) ``` * **Query Rewriting:** Directly using the user's query to do RAG retrieval is often a bad idea. Users write pretty random things, which may not match the distribution of text in your corpus. Query rewriting does what it says - it "rewrites" the query, perhaps fixing grammar, spelling errors, contextualizes it with past conversation, or even adds additional keywords that make querying easier. python ``` class QueryRewriting(dspy.Signature): user_query: str = dspy.InputField() conversation: str = dspy.InputField( description="The conversation so far") modified_query: str = dspy.OutputField( description="a query contextualizing the user query with the conversation's context and optimized for retrieval search" ) ``` * HYDE or **Hypothetical Document Embedding** is a type of Query Rewriting system. In HYDE, we generate an artificial (or hypothetical) answer from the LLM's internal knowledge. This response often contains important keywords that try to directly match with the answers database. Vanilla query rewriting is great for searching a database of questions, and HYDE is great for searching a database with answers. ![Direct retrieval vs Query rewriting vs HYDE](../images/007.jpeg) Direct Retrieval vs Query Rewriting vs HYDE (Source: Author) * Hybrid search is almost always better than purely semantic or purely keyword-based search. For semantic search, I'd use cosine similarity nearest neighbor search with vector embeddings. And for semantic search, use BM25. * **RRF:** You can choose multiple strategies to retrieve documents, and then use reciprocal rank fusion to combine them into one unified list! ![Multi-Hop Retrieval](../images/008.jpeg) Multi-Hop Retrieval and Hybrid HyDE Search (Illustrated by Author) * **Multi-Hop Search** is an option to consider as well if you can afford additional latency. Here, you pass the retrieved documents back into the LLM to generate new queries, which are used to conduct additional searches on the database. python ``` class MultiHopHyDESearch(dspy.Module): def __init__(self, retriever): self.generate_queries = dspy.ChainOfThought(QueryGeneration) self.retriever = retriever def forward(self, query, n_hops=3): results = [] for hop in range(n_hops): # Notice we loop multiple times # Generate optimized search queries search_queries = self.generate_queries( query=query, previous_jokes=retrieved_jokes ) # Retrieve using both semantic and keyword search semantic_results = self.retriever.semantic_search( search_queries.semantic_query ) bm25_results = self.retriever.bm25_search( search_queries.bm25_query ) # Fuse results hop_results = reciprocal_rank_fusion([ semantic_results, bm25_results ]) results.extend(hop_results) return results ``` * **Citations** : When asking LLM to generate responses from the retrieved documents, we can also ask the LLM to cite references to the documents it found useful. This allows the LLM to first generate a plan of how it's going to use the retrieved content. * **Memory:** If you are building a chatbot, it is important to figure out the question of memory. You can imagine Memory as a combination of Retrieval and Tool Calling. A well-known system is the Mem0 system. The LLM observes new data and calls tools to decide if it needs to add or modify its existing memories. During question-answering, it retrieves relevant memories using RAG to generate answers. ![](../images/body-9c37cb386150d1555dc7.png) The Mem0 architecture (Source: [The Mem0 paper](https://arxiv.org/pdf/2504.19413)) ## Best Practices and Production Considerations This section is not directly about Context Engineering, but more about best practices to build LLM apps for production. > _Additionally, systems need to be evaluated with metrics and maintained with observability. Monitoring token usage, latency, and cost to output quality is a key consideration._ ### 1. Design Evaluation First Before building features, decide how you'll measure success. This helps scope your application and guides optimization decisions. ![Hyperparameters impacting LLM outputs](../images/body-d2d7115f6f0a93b22ccd.jpeg) A lot of parameters impact the quality of LLM's outputs (Illustrated by the author) * If you can design verifiable or objective rewards, that's the best. (example: classification tasks where you have a validation dataset) * If not, can you define functions that heuristically evaluate LLM responses for your use case? (example: number of times a specific chunk is retrieved given a question) * If not, can you get humans to annotate your LLM's responses? * If nothing works, use an LLM as a judge to evaluate responses. In most cases, you want to set your evaluation task as a comparison study, where the Judge receives multiple responses produced using different hyperparameters/prompts, and the judge must rank which ones are the best. ![Evaluation of LLM apps](../images/body-0c95d42eaa7a48114312.jpeg) A simple flowchart about evaluating LLM apps (Illustration by author) ### 3. Use Structured Outputs Almost Everywhere Always prefer structured outputs over free-form text. It makes your system more reliable and easier to debug. You can add validation and retries as well! ### 4. Design for failure When designing prompts or dspy modules, make sure you always consider "what happens if things go wrong?" Like any good software, cutting down error states and failing with swagger is the ideal scenario. ### 5. Monitor Everything DSpy integrates with MLflow to track: * Individual prompts passed into the LLM and their responses * Token usage and costs * Latency per module * Success/failure rates * Model performance over time Langfuse, Logfire are equally great alternatives. ## Outro Context engineering represents a paradigm shift from simple prompt engineering to building comprehensive and modular LLM applications. The DSPy framework provides the tools and abstractions needed to implement these patterns systematically. As LLM capabilities continue to evolve, context engineering will become increasingly crucial for building applications that effectively leverage the power of large language models. To watch the full video course on which this article is based, please visit this YouTube link. Please accept cookies to access this content To access the full GitHub repo, visit: ![](../images/body-4dc8bb1a97c2d6148954.png) [Visit the Context Engineering repo for code access! ](https://github.com/avbiswas/context-engineering-dspy) ## References **Author's YouTube channel** : **Author's Patreon** : [www.patreon.com/NeuralBreakdownwithAVB](https://www.patreon.com/c/NeuralBreakdownwithAVB) **Author's Twitter (X) account** : **Full Context Engineering video course:** **Github Link:** ## Register Now to Continue Reading ### Unlock Towards Data Science with a Free Account All articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more. Register for freeSign in to your account * * * ![](../images/body-849620ef9680d77fc51e.jpeg) Written By ## [Avishek Biswas](https://towardsdatascience.com/author/neural.avb/) [See all from Avishek Biswas](https://towardsdatascience.com/author/neural.avb/) [Deep Dives](https://towardsdatascience.com/tag/deep-dives/)[Context Engineering](https://towardsdatascience.com/tag/context-engineering/)[Dspy](https://towardsdatascience.com/tag/dspy/)[Llm Applications](https://towardsdatascience.com/tag/llm-applications/)[Machine Learning](https://towardsdatascience.com/tag/machine-learning/) Share This Article * [](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Ftowardsdatascience.com%2Fcontext-engineering-a-comprehensive-hands-on-tutorial-with-dspy%2F&title=Context%20Engineering%20%E2%80%94%20A%20Comprehensive%20Hands-On%20Tutorial%20with%20DSPy) * [](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Ftowardsdatascience.com%2Fcontext-engineering-a-comprehensive-hands-on-tutorial-with-dspy%2F&title=Context%20Engineering%20%E2%80%94%20A%20Comprehensive%20Hands-On%20Tutorial%20with%20DSPy) * [](https://x.com/share?url=https%3A%2F%2Ftowardsdatascience.com%2Fcontext-engineering-a-comprehensive-hands-on-tutorial-with-dspy%2F&text=Context%20Engineering%20%E2%80%94%20A%20Comprehensive%20Hands-On%20Tutorial%20with%20DSPy) Towards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program. [Write for TDS](https://towardsdatascience.com/submissions/) ## Related Articles * [![](../images/009.png)](https://towardsdatascience.com/pandas-indexes-and-headers-have-you-ever-been-confused-b5a11f143876/) ## [Pandas Indexes And Headers, Have You Ever Been Confused?](https://towardsdatascience.com/pandas-indexes-and-headers-have-you-ever-been-confused-b5a11f143876/) [Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/) From single-level index and headers to multi-level, why and how? [Christopher Tao](https://towardsdatascience.com/author/christophertao/) June 9, 20248 min read * [![](../images/010.jpeg)](https://towardsdatascience.com/mathematical-optimization-heuristics-every-data-scientist-should-know-b26de0bd43e6/) ## [Mathematical Optimization Heuristics Every Data Scientist Should Know](https://towardsdatascience.com/mathematical-optimization-heuristics-every-data-scientist-should-know-b26de0bd43e6/) [Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/) Local search, genetic algorithms and more. [Hennie de Harder](https://towardsdatascience.com/author/hennie-de-harder/) December 9, 20229 min read * [![](../images/011.jpeg)](https://towardsdatascience.com/increase-llama-2s-latency-and-throughput-performance-by-up-to-4x-23034d781b8c/) ## [Increase Llama 2's Latency and Throughput Performance by Up to 4X](https://towardsdatascience.com/increase-llama-2s-latency-and-throughput-performance-by-up-to-4x-23034d781b8c/) [Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/) Real-world benchmarks for Llama-2 13B [Het Trivedi](https://towardsdatascience.com/author/het.trivedi05/) August 8, 20237 min read * [![](../images/012.jpg)](https://towardsdatascience.com/step-by-step-approach-of-building-data-pipelines-as-a-data-scientist-or-a-machine-learning-engineer-4b3e8da3bcd1/) ## [Step-by-step Approach of Building Data Pipelines as a Data Scientist or a Machine Learning Engineer](https://towardsdatascience.com/step-by-step-approach-of-building-data-pipelines-as-a-data-scientist-or-a-machine-learning-engineer-4b3e8da3bcd1/) [Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/) Learn how to build interesting pipelines in the field of machine learning to develop a system that can perform AI capabilities [Suhas Maddali](https://towardsdatascience.com/author/suhas-maddali007/) November 2, 20228 min read * [![](../images/013.png)](https://towardsdatascience.com/what-is-universality-in-llm-and-how-to-find-universal-neurons/) ## [What is Universality in LLMs? How to Find Universal Neurons](https://towardsdatascience.com/what-is-universality-in-llm-and-how-to-find-universal-neurons/) [Large Language Models](https://towardsdatascience.com/category/artificial-intelligence/large-language-models/) How independently trained transformers form same the neurons [Shuyang Xiang](https://towardsdatascience.com/author/vanillaxiangshuyang/) September 2, 20255 min read * [![](../images/014.jpeg)](https://towardsdatascience.com/fit-vs-predict-vs-fit-predict-in-python-scikit-learn-f15a34a8d39f/) ## [fit() vs predict() vs fit_predict() in Python scikit-learn](https://towardsdatascience.com/fit-vs-predict-vs-fit-predict-in-python-scikit-learn-f15a34a8d39f/) [Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/) What's the difference between fit, predict and fit_predict methods in sklearn [Giorgos Myrianthous](https://towardsdatascience.com/author/gmyrianthous/) March 9, 20215 min read * [![](../images/015.jpeg)](https://towardsdatascience.com/simulated-data-real-learnings-power-analysis-652045eeae22/) ## [Simulated Data, Real Learnings : Power Analysis](https://towardsdatascience.com/simulated-data-real-learnings-power-analysis-652045eeae22/) [Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/) Part 2 - Experimental Power Analysis [Jarom Hulet](https://towardsdatascience.com/author/jarom.hulet/) March 26, 202411 min read * [![](../images/016.jpg)](https://towardsdatascience.com/how-to-develop-powerf-interal-llm-benchmarks/) ## [How to Develop Powerful Internal LLM Benchmarks](https://towardsdatascience.com/how-to-develop-powerf-interal-llm-benchmarks/) [Large Language Models](https://towardsdatascience.com/category/artificial-intelligence/large-language-models/) Learn how to compare LLMs using your own internal benchmark [Eivind Kjosbakken](https://towardsdatascience.com/author/oieivind/) August 26, 20256 min read * [![](../images/017.jpg)](https://towardsdatascience.com/how-to-use-causal-inference-when-a-b-testing-is-not-possible-c87c1252724a/) ## [How to use Causal Inference when A/B testing is not available](https://towardsdatascience.com/how-to-use-causal-inference-when-a-b-testing-is-not-possible-c87c1252724a/) [Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/) Evaluating ad targeting product using causal inference: propensity score matching! [Harry Lu](https://towardsdatascience.com/author/chinheng.h.lu/) January 8, 20247 min read * [YouTube](https://www.youtube.com/c/TowardsDataScience) * [X](https://x.com/TDataScience) * [LinkedIn](https://www.linkedin.com/company/towards-data-science/) * [Threads](https://www.threads.net/@towardsdatascience) * [Bluesky](https://bsky.app/profile/towardsdatascience.com) [](https://towardsdatascience.com/) Your home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals. Subscribe to Our Newsletter[Write For TDS](https://towardsdatascience.com/submissions/)•[About](https://towardsdatascience.com/about-towards-data-science-d691af11cc2f/)•[Advertise](https://contact.towardsdatascience.com/advertise-with-towards-data-science)•[Privacy Policy](https://towardsdatascience.com/privacy-policy/)•[Terms of Use](https://towardsdatascience.com/website-terms-of-use/) © Insight Media Group, LLC 2026•[The New Stack](https://thenewstack.io)•[roadmap.sh](https://roadmap.sh)•[Creator Network](https://insightmediagroup.io/creator-network)