{
  "source_file": "0930",
  "source_platform": "研究机构",
  "author": "Anthropic Research",
  "title_listed": "Prompt Engineering for Claude's Long Context Window",
  "keywords": "Prompt,长上下文,Claude",
  "domain": "大模型算法",
  "content_type": "教程",
  "is_paid_flag": "否",
  "url": "https://www.anthropic.com/news/prompting-long-context",
  "publish_date": "2024-01",
  "corpus_stage": "预训练/SFT",
  "notes": "Claude长上下文提示工程",
  "id": "28088f56b7f5",
  "collected_at": "2026-10-02T15:38:01",
  "title": "Prompting Claude's long context window \\ Anthropic",
  "body_markdown": "Product\nPrompt engineering for Claude's long context window\nSep 23, 2023\nClaude’s\n100,000 token long context window\nenables the model to operate over hundreds of pages of technical documentation, or even an\nentire book\n. As we continue to scale the Claude API, we’re seeing increased demand for prompting guidance on how to maximize Claude’s potential. Today, we’re pleased to share a quantitative case study on two techniques that can improve Claude’s recall over long contexts:\nExtracting reference quotes relevant to the question before answering\nSupplementing the prompt with examples of correctly answered questions about other sections of the document\nLet’s get into the details.\nTesting long context recall: Multiple choice Q&A\nOur goal with this experiment is to evaluate techniques to maximize Claude’s chance of correctly recalling a specific piece of information from a long document.\nAs the underlying data source for our testing, we used a daily-issue, publicly available\ngovernment document\nthat contains the meeting transcripts and activities of many different departments. We chose a document from July 13, which is after Claude’s training data cutoff. This minimizes the chance that Claude already knows the information in the document.\nIn order to generate a dataset of question and answer (Q&A) pairs, we used an approach we call “randomized collage”. We split the document into sections and used Claude to generate five multiple choice questions for each section, each with three wrong answers and one right answer. We then reassembled randomized sets of these sections into long documents so that we could pass them to Claude and test its recall of their contents.\nPrompting Claude to generate multiple choice questions\nGetting Claude to write your evaluations (evals) for you is a powerful tool and one that we as a prompt engineering team use often. However, it takes careful prompting to get Claude to write questions in the sweet spot of difficulty. Here are some challenges we worked through along the way to designing an effective test suite:\nClaude may propose questions that it can answer off the top of its head without reference to any document, like “What does the Department of Transportation do?”\nClaude may include questions that it struggles to answer correctly, even in short context situations. For example, because Claude sees the world in terms of tokens, not words, a question like “How many words are in the passage?” tends to give Claude trouble.\nClaude may leave unintentional clues in its answers that make the correct answer easy to guess. Notably, we noticed its default was to make the right answer very detailed in comparison to the wrong answers.\nClaude may reference “this document” or “this passage” without specifying what passage it means. This is problematic because when we stitch together multiple documents to form the long context, Claude has no way to know which document the question is asking about.\nThere’s also the risk of going too far and making the questions so explicit that they contain the answer. For example, “What is the publication date of the Department of the Interior’s July 2, 2023 notice about additional in-season actions for fisheries?” would be an ineffective question.\nTo help navigate these pitfalls, we used a prompt template that includes two sample meeting chunks along with hand-written questions to act as few-shot question writing examples, as well as some writing guidelines to get Claude to specify details about the passage. The template and all other prompts used in this experiment are available\nhere\n.\nEvaluation\nFor this evaluation, we primarily focused on our smaller Claude Instant model (version 1.2) as opposed to our Claude 2 model. This is for a couple of reasons. First, Claude 2 is already very good at recalling information after reading very long documents. Claude Instant needs more help, which makes it easy to see when changes to prompting improve performance. Additionally, Claude Instant is so fast that you can reproduce these evals yourself with the notebook we share.\nWhen provided with only the exact passage Claude used to write the question, Claude Instant was able to answer its own generated question about 90% of the time. We discarded the 10% of questions it answered incorrectly – since Claude was getting them wrong even in a short-context situation, they’re too difficult to be useful for testing long context.\n1\nWe also tested Claude’s recall when given a random section not containing the question’s source material. Theoretically, with three wrong answers, Claude should be able to guess the right answer only 25% of the time. In practice, it guessed right 34% of the time; above chance, but not too much so. Claude is likely sometimes intuiting the right answer based on its general knowledge, or based on subtle cues in the answer.\nIn order to build long documents from shorter chunks, we artificially generated a new long context for each question by stitching together a random assortment of passages until we reached the desired number of tokens. We tested Claude on stitched-together documents containing roughly 75,000 and 90,000 tokens.\nThis patchwork context formation is why Claude referencing ambiguous phrases like “this document” or “this passage” in its question was causing problems. A question like “What is the publication date of this notice?” ceases to be answerable when there are a dozen different notices in the context to which “this notice” could refer. This is why our question generation prompt includes language telling Claude to be explicit about the passage to which the question refers – for example, “What is the publication date of the notice about additional in-season actions for fisheries?\"\nAfter generating the long documents (collages), we tested Claude’s recall using four different prompting strategies:\nBase\n– just ask Claude to answer\nNongov examples\n- Give Claude two fixed examples of correctly answered general knowledge multiple choice questions that are unrelated to the government document\n2\nTwo examples\n- Give Claude two examples of correctly answered multiple choice questions, dynamically chosen at random from the set of Claude-generated questions about other chunks in the context\nFive examples\n- Same strategy as #3, but with five examples\nFor each of the four strategies above, we also tested them with and without the use of a <scratchpad>, in which we instruct Claude to pull relevant quotes. Furthermore, we tested each of these strategies with the passage containing the answer positioned either at the beginning, the end, or in the middle of the input. Finally, to get a sense of the effect of context length on the results, we tested with both 70K and 95K token documents.\nWe used Claude Instant 1.2 for the above test. We also show results for Claude 2 on the baseline strategy and the strategy that performed best for Claude Instant 1.2.\nResults\nSome notes on the experiment:\nWhile performance on the beginning and middle of the doc are substantially improved by the use of a scratchpad and examples, performance on the end can be degraded. This could be because the addition of the examples in the prompt increases the distance between the very end of the document (where the relevant information is) and when Claude needs to answer it. This is likely of only minor concern as only a small fraction of the data in the source document is at the very end. However, it does emphasize the importance of putting the instructions\nat the end of the prompt\n, as we want Claude’s recall of them to be as high as possible.\nClaude 2’s improvement from 0.939 to 0.961 with prompting might seem small in absolute terms, but it reflects a 36% reduction in errors.\nSome takeaways you can use for writing your long-context Q&A prompts:\nUse many examples and the scratchpad for best performance on both context lengths.\nPulling relevant quotes into the scratchpad is helpful in all head-to-head comparisons. It comes at a small cost to latency, but improves accuracy. In Claude Instant’s case, the latency is already so low that this shouldn’t be a concern.\nContextual examples help on both 70K and 95K, and more examples is better.\nGeneric examples on general/external knowledge do not seem to help performance.\nFor Claude Instant, there seems to be a monotonic inverse relationship between performance and the distance of the relevant passage to the question and the end of the prompt, while Claude 2 performance on 95K sees a small dip in the middle.\n3\nIntroducing the new Anthropic Cookbook\nFully reproducible code for this experiment is live in the new\nAnthropic Cookbook\n. This growing set of resources also contains two other recipes at the moment:\nA\nSearch and Retrieval demo\nshowcasing a tool use flow for searching Wikipedia.\nGuidance on implementing\nmock-PDF uploading functionality\nvia the Anthropic API.\nWe’re looking forward to expanding the Anthropic Cookbook and our other prompt engineering resources in the future, and we hope they inspire you to dream big about what you can build with Claude. If you haven’t received access to the Claude API yet, please\nregister your interest\n.\nFootnotes\n1\nA common theme in the questions Claude gets wrong is counting, e.g. “What is the estimated number of responses per respondent that the notice states for each form in the Common Forms Package for Guaranteed Loan Forms?” and “How many proposed finished products are listed in the notification for production activity at the Getinge Group Logistics Americas LLC facility?” Notably, on some of these questions, Claude’s pre-specified “correct” answer (from when it generated the QA pairs) is not in reality correct. This is a source of noise in this experiment.\n2\nThe questions are: 1. Who was the first president of the United States? A. Thomas Jefferson, B. George Washington, C. Abraham Lincoln, D. John Adams, 2. What is the boiling temperature of water, in degrees Fahrenheit? A. 200, B. 100, C. 287, D. 212.\n3\nA\nrecent paper\nfound a U-shaped relationship between performance and location in the context for a similar task. A possible explanation for the differing results is that the examples in the paper have avg. length 15K tokens (Appendix F), compared to 70K/95K here.\nRelated content\nAnthropic opens Singapore office and signs MOU with the Government of Singapore\nAnthropic has opened an office in Singapore and signed an MOU with the country’s government to advance safe, responsible AI adoption.\nRead more\n2026 Usage Policy update\nWe’re publishing a new version of our Usage Policy. In this post, we summarize the changes we’ve made.\nRead more\nBuilding on our commitment to American scientific discovery\nAnthropic is committing $150 million over three years to the Genesis Mission, a federal initiative to accelerate scientific and technological discovery through AI.\nRead more",
  "metadata": {
    "title": "Prompting Claude's long context window \\ Anthropic",
    "description": "Two techniques improve Claude's long-document Q&A: extracting relevant quotes first and providing in-context examples.",
    "keywords": null,
    "author": null,
    "og:title": "Prompting Claude's long context window",
    "og:description": "Two techniques improve Claude's long-document Q&A: extracting relevant quotes first and providing in-context examples.",
    "og:image": "https://www-cdn.anthropic.com/images/4zrzovbb/website/6d4a0d28992ade92d6fa63646fd9c9d318245c6c-2400x1260.jpg",
    "og:image:alt": "Anthropic logo",
    "og:type": "article",
    "twitter:card": "summary_large_image",
    "twitter:site": "@AnthropicAI",
    "twitter:creator": "@AnthropicAI",
    "twitter:title": "Prompting Claude's long context window",
    "twitter:description": "Two techniques improve Claude's long-document Q&A: extracting relevant quotes first and providing in-context examples.",
    "twitter:image": "https://www-cdn.anthropic.com/images/4zrzovbb/website/6d4a0d28992ade92d6fa63646fd9c9d318245c6c-2400x1260.jpg",
    "twitter:image:alt": "Anthropic logo",
    "article:published_time": "2023-09-23T16:00:00.000Z",
    "article:modified_time": "2026-09-09T20:09:35.000Z"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [],
  "images": [
    {
      "status": "ok",
      "url": "https://www.anthropic.com/_next/image?url=https%3A%2F%2Fwww-cdn.anthropic.com%2Fimages%2F4zrzovbb%2Fwebsite%2F112b3d86f1ee7ddbf62ad26f2c311dd6dbd53177-2880x1620.png&w=3840&q=75",
      "bytes": 71634,
      "reused": false,
      "local_path": "attachments/28088f56b7f5/images/body-fea4d88468dca3ec1495.img",
      "src": "https://www.anthropic.com/_next/image?url=https%3A%2F%2Fwww-cdn.anthropic.com%2Fimages%2F4zrzovbb%2Fwebsite%2F112b3d86f1ee7ddbf62ad26f2c311dd6dbd53177-2880x1620.png&w=3840&q=75",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://www.anthropic.com/_next/image?url=https%3A%2F%2Fwww-cdn.anthropic.com%2Fimages%2F4zrzovbb%2Fwebsite%2Fd8f6e82f34d694395cb24fee8265d5b4ac28759f-3758x2406.png&w=3840&q=75",
      "bytes": 226753,
      "reused": false,
      "local_path": "attachments/28088f56b7f5/images/body-78d8424ffbd46c44433a.img",
      "src": "https://www.anthropic.com/_next/image?url=https%3A%2F%2Fwww-cdn.anthropic.com%2Fimages%2F4zrzovbb%2Fwebsite%2Fd8f6e82f34d694395cb24fee8265d5b4ac28759f-3758x2406.png&w=3840&q=75",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://www.anthropic.com/_next/image?url=https%3A%2F%2Fwww-cdn.anthropic.com%2Fimages%2F4zrzovbb%2Fwebsite%2F6a76b74a871eff3c91334df78f6f1b57d189e329-3758x2406.png&w=3840&q=75",
      "bytes": 160584,
      "reused": false,
      "local_path": "attachments/28088f56b7f5/images/body-c47ca39f496e70de6a2e.img",
      "src": "https://www.anthropic.com/_next/image?url=https%3A%2F%2Fwww-cdn.anthropic.com%2Fimages%2F4zrzovbb%2Fwebsite%2F6a76b74a871eff3c91334df78f6f1b57d189e329-3758x2406.png&w=3840&q=75",
      "alt": "",
      "downloaded": true
    }
  ],
  "audio": [],
  "videos": [],
  "links": [
    {
      "href": "https://www.anthropic.com/news/100k-context-windows",
      "label": "100,000 token long context window",
      "type": "a"
    },
    {
      "href": "https://twitter.com/AnthropicAI/status/1656700156518060033",
      "label": "entire book",
      "type": "a"
    },
    {
      "href": "https://www.govinfo.gov/content/pkg/FR-2023-07-13/xml/FR-2023-07-13.xml",
      "label": "government document",
      "type": "a"
    },
    {
      "href": "https://www-cdn.anthropic.com/d1198e53f52c64b6f647fa49fc3399c7eafb2b78/Prompt_Eng_pdf.pdf",
      "label": "here",
      "type": "a"
    },
    {
      "href": "https://docs.anthropic.com/claude/docs/advanced-text-analysis",
      "label": "at the end of the prompt",
      "type": "a"
    },
    {
      "href": "https://platform.claude.com/cookbook",
      "label": "Anthropic Cookbook",
      "type": "a"
    },
    {
      "href": "https://platform.claude.com/cookbook/third-party-wikipedia-wikipedia-search-cookbook",
      "label": "Search and Retrieval demo",
      "type": "a"
    },
    {
      "href": "https://platform.claude.com/cookbook/misc-pdf-upload-summarization",
      "label": "mock-PDF uploading functionality",
      "type": "a"
    },
    {
      "href": "https://www.anthropic.com/api",
      "label": "register your interest",
      "type": "a"
    },
    {
      "href": "https://arxiv.org/abs/2307.03172",
      "label": "recent paper",
      "type": "a"
    },
    {
      "href": "https://twitter.com/intent/tweet?text=https://www.anthropic.com/news/prompting-long-context",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://www.linkedin.com/shareArticle?mini=true&url=https://www.anthropic.com/news/prompting-long-context",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://www.anthropic.com/news/singapore-office-opening",
      "label": "Read more",
      "type": "a"
    },
    {
      "href": "https://www.anthropic.com/news/2026-usage-policy-update",
      "label": "Read more",
      "type": "a"
    },
    {
      "href": "https://www.anthropic.com/news/genesis-mission-commitment",
      "label": "Read more",
      "type": "a"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:53:38Z",
  "source_url": "https://www.anthropic.com/news/prompting-long-context",
  "papers": [
    {
      "status": "ok",
      "url": "https://www-cdn.anthropic.com/d1198e53f52c64b6f647fa49fc3399c7eafb2b78/Prompt_Eng_pdf.pdf",
      "bytes": 70315,
      "reused": false,
      "local_path": "papers/repair/b472ecaa2d698045f46a.pdf"
    },
    {
      "status": "ok",
      "url": "https://arxiv.org/pdf/2307.03172",
      "bytes": 747542,
      "reused": true,
      "local_path": "papers/repair/9a55f69abe89ff9892a1.pdf"
    }
  ],
  "actual_video_files": [],
  "body_chars": 10873,
  "expected_body_images": 3,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/28088f56b7f5/pages/source.html",
  "local_html": "attachments/28088f56b7f5/pages/article.html",
  "body_base_path": "attachments/28088f56b7f5/pages/",
  "body_markdown_path": "attachments/28088f56b7f5/pages/article.md"
}