{
  "source_file": "0930",
  "source_platform": "Ghost",
  "author": "Avi Chawla / Daily Dose of Data Science",
  "title_listed": "Speculative Decoding In LLMs",
  "keywords": "",
  "domain": "大模型算法",
  "content_type": "深度技术解析",
  "is_paid_flag": "否",
  "url": "https://www.dailydoseofds.com/p/speculative-decoding-in-llms/",
  "publish_date": "2026-06-04",
  "corpus_stage": "",
  "notes": "",
  "id": "3443152d7324",
  "collected_at": "2026-10-02T13:47:15",
  "title": "Speculative Decoding in LLMs",
  "body_markdown": "May 13, 2026\nLLMs\nSpeculative Decoding in LLMs\n...explained with code and tradeoffs.\nAvi Chawla\n👉\nGoogle uses speculative decoding in AI Overviews to serve over a billion Search users.\nIt’s also how Anthropic, Meta, and most major inference providers reduce latency at scale and get 2-3x more tokens per second, with mathematically identical outputs.\nWe covered the mechanics of LLM inference (prefill and decode phases, KV caching, speculative decoding, batching, and optimization techniques that improve latency and throughput here in the LLMOps course →\nThe idea comes from CPU architecture. A small draft model proposes several tokens ahead, and then a large model verifies them all in a single forward pass. Correct predictions (which are typically 60-80% of tokens) get accepted at the inference cost of the small model.\nThe diagram below depicts how speculative decoding differs from standard decoding:\nLet's dive in to understand the key problem it solves and how it works!\nThe problem\nA 70B parameter model in FP16 occupies roughly 140 GB of GPU memory.\nDuring the decode phase, EVERY SINGLE TOKEN requires a full forward pass through all 70B parameters (due to autoregressive generation), reading the entire model weights from memory.\nOn an H100 (3.35 TB/s memory bandwidth), it would take around 42 milliseconds just to move the weights and produce exactly one token.\nAnd here's what makes it feel particularly wasteful. The token the 70B model produces at each step is often a common, predictable word (\"the\", \"is\", \"of\") that even a 0.5B model could have gotten right too.\nNonetheless, you end up paying the full cost of reading 140 GB of weights to generate a token that a model 100x smaller could have produced in a fraction of the time.\nSpeculative decoding exploits this observation.\nHow speculative decoding works\nEssentially, you let the small model handle the core generation process.\nAfter a certain number of tokens (say, 5), use the large model to verify the tokens generated in a single forward pass.\nSince verification processes multiple tokens at once (this is structurally identical to the prefill phase), the GPU's compute units are actually saturated rather than sitting idle waiting on memory reads.\nHere's how the loop runs:\nFirst, a small model (e.g., Qwen2.5 0.5B) generates 5 candidate tokens autoregressively. Because it's 100x smaller than the target, this costs roughly 1-2% of a single large model forward pass.\nThen the large model (e.g., Qwen2.5 7B) processes all 5 draft tokens in a single forward pass, computing its probability distribution at each position. The draft model already produced its own distribution during drafting. Both distributions now exist side by side for comparison.\nThe verification step walks through draft tokens sequentially. If the large model agrees with tokens 1, 2, and 3, but disagrees at token 4, tokens 1-3 are accepted as-is (the small model got them right), the large model replaces token 4 with its own prediction (which it already computed during the verification pass), and token 5 is discarded without evaluation.\nBest case, all 5 draft tokens are accepted plus one bonus token from the large model, giving you 6 tokens from just a single forward pass through the large model.\nWorst case, every draft token is rejected, and you still get 1 token from the large model, which is the same as standard decoding.\nThe worst-case scenario is marginally more expensive than using the large model itself, but at the same time, it's also a rare incident.\nMoreover, the output is mathematically identical to running the large model alone.\nThis is because the large model doesn't just check whether the draft tokens look right.\nInstead, it computes its full probability distribution at every position and compares it against the small model's distribution.\nTokens are accepted only when the large model would have produced the same prediction. When it disagrees, it substitutes its own token. So every token in the final output is either directly approved or directly generated by the large model, guaranteeing an identical output distribution regardless of how good or bad the draft model is.\nImplementation\nHugging Face Transformers already exposes speculative decoding as\nassistant_model\nin the\ngenerate()\ncall.\nLet's look at how to use it:\n1) Load the models\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\nimport torch, time\n\ntarget_id = \"facebook/opt-6.7b\"\ndraft_id  = \"facebook/opt-125m\"\n\ntokenizer = AutoTokenizer.from_pretrained(target_id)\n\ntarget = AutoModelForCausalLM.from_pretrained(target_id, torch_dtype=torch.float16, device_map=\"auto\")\ndraft  = AutoModelForCausalLM.from_pretrained(draft_id,  torch_dtype=torch.float16, device_map=\"auto\")\nThe target model (6.7B) is the one whose output quality we want.\nThe draft model (125M) is the cheap, fast one that proposes candidate tokens. Both are from the OPT family and share the exact same tokenizer, so verification happens directly at the token ID level with zero overhead.\n2)\nSet up streaming and the input prompt\nfrom transformers import TextStreamer\nstreamer = TextStreamer(tokenizer, skip_special_tokens=True)\n\nprompt = \"Explain how a transformer model processes input tokens step by step.\"\ninputs = tokenizer(prompt, return_tensors=\"pt\").to(target.device)\nThe\nTextStreamer\nprints each token to the console as it's generated. With standard decoding, tokens appear one at a time. With speculative decoding, they arrive in visible bursts whenever a batch of draft tokens gets accepted.\n3) Baseline (standard decoding)\nt0 = time.perf_counter()\n\nbase_out = target.generate(**inputs, max_new_tokens=200, do_sample=False, streamer=streamer)\n\nbase_time = time.perf_counter() - t0\n\nprint(f\"\\n⏱ Without Speculative Decoding:\")\nprint(f\"  Runtime: {base_time:.2f}s\")\nprint(f\"  Token speed: {len(base_out[0]) / base_time:.1f} tok/s\")\nThis is the standard decode loop.\nEvery token requires a full forward pass through the 6.7B model. The GPU reads ~13 GB of weights from memory per token, producing one token at a time.\n4) Speculative decoding (6.7B + 125M)\nt0 = time.perf_counter()\n\nspec_out = target.generate(\n    **inputs, max_new_tokens=200, do_sample=False,\n    assistant_model=draft,\n    streamer=streamer,\n)\n\nspec_time = time.perf_counter() - t0\n\nprint(f\"\\n⏱ With Speculative Decoding:\")\nprint(f\"  Runtime: {spec_time:.2f}s\")\nprint(f\"  Token speed: {len(spec_out[0]) / spec_time:.1f} tok/s\")\nprint(f\"  Speedup over standard decoding: {base_time / spec_time:.2f}x\")\nIn this specific invocation, the only addition is assistant_model=draft.\nBehind the scenes, the 125M model drafts ~5 tokens autoregressively, the 6.7B model verifies all of them in a single forward pass, accepted tokens go straight to the output, and the first rejected token is replaced with the 6.7B model's own prediction. This repeats until 200 tokens are generated.\nThe video below shows this in action:\n0:00\n/\n0:09\n1×\nThe left side runs standard decoding (one token per forward pass), while the right side runs speculative decoding with the same 6.7B target model.\nThe speculative run produces tokens in visible bursts as batches of draft tokens get accepted, finishing in roughly half the time with a 1.9x speedup.\nFor production serving, vLLM supports draft models, EAGLE, n-gram, and MTP out of the box:\nfrom vllm import LLM, SamplingParams\n\nllm = LLM(\n    model=\"Qwen/Qwen2.5-72B-Instruct\",\n    speculative_model=\"Qwen/Qwen2.5-0.5B-Instruct\",\n    num_speculative_tokens=5,\n    tensor_parallel_size=4,\n)\n\noutputs = llm.generate([\"Explain speculative decoding.\"],\n                       SamplingParams(temperature=0, max_tokens=256))\nThe tokenizer constraint\nTraditionally, both models had to share the same tokenizer.\nThis is because the verification step discussed above compares the probability distributions of the large and small models at each token position.\nIf both models use the same tokenizer, token positions align perfectly, and the comparison is direct.\nBut if the tokenizers differ, the same word might get split into three tokens by one tokenizer and two tokens by another.\nThe probability distributions are now over different token spaces at different positions, and there's no meaningful way to compare them directly.\nThis limited speculative decoding to same-family pairs like Llama 3.2 1B with Llama 3.1 70B, Qwen2.5 0.5B with Qwen2.5 72B, etc.\nIn fact, at times, even models from the same family don't always share the same tokenizer.\nFor instance, Qwen2.5-7B-Instruct and Qwen2.5-0.5B-Instruct have the same vocabulary:\nBut they still differ in chat template configs, so HF Transformers framework detects this as a tokenizer mismatch:\nHugging Face shipped Universal Assisted Generation (UAG) in Transformers 4.46 (October 2024), removing that constraint entirely.\nUnder the hood, UAG decodes tokens generated by the small model back to text, re-encodes them with the large model's tokenizer, and aligns the sequences using longest common subsequence matching.\nThis lets you pair any two models regardless of tokenizer, i.e., a Qwen small model can have a Gemma large model, despite completely different tokenizers.\nThe tradeoff is that the same-tokenizer pairs still give better speedups (1.5-3x) compared to cross-tokenizer pairs (1.5-1.9x) because the native path skips the re-encoding overhead entirely.\nOne thing I noticed with speculative decoding is that even Instruct variants within the same family (e.g., Qwen2.5-7B-Instruct and Qwen2.5-0.5B-Instruct) can trigger the UAG path because Transformers compares tokenizer configs including chat templates, not just vocabulary. Using base models instead of Instruct variants avoids this and keeps you on the faster native path.\nProduction tradeoffs\nA larger draft model (the model that generates the tokens autoregressively) improves the acceptance rate. But at the same time, it also increases drafting latency.\nFor instance:\nWhen Llama 3.2 1B was used as the draft model for Llama 3.1 70B, it achieved 2.31x speedup\nBut when Llama 3.2 8B was used as the draft model, it achieved 2.08x speed up.\nSo despite its higher acceptance rate, the overall setup runs a bit slower.Sampling temperature also affects speedup.\nAt temperature ~0 (greedy decoding), both models tend to agree on the top token, so acceptance rates are highest.\nAs temperature increases, the probability distributions flatten, the small and large models agree less often, and more draft tokens get rejected. Creative generation tasks at high temperature see noticeably lower speedups than deterministic tasks.\nWe covered Temperature in LLMs here if you want to learn more →\nSpeculative decoding without a small model\nThe two-model setup has two pain points:\nYou need a matched small model from the same family (for better speedups)\nHaving a small model means extra GPU memory.\nSeveral variants eliminate one or both of these problems.\n1) EAGLE\nEAGLE replaces the separate draft model with a lightweight head (< 1B params) trained directly on top of the larger model.\nHere's how it works step by step.\nWhen the large model generates a token, every layer in the transformer produces a hidden state vector for that token.\nEAGLE takes the hidden state from the second-to-last layer (the one just before the final prediction layer) because it contains rich semantic information about what the model \"thinks\" should come next.\nThe EAGLE head is a small neural network that takes this hidden state as input and outputs a prediction for the next token, without running the full model again.\nThink of it as a shortcut. Instead of passing through all the layers of a large model to predict the next token, you pass through just the EAGLE head, which is a few layers at most.\nNow I want to depict how EAGLE works by depicting this particular idea that the first token is emitted from the large model, and then it uses the prediction head 2 or whatever.\nOnce it has drafted K tokens this way, the large model verifies all of them in a single forward pass, exactly like standard speculative decoding.\nThe key advantage is that these predictions are grounded in the large model's own internal representations, not from a separate, smaller model that was trained independently.\nThis is also why there's no tokenizer issue since the EAGLE head lives inside the target model's architecture and operates in the same token space.\n2) Medusa\nInstead of drafting tokens one after another, it predicts multiple future tokens simultaneously from a single forward pass.\nUnder the hood, Medusa attaches several small prediction heads to the large model, each responsible for a different position ahead.\nHead 1 predicts what the next token will be.\nHead 2 predicts what the token two positions ahead will be.\nHead 3 predicts three positions ahead, and so on.\nAll these heads run in parallel during a single forward pass of the large model. So from one pass, you get predictions for the next 5 positions at once, not just one.\nAs expected, one key problem is that each head makes its prediction independently. Head 2 doesn't know what Head 1 predicted.\nIn normal autoregressive generation, the prediction for token 3 depends on what token 2 actually was.\nMedusa can't do that because all heads run at the same time. So head 3 might predict \"model\" assuming token 2 was \"language,\" while head 2 actually predicted \"learning.\"\nThe candidates can be inconsistent with each other, which lowers acceptance rates compared to EAGLE, where each drafted token is conditioned on the previous one.\nThe upside is that there's no separate model and extra memory required for a drafter. The heads are also tiny (a few linear layers each), so the overhead per forward pass is minimal.\n3) Self-speculative decoding\nThis method requires no separate model, additional heads, or training at all.\nIt uses the large model itself as both the drafter and the verifier. The idea relies on how transformers are structured.\nA 70B model might have 80 layers, and each layer refines the prediction from the previous one.\nFor many tokens, the model's early layers (say, the first 12 out of 80) already have enough information to make the right prediction.\nThe remaining 68 layers just confirm what the early layers already figured out.\nSelf-speculative decoding exploits this observation.\nDuring drafting, the model runs only its first N layers and exits early with a prediction.\nIt does this K times to produce K draft tokens.\nThen, during verification, the full model (all 80 layers) runs a single forward pass over all K tokens to check which ones the early exit got right.\nFor easy, predictable tokens, the early layers are usually sufficient, and those tokens get accepted.\nFor harder tokens where the later layers would have changed the prediction, the draft gets rejected, and the full model's output is used instead.\nThe speedups are more modest (1.3-1.8x) because the drafter isn't a separate cheap model.\nIt's still the same large model, just running fewer layers. But the tradeoff is that you need absolutely nothing extra, like a second model to download, heads to train, or extra GPU memory.\n4) Diffusion-based drafting\nThis method addresses a fundamental inefficiency in all the methods above.\nEven with EAGLE, Medusa, or a separate draft model, the drafting process itself is still autoregressive. The drafter generates token 1, then token 2, then token 3, one at a time.\nThe draft model is faster because it's smaller, but it's still sequential.\nDiffusion models work completely differently.\nInstead of predicting one token at a time, a diffusion model starts with a block of random noise tokens and iteratively refines all of them in parallel.\nIn just a few refinement steps, the entire block converges to coherent text. So where an autoregressive drafter generates 8 tokens in 8 sequential steps, a diffusion drafter generates all 8 in a single forward pass (or a small fixed number of passes regardless of how many tokens you want).\nDFlash takes this idea and makes it practical for speculative decoding. Instead of training a standalone diffusion model from scratch (which would be large and expensive), it trains a small diffusion head that takes the target model's hidden states as conditioning input, similar to how EAGLE uses the target model's representations.\nThe diffusion head then generates an entire draft block in parallel, conditioned on what the large model has already computed.\nThis way, the drafting cost no longer scales with the number of tokens you want to propose.\nAn autoregressive drafter that proposes 16 tokens needs 16 sequential steps. A diffusion drafter proposes all 16 in one pass.\nThe speed up is evident from the video below:\n0:00\n/\n0:26\n1×\nThe verification step remains the same as standard speculative decoding. The large model checks all proposed tokens in a single forward pass and accepts or rejects them.\nThe only difference is how the draft tokens were generated, i.e., in parallel via diffusion rather than sequentially via autoregression.\nSpeculative decoding is converging toward single-model solutions where the draft capability is built into the target model itself.\nFor most production setups today, the two-model approach with a same-family drafter remains the simplest path to 2-3x speedups.\nBut regardless of which variant you use, all of them rely on KV caching internally to work efficiently.\nThe draft model maintains its own KV cache, so each draft token builds on the previous one cheaply.\nThe target model's verification pass extends its KV cache with accepted tokens, so the next iteration doesn't recompute anything for them.\nWithout KV caching, both drafting and verification would be far more expensive, and the speedups from speculative decoding would largely disappear.\nIn other words, speculative decoding reduces how many forward passes you need, and KV caching makes each of those forward passes efficient. They work together, not as alternatives.\nAs further reading:\nWe covered the mechanics of LLM inference (prefill and decode phases, KV caching, speculative decoding, batching, and optimization techniques that improve latency and throughput here in the LLMOps course →\nAnd we covered 72 techniques to optimize LLMs in production here →\nThanks for reading!\nPublished on May 13, 2026\nComments\nShare\nCopy link\nShare to X\nShare to Facebook\nShare to Linkedin\nCopied\nPrevious\nBellman Equations and Dynamic Programming\nNext\nHermes Agent Masterclass",
  "metadata": {
    "title": "Speculative Decoding in LLMs",
    "description": "...explained with code and tradeoffs.",
    "keywords": null,
    "author": null,
    "og:site_name": "Daily Dose of Data Science",
    "og:type": "article",
    "og:title": "Speculative Decoding in LLMs",
    "og:description": "...explained with code and tradeoffs.",
    "og:url": "https://www.dailydoseofds.com/p/speculative-decoding-in-llms/",
    "og:image": "https://storage.ghost.io/c/3f/df/3fdf6ed2-17ac-4b12-a693-8078bd13e748/content/images/2026/06/speculative-decoding.jpg",
    "og:image:width": "1166",
    "og:image:height": "1166",
    "twitter:card": "summary_large_image",
    "twitter:title": "Speculative Decoding in LLMs",
    "twitter:description": "...explained with code and tradeoffs.",
    "twitter:url": "https://www.dailydoseofds.com/p/speculative-decoding-in-llms/",
    "twitter:image": "https://storage.ghost.io/c/3f/df/3fdf6ed2-17ac-4b12-a693-8078bd13e748/content/images/2026/06/speculative-decoding.jpg",
    "twitter:label1": "Written by",
    "twitter:data1": "Avi Chawla",
    "twitter:label2": "Filed under",
    "twitter:data2": "LLMs",
    "twitter:site": "@_avichawla",
    "article:published_time": "2026-05-12T21:05:00.000Z",
    "article:modified_time": "2026-06-04T21:17:48.000Z",
    "article:tag": "LLMs"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [
    {
      "file": "attachments/3443152d7324/code/body-000.txt",
      "chars": 405,
      "language": "text"
    },
    {
      "file": "attachments/3443152d7324/code/body-001.txt",
      "chars": 245,
      "language": "text"
    },
    {
      "file": "attachments/3443152d7324/code/body-002.txt",
      "chars": 305,
      "language": "text"
    },
    {
      "file": "attachments/3443152d7324/code/body-003.txt",
      "chars": 413,
      "language": "text"
    },
    {
      "file": "attachments/3443152d7324/code/body-004.txt",
      "chars": 328,
      "language": "text"
    }
  ],
  "images": [
    {
      "status": "ok",
      "url": "https://storage.ghost.io/c/3f/df/3fdf6ed2-17ac-4b12-a693-8078bd13e748/content/images/size/w400/2026/03/avi-google.jpg",
      "bytes": 11344,
      "reused": true,
      "local_path": "attachments/3443152d7324/images/000.jpg",
      "src": "https://storage.ghost.io/c/3f/df/3fdf6ed2-17ac-4b12-a693-8078bd13e748/content/images/size/w400/2026/03/avi-google.jpg",
      "alt": "Avi Chawla",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!jvKB!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6515aeef-10b9-44dc-968f-04ca92e8e9c0_1057x463.png",
      "bytes": 56868,
      "reused": true,
      "local_path": "attachments/3443152d7324/images/001.png",
      "src": "https://substackcdn.com/image/fetch/$s_!jvKB!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6515aeef-10b9-44dc-968f-04ca92e8e9c0_1057x463.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!IBkn!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fce4e25ca-1b23-4118-a4c7-ecbf8087e961_1166x1166.gif",
      "bytes": 524657,
      "reused": true,
      "local_path": "attachments/3443152d7324/images/005.gif",
      "src": "https://substackcdn.com/image/fetch/$s_!IBkn!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fce4e25ca-1b23-4118-a4c7-ecbf8087e961_1166x1166.gif",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://storage.ghost.io/c/3f/df/3fdf6ed2-17ac-4b12-a693-8078bd13e748/content/images/2026/06/image-23.png",
      "bytes": 137961,
      "reused": true,
      "local_path": "attachments/3443152d7324/images/003.png",
      "src": "https://storage.ghost.io/c/3f/df/3fdf6ed2-17ac-4b12-a693-8078bd13e748/content/images/2026/06/image-23.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://storage.ghost.io/c/3f/df/3fdf6ed2-17ac-4b12-a693-8078bd13e748/content/images/2026/06/image-24.png",
      "bytes": 256604,
      "reused": true,
      "local_path": "attachments/3443152d7324/images/006.png",
      "src": "https://storage.ghost.io/c/3f/df/3fdf6ed2-17ac-4b12-a693-8078bd13e748/content/images/2026/06/image-24.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://storage.ghost.io/c/3f/df/3fdf6ed2-17ac-4b12-a693-8078bd13e748/content/images/2026/06/image-25.png",
      "bytes": 249177,
      "reused": true,
      "local_path": "attachments/3443152d7324/images/008.png",
      "src": "https://storage.ghost.io/c/3f/df/3fdf6ed2-17ac-4b12-a693-8078bd13e748/content/images/2026/06/image-25.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://storage.ghost.io/c/3f/df/3fdf6ed2-17ac-4b12-a693-8078bd13e748/content/images/2026/06/image-26.png",
      "bytes": 212392,
      "reused": true,
      "local_path": "attachments/3443152d7324/images/010.png",
      "src": "https://storage.ghost.io/c/3f/df/3fdf6ed2-17ac-4b12-a693-8078bd13e748/content/images/2026/06/image-26.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://storage.ghost.io/c/3f/df/3fdf6ed2-17ac-4b12-a693-8078bd13e748/content/images/2026/06/image-27.png",
      "bytes": 170747,
      "reused": true,
      "local_path": "attachments/3443152d7324/images/012.png",
      "src": "https://storage.ghost.io/c/3f/df/3fdf6ed2-17ac-4b12-a693-8078bd13e748/content/images/2026/06/image-27.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://storage.ghost.io/c/3f/df/3fdf6ed2-17ac-4b12-a693-8078bd13e748/content/images/2026/06/image-29.png",
      "bytes": 224617,
      "reused": true,
      "local_path": "attachments/3443152d7324/images/014.png",
      "src": "https://storage.ghost.io/c/3f/df/3fdf6ed2-17ac-4b12-a693-8078bd13e748/content/images/2026/06/image-29.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://storage.ghost.io/c/3f/df/3fdf6ed2-17ac-4b12-a693-8078bd13e748/content/images/2026/06/image-31.png",
      "bytes": 303524,
      "reused": true,
      "local_path": "attachments/3443152d7324/images/016.png",
      "src": "https://storage.ghost.io/c/3f/df/3fdf6ed2-17ac-4b12-a693-8078bd13e748/content/images/2026/06/image-31.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://storage.ghost.io/c/3f/df/3fdf6ed2-17ac-4b12-a693-8078bd13e748/content/images/2026/06/image-32.png",
      "bytes": 287536,
      "reused": true,
      "local_path": "attachments/3443152d7324/images/018.png",
      "src": "https://storage.ghost.io/c/3f/df/3fdf6ed2-17ac-4b12-a693-8078bd13e748/content/images/2026/06/image-32.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://storage.ghost.io/c/3f/df/3fdf6ed2-17ac-4b12-a693-8078bd13e748/content/images/2026/06/image-33.png",
      "bytes": 175388,
      "reused": true,
      "local_path": "attachments/3443152d7324/images/020.png",
      "src": "https://storage.ghost.io/c/3f/df/3fdf6ed2-17ac-4b12-a693-8078bd13e748/content/images/2026/06/image-33.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://storage.ghost.io/c/3f/df/3fdf6ed2-17ac-4b12-a693-8078bd13e748/content/images/2026/06/image-34.png",
      "bytes": 253259,
      "reused": true,
      "local_path": "attachments/3443152d7324/images/022.png",
      "src": "https://storage.ghost.io/c/3f/df/3fdf6ed2-17ac-4b12-a693-8078bd13e748/content/images/2026/06/image-34.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://storage.ghost.io/c/3f/df/3fdf6ed2-17ac-4b12-a693-8078bd13e748/content/images/2026/06/image-35.png",
      "bytes": 264090,
      "reused": true,
      "local_path": "attachments/3443152d7324/images/024.png",
      "src": "https://storage.ghost.io/c/3f/df/3fdf6ed2-17ac-4b12-a693-8078bd13e748/content/images/2026/06/image-35.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://storage.ghost.io/c/3f/df/3fdf6ed2-17ac-4b12-a693-8078bd13e748/content/images/2026/06/image-36.png",
      "bytes": 222917,
      "reused": true,
      "local_path": "attachments/3443152d7324/images/026.png",
      "src": "https://storage.ghost.io/c/3f/df/3fdf6ed2-17ac-4b12-a693-8078bd13e748/content/images/2026/06/image-36.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://storage.ghost.io/c/3f/df/3fdf6ed2-17ac-4b12-a693-8078bd13e748/content/images/2026/06/image-37.png",
      "bytes": 218441,
      "reused": true,
      "local_path": "attachments/3443152d7324/images/028.png",
      "src": "https://storage.ghost.io/c/3f/df/3fdf6ed2-17ac-4b12-a693-8078bd13e748/content/images/2026/06/image-37.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://storage.ghost.io/c/3f/df/3fdf6ed2-17ac-4b12-a693-8078bd13e748/content/images/2026/06/image-38.png",
      "bytes": 325388,
      "reused": true,
      "local_path": "attachments/3443152d7324/images/030.png",
      "src": "https://storage.ghost.io/c/3f/df/3fdf6ed2-17ac-4b12-a693-8078bd13e748/content/images/2026/06/image-38.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://storage.ghost.io/c/3f/df/3fdf6ed2-17ac-4b12-a693-8078bd13e748/content/images/2026/06/72-llm-optimization-techniques-visual.png",
      "bytes": 505380,
      "reused": true,
      "local_path": "attachments/3443152d7324/images/032.png",
      "src": "https://storage.ghost.io/c/3f/df/3fdf6ed2-17ac-4b12-a693-8078bd13e748/content/images/2026/06/72-llm-optimization-techniques-visual.png",
      "alt": "",
      "downloaded": true
    }
  ],
  "audio": [],
  "videos": [
    {
      "url": "https://storage.ghost.io/c/3f/df/3fdf6ed2-17ac-4b12-a693-8078bd13e748/content/media/2026/06/Speculative-decoding-video.mp4",
      "src": "https://storage.ghost.io/c/3f/df/3fdf6ed2-17ac-4b12-a693-8078bd13e748/content/media/2026/06/Speculative-decoding-video.mp4",
      "status": "invalid_or_wrong_resolution",
      "kind": "direct_video",
      "bytes": 1568597,
      "reused": true,
      "content_type": "video/mp4",
      "valid": true,
      "height": 650,
      "width": 1080,
      "duration": "9.916667",
      "local_path": "attachments/3443152d7324/videos/repair-61cff7263a5ad82537af.mp4",
      "title": "",
      "retried_at": "2026-10-20T17:55:56Z"
    },
    {
      "url": "https://storage.ghost.io/c/3f/df/3fdf6ed2-17ac-4b12-a693-8078bd13e748/content/media/2026/06/dflash-github-shoutout.mp4",
      "src": "https://storage.ghost.io/c/3f/df/3fdf6ed2-17ac-4b12-a693-8078bd13e748/content/media/2026/06/dflash-github-shoutout.mp4",
      "status": "ok",
      "kind": "direct_video",
      "bytes": 18082546,
      "reused": false,
      "content_type": "video/mp4",
      "valid": true,
      "height": 1080,
      "width": 1150,
      "duration": "26.533333",
      "local_path": "attachments/3443152d7324/videos/repair-6f21e3f663051f76dcdd.mp4"
    }
  ],
  "links": [
    {
      "href": "https://www.dailydoseofds.com/llmops-crash-course-part-13/",
      "label": "We covered the mechanics of LLM inference (prefill and decode phases, KV caching, speculative decoding, batching, and op",
      "type": "a"
    },
    {
      "href": "https://storage.ghost.io/c/3f/df/3fdf6ed2-17ac-4b12-a693-8078bd13e748/content/media/2026/06/Speculative-decoding-video.mp4",
      "label": "",
      "type": "video"
    },
    {
      "href": "https://x.com/_avichawla/article/2054860740541207032/media/2054842485630005248?ref=dailydoseofds.com",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://www.dailydoseofds.com/p/what-is-temperature-in-llms/",
      "label": "We covered Temperature in LLMs here if you want to learn more →",
      "type": "a"
    },
    {
      "href": "https://storage.ghost.io/c/3f/df/3fdf6ed2-17ac-4b12-a693-8078bd13e748/content/media/2026/06/dflash-github-shoutout.mp4",
      "label": "",
      "type": "video"
    },
    {
      "href": "https://www.dailydoseofds.com/p/72-techniques-to-optimize-llms-in-production/",
      "label": "And we covered 72 techniques to optimize LLMs in production here →",
      "type": "a"
    },
    {
      "href": "https://x.com/intent/tweet?url=https%3A%2F%2Fwww.dailydoseofds.com%2Fp%2Fspeculative-decoding-in-llms%2F&text=Speculative%20Decoding%20in%20LLMs",
      "label": "Share to X",
      "type": "a"
    },
    {
      "href": "https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Fwww.dailydoseofds.com%2Fp%2Fspeculative-decoding-in-llms%2F",
      "label": "Share to Facebook",
      "type": "a"
    },
    {
      "href": "https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fwww.dailydoseofds.com%2Fp%2Fspeculative-decoding-in-llms%2F",
      "label": "Share to Linkedin",
      "type": "a"
    },
    {
      "href": "https://www.dailydoseofds.com/rl-course-part-3/",
      "label": "Previous Bellman Equations and Dynamic Programming",
      "type": "a"
    },
    {
      "href": "https://www.dailydoseofds.com/p/hermes-agent-masterclass/",
      "label": "Next Hermes Agent Masterclass",
      "type": "a"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:47:58Z",
  "source_url": "https://www.dailydoseofds.com/p/speculative-decoding-in-llms/",
  "papers": [],
  "actual_video_files": [],
  "body_chars": 18431,
  "expected_body_images": 18,
  "discovery_status": "ok",
  "delivery_status": "partial",
  "missing_reason": "invalid_or_wrong_resolution",
  "raw_html": "attachments/3443152d7324/pages/source.html",
  "local_html": "attachments/3443152d7324/pages/article.html",
  "body_base_path": "attachments/3443152d7324/pages/",
  "body_markdown_path": "attachments/3443152d7324/pages/article.md"
}