{
  "source_file": "0930",
  "source_platform": "研究机构",
  "author": "Hugging Face Blog",
  "title_listed": "Supercharge your LLM with KV Cache",
  "keywords": "KV缓存,推理,加速",
  "domain": "大模型算法",
  "content_type": "深度技术解析",
  "is_paid_flag": "否",
  "url": "https://huggingface.co/blog/kv-cache",
  "publish_date": "2025-06",
  "corpus_stage": "推理部署",
  "notes": "KV缓存加速LLM推理",
  "id": "c1930feaaf3c",
  "collected_at": "2026-10-02T15:23:59",
  "title": "KV Cache from scratch in nanoVLM",
  "body_markdown": "Back to Articles\nKV Cache from scratch in nanoVLM\nPublished\n\t\t\t\t\tJune 4, 2025\nUpdate on GitHub\nUpvote\n121\n+115\nAritra Roy Gosthipaty\nariG23498\nFollow\nKashif Rasul\nkashif\nFollow\nLuis\nlusxvr\nFollow\nAndres Marafioti\nandito\nFollow\nPedro Cuenca\npcuenq\nFollow\nTL;DR\nWe have implemented KV Caching from scratch in our\nnanoVLM\nrepository (a small codebase to train your own Vision Language Model with pure PyTorch). This gave us a\n38%\nspeedup in generation. In this blog post we cover KV Caching and all our experiences while implementing it. The lessons learnt are general and can be applied to all autoregressive language model generations. Implementing from scratch on a small codebase is a great learning experience, come along for the ride!\nIntroduction\nAutoregressive language models generate text by sampling\none token at a time\n. During inference, the model processes a given input sequence, predicts the next token, appends it to the sequence, and repeats this process until some stopping criterion:\nThis step-by-step generation is inherently sequential:\nTo generate token\nt\ni\n+\n1\nt_{i+1}\nt\ni\n+\n1\n​\n, the model must consider the entire sequence from\nt\n0\nt_0\nt\n0\n​\nto\nt\ni\nt_i\nt\ni\n​\n. From the first instance in the above example\nt\ni\n+\n1\nt_{i+1}\nt\ni\n+\n1\n​\nwould be\nthe\n, while all the previous tokens\nt\n0\nt_0\nt\n0\n​\nto\nt\ni\nt_i\nt\ni\n​\nwould be\n[What, is, in]\n.\nAlthough transformers are internally parallel, each new prediction requires a full forward pass through all transformer layers, which incurs a quadratic memory/compute in terms of the sequence length.\nThis repetition also leads to computational\nredundancy\n. In this post, we explore\nKV Caching\n, an optimisation technique that mitigates this inefficiency.\nTable of contents:\nRevisiting the Transformer Architecture\nWhere Redundancy Creeps In\nHow KV Caching Fixes It\nKV Caching in nanoVLM: From Theory to Practice\nSummary: Why KV Caching Matters\nRevisiting the Transformer Architecture\nBefore diving into caching, let’s revisit how attention operates in transformer models. A Transformer language model consists of stacked layers, each composed of:\nMulti-head self-attention\nFeed-forward network (MLP)\nResidual connections and layer normalisation\nTo understand\nwhere KV Caching helps\n, we focus on the\nself-attention\nmechanism, specifically within a single attention head.\nLet’s walk through a simple PyTorch implementation to visualise the key computations.\nimport\ntorch\n\ninput_seq_length =\n5\ndim_model =\n10\ninput_ids_emb = torch.randn(input_seq_length, dim_model)\nW_q = torch.randn(dim_model, dim_model)\nW_k = torch.randn(dim_model, dim_model)\nW_v = torch.randn(dim_model, dim_model)\n\nQ = input_ids_emb @ W_q\nK = input_ids_emb @ W_k\nV = input_ids_emb @ W_v\nSelf-Attention Computation\nFor a sequence of\nT\nT\nT\ninput embeddings represented as\nX\n∈\nR\nT\n×\nD\nX \\in \\mathbb{R}^{T \\times D}\nX\n∈\nR\nT\n×\nD\n, self-attention is computed as:\nQ\n=\nX\nW\nQ\nQ = XW_Q\nQ\n=\nX\nW\nQ\n​\n, with\nW\nQ\n∈\nR\nD\n×\nD\nq\nW_Q \\in \\mathbb{R}^{D \\times D_q}\nW\nQ\n​\n∈\nR\nD\n×\nD\nq\n​\nK\n=\nX\nW\nK\nK = XW_K\nK\n=\nX\nW\nK\n​\n, with\nW\nK\n∈\nR\nD\n×\nD\nk\nW_K \\in \\mathbb{R}^{D \\times D_k}\nW\nK\n​\n∈\nR\nD\n×\nD\nk\n​\nV\n=\nX\nW\nV\nV = XW_V\nV\n=\nX\nW\nV\n​\n, with\nW\nV\n∈\nR\nD\n×\nD\nv\nW_V \\in \\mathbb{R}^{D \\times D_v}\nW\nV\n​\n∈\nR\nD\n×\nD\nv\n​\nCausal mask\nM\nM\nM\nto prevent future token access\nThe final output is:\nAttention\n(\nX\n;\nQ\n,\nK\n,\nV\n)\n=\nsoftmax\n(\nQ\nK\n⊤\n⋅\nM\nd\nk\n)\nV\n\\text{Attention}(X; Q, K, V) = \\text{softmax}\\left( \\frac{QK^\\top \\cdot M}{\\sqrt{d_k}} \\right)V\nAttention\n(\nX\n;\nQ\n,\nK\n,\nV\n)\n=\nsoftmax\n(\nd\nk\n​\n​\nQ\nK\n⊤\n⋅\nM\n​\n)\nV\nHere’s a minimal PyTorch equivalent using a causal mask:\nimport\ntorch.nn.functional\nas\nF\nimport\nmath\n\nd_k = K.shape[-\n1\n]\nattention_scores = (Q @ K.T) / math.sqrt(d_k)\n# Lower triangular mask to prevent future token access\ncausal_mask = torch.tril(torch.ones(input_seq_length, input_seq_length))\nmasked_scores = attention_scores.masked_fill(causal_mask ==\n0\n,\nfloat\n(\n'-inf'\n))\n\nattention_weights = F.softmax(masked_scores, dim=-\n1\n)\noutput = attention_weights @ V\nWhere Redundancy Creeps In\nIn autoregressive generation, the model generates one token at a time. With each step, it recomputes\nQ\nQ\nQ\n,\nK\nK\nK\n, and\nV\nV\nV\nfor\nthe entire sequence\n, even though the earlier tokens haven’t changed.\nnew_token_emb = torch.randn(\n1\n, dim_model)\nextended_input = torch.cat([input_ids_emb, new_token_emb], dim=\n0\n)\n\nQ_ext = extended_input @ W_q\nK_ext = extended_input @ W_k\nV_ext = extended_input @ W_v\n# (output_ext would be computed using Q_ext, K_ext, V_ext + masking)\nTo confirm the redundancy:\ntorch.testing.assert_close(K, K_ext[:input_seq_length])\n# test pass\ntorch.testing.assert_close(V, V_ext[:input_seq_length])\n# test pass\nThese checks show that for all but the newest token,\nK\nK\nK\nand\nV\nV\nV\nare identical to previously computed values.\nOriginal (5×5):         Extended (6×6):\n■ ■ ■ ■ ■              ■ ■ ■ ■ ■ □\n■ ■ ■ ■ ■              ■ ■ ■ ■ ■ □\n■ ■ ■ ■ ■    →         ■ ■ ■ ■ ■ □\n■ ■ ■ ■ ■              ■ ■ ■ ■ ■ □\n■ ■ ■ ■ ■              ■ ■ ■ ■ ■ □\n                       □ □ □ □ □ □\n■\n= Already computed and reused\n□\n= Recomputed unnecessarily\nMost of the attention computation is repeated needlessly. This gets more expensive as sequences grow.\nHow KV Caching Fixes It\nTo eliminate this inefficiency, we use\nKV Caching\n:\nAfter processing the initial prompt, we\ncache\nthe computed keys (\nK\nK\nK\n) and values (\nV\nV\nV\n) for each layer.\nDuring generation, we\nonly compute\nK\nK\nK\nand\nV\nV\nV\nfor the new token\n, and\nappend\nthem to the cache.\nWe compute\nQ\nQ\nQ\nfor the current token and use it with the\ncached\nK\nK\nK\nand\nV\nV\nV\nto get the output.\nThis changes generation from full-sequence re-computation to a lightweight, incremental update.\n✅ In practice, this cache is a per-layer dictionary with keys \"key\" and \"value\", each of shape (\nbatch_size\n,\nnum_heads\n,\nseq_len_cached\n,\nhead_dim\n).\nThis is the foundation of how modern LLMs can generate long outputs efficiently.\nKV Caching in nanoVLM: From Theory to Practice\nNow that we understand the theory behind KV Caching, let’s see how it’s implemented in practice inside our\nnanoVLM\nrepository. This is an ideal testbed, as it's a super concise and self-contained codebase.\nKV caching is enabled across three key components in our model:\nThe\nAttention block\nthat uses and updates the KV cache\nThe\nLanguage model\nthat tracks cache per layer\nThe\nGeneration loop\nthat separates\nprefill\n(the initial pass with the input prompt) and sequential\ndecode\nphases\n1. Updating KV Cache in the Attention Block\nIn the\nLanguageModelGroupedAttention\nclass, we modify the\nforward\nfunction to accept and update a cache of keys and values (\nblock_kv_cache\n).\nPreviously, the model recomputed\nK\nK\nK\nand\nV\nV\nV\nat every generation step. Now we only compute\nK\nnew\nK_{\\text{new}}\nK\nnew\n​\n,\nV\nnew\nV_{\\text{new}}\nV\nnew\n​\nfor the current token, and append them to the cached values.\ndef\nforward\n(\nself, x, cos, sin, attention_mask=\nNone\n, block_kv_cache=\nNone\n):\n    is_prefill = block_kv_cache\nis\nNone\nB, T_curr, C = x.size()\n# Project inputs to Q, K, V\nq_curr, k_curr, v_curr = project_current_tokens(x)\n    q, k_rotated = apply_rotary_pos_embd(q_curr, k_curr, cos, sin)\nif\nnot\nis_prefill\nand\nblock_kv_cache[\n'key'\n]\nis\nnot\nNone\n:\n# Append new keys and values to the cache\nk = torch.cat([block_kv_cache[\n'key'\n], k_rotated], dim=\n2\n)\n        v = torch.cat([block_kv_cache[\n'value'\n], v_curr], dim=\n2\n)\nelse\n:\n# First pass (prefill) — no cache\nk, v = k_rotated, v_curr\n\n    block_kv_cache = {\n'key'\n: k,\n'value'\n: v}\nreturn\nattention_output, block_kv_cache\n2. Tracking Cache Across Layers\nIn the\nLanguageModel\nclass, we introduce\nlayer-wise cache tracking\n. The\nstart_pos\nargument helps the model compute correct\nrotary positional encodings\nfor newly generated tokens.\ndef\nforward\n(\nself, x, kv_cache=\nNone\n, start_pos=\n0\n):\n    T_curr = x.size(\n1\n)\n    position_ids = torch.arange(start_pos, start_pos + T_curr, device=x.device)\n    cos, sin = self.rotary_embd(position_ids)\nfor\ni, block\nin\nenumerate\n(self.blocks):\n# Pass per-layer KV cache\nx, kv_cache[i] = block(x, cos, sin, attention_mask, kv_cache[i])\nreturn\nx, kv_cache\nkv_cache\n: A list of dictionaries, one per transformer layer, holding previous keys and values.\nstart_pos\n: Ensures that rotary embeddings are aligned with current generation index.\n3. Prefill vs Decode in the Generation Loop\nThe biggest architectural change is in the\ngenerate()\nmethod of the\nVisionLanguageModel\n.\nWe\nsplit generation into two stages\n:\nPREFILL PHASE:\nEncode the full prompt and build the initial cache.\nDECODE PHASE:\nGenerate tokens one at a time using cached keys/values.\nPREFILL PHASE (cache construction)\n[Prompt: \"What is\"] → [Transformer] → [Cache: K, V for all layers]\n\nDECODE PHASE (token-by-token)\n[Token: \"the\"] → [Q(\"the\") + cached K/V] → [next token: \"?\"] → ...\nHere’s the corresponding code:\n# PREFILL: Process the input prompt, fill the cache\nprompt_output, kv_cache_list = self.forward(\n    inputs,\n    kv_cache=\nNone\n,\n    start_pos=\n0\n)\n# DECODE: Generate one token at a time using cached K/V\nfor\ni\nin\nrange\n(max_new_tokens):\n    next_token = sample_from(prompt_output)\n\n    decode_output, kv_cache_list = self.forward(\n        next_token,\n        kv_cache=kv_cache_list,\n        start_pos=current_position\n# updated with each step\n)\n\n    prompt_output = decode_output\nBy separating these phases, we avoid redundant computation and dramatically speed up inference, especially for long prompts.\nSummary of Changes\nModule\nOriginal Behaviour\nNew Behaviour\nLanguageModelGroupedAttention.forward\nRecomputes\nQ\nQ\nQ\n,\nK\nK\nK\n,\nV\nV\nV\non every step\nUses and updates KV cache\nLanguageModel.forward\nNo memory of previous state\nTracks per-layer KV cache, handles\nstart_pos\nVisionLanguageModel.generate\nOne-phase generation loop\nSplit into\nprefill\nand\ndecode\nphases\nSummary: Why KV Caching Matters\nBenefit\nExplanation\nIncremental growth\nCache grows by one row per new token\nPosition-aware decoding\nstart_pos\nensures correctness of position encoding calculations\nEfficiency\nReduces per-token inference to O(\nseq len\n) instead of quadratic\nKV caching eliminates unnecessary computation during autoregressive generation, enabling faster and more efficient inference, especially in long sequences and real-time applications. This is a trade-off between speed and memory, and its drawbacks can be more complex code and restricting fancier inference schemes, like beam-search, etc. KV caching is a popular method for speeding up LLM inference, making it possible to run them on consumer hardware, and now you know how it works too!\nMore Articles from our Blog\nnlp\nguide\ncommunity\nFine-tuning a 350M Model for Better Structured Outputs in 100 GRPO Steps\n149\nSeptember 3, 2026\nnlp\nguide\ncommunity\nTraining and Finetuning Multi-Vector Embedding Models with Sentence Transformers\n145\nAugust 26, 2026\nCommunity\nsamuellimabraz\nJun 4, 2025\nThanks  for this great article! I'm learning a lot from the nanoVLM project.\nI'm not an expert in gen ai but I noticed the attention calculation example seems to be missing the scaling √(d_k). Is this intentional for simplification?\nd_k = K.shape[-\n1\n]\nattention_scores = (Q @ K.T) / math.sqrt(d_k)\nFrom my understanding this scaling prevents the dot product growing too large and control the softmax region\nSee translation\n1 reply\n·\n➕\n3\n3\n🔥\n2\n2\n❤️\n2\n2\n+\nariG23498\nArticle author\nJun 4, 2025\nThis is such a good catch!\nWould you like to raise a PR to the blog post with the changes to the code?\nHere is the source for the blog post:\nhttps://github.com/huggingface/blog/blob/main/kv-cache.md\nSee translation\n👍\n2\n2\n🤗\n2\n2\n❤️\n1\n1\n+\nritwikraha\nJun 4, 2025\nNice read, I found the prefill and decode explanation very intuitive. Good job 👏\nSee translation\n😎\n2\n2\n+\nReply\nnot-lain\nJun 5, 2025\nmade this visual representation of what happens inside the attention mechanism when using kv caching.\nthought i'd share it with the community 🤗\nSee translation\n1 reply\n·\n🔥\n8\n8\n+\nariG23498\nArticle author\nJun 6, 2025\nVERY COOL! Thanks for sharing.\nSee translation\n😎\n1\n1\n+\nAmarnathReddy0201\nAug 22, 2025\nHi Team,\nkv_cache only for text? Is it doing for images in this Articles.\nSee translation\n1 reply\n·\nnot-lain\nAug 22, 2025\n•\nedited Aug 23, 2025\ncaching results can be used for diffusion applications as well, although not similar to KV caching since kv caching is optimized for next token prediction while in diffusion the full sequence already exists.\nthe short answer to your question\nis yes kv caching is only for next token prediction but there are other caching techniques out there for other tasks\ntry checking what PrunaAI has been doing to optimize diffusion models, here's a link to one of their previous presentations :\nLINKEDIN_POST\nSee translation\nEdit\nPreview\nUpload images, audio, and videos by dragging in the text input, pasting, or\nclicking here\n.\nTap or paste here to upload images\nComment\n·\nSign up\nor\nlog in\nto comment\nUpvote\n121\n+109",
  "metadata": {
    "title": "KV Cache from scratch in nanoVLM",
    "description": "We’re on a journey to advance and democratize artificial intelligence through open source and open science.",
    "keywords": null,
    "author": null,
    "og:title": "KV Cache from scratch in nanoVLM",
    "og:description": "We’re on a journey to advance and democratize artificial intelligence through open source and open science.",
    "og:type": "website",
    "og:url": "https://huggingface.co/blog/kv-cache",
    "og:image": "https://huggingface.co/blog/assets/kv-cache/thumbnail.png",
    "twitter:card": "summary_large_image",
    "twitter:site": "@huggingface",
    "twitter:image": "https://huggingface.co/blog/assets/kv-cache/thumbnail.png"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [
    {
      "file": "attachments/c1930feaaf3c/code/body-000.txt",
      "chars": 301,
      "language": "text"
    },
    {
      "file": "attachments/c1930feaaf3c/code/body-001.txt",
      "chars": 401,
      "language": "text"
    },
    {
      "file": "attachments/c1930feaaf3c/code/body-002.txt",
      "chars": 266,
      "language": "text"
    },
    {
      "file": "attachments/c1930feaaf3c/code/body-003.txt",
      "chars": 136,
      "language": "text"
    },
    {
      "file": "attachments/c1930feaaf3c/code/body-004.txt",
      "chars": 250,
      "language": "text"
    },
    {
      "file": "attachments/c1930feaaf3c/code/body-005.txt",
      "chars": 712,
      "language": "text"
    },
    {
      "file": "attachments/c1930feaaf3c/code/body-006.txt",
      "chars": 375,
      "language": "text"
    },
    {
      "file": "attachments/c1930feaaf3c/code/body-007.txt",
      "chars": 200,
      "language": "text"
    },
    {
      "file": "attachments/c1930feaaf3c/code/body-008.txt",
      "chars": 483,
      "language": "text"
    },
    {
      "file": "attachments/c1930feaaf3c/code/body-009.txt",
      "chars": 64,
      "language": "text"
    }
  ],
  "images": [
    {
      "status": "ok",
      "url": "https://cdn-avatars.huggingface.co/v1/production/uploads/608aabf24955d2bfc3cd99c6/-YxmtpzEmf3NKOTktODRP.jpeg",
      "bytes": 9712,
      "reused": true,
      "local_path": "attachments/c1930feaaf3c/images/000.jpeg",
      "src": "https://cdn-avatars.huggingface.co/v1/production/uploads/608aabf24955d2bfc3cd99c6/-YxmtpzEmf3NKOTktODRP.jpeg",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://cdn-avatars.huggingface.co/v1/production/uploads/6630afa2d40e9b46f95b4c15/H1xNxJztbOFi8IZQu7XjV.jpeg",
      "bytes": 5590,
      "reused": true,
      "local_path": "attachments/c1930feaaf3c/images/002.jpeg",
      "src": "https://cdn-avatars.huggingface.co/v1/production/uploads/6630afa2d40e9b46f95b4c15/H1xNxJztbOFi8IZQu7XjV.jpeg",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://cdn-avatars.huggingface.co/v1/production/uploads/65d66b494bbd0d92b641cdbb/6-7dm7B-JxcoS1QlCPdMN.jpeg",
      "bytes": 12320,
      "reused": true,
      "local_path": "attachments/c1930feaaf3c/images/003.jpeg",
      "src": "https://cdn-avatars.huggingface.co/v1/production/uploads/65d66b494bbd0d92b641cdbb/6-7dm7B-JxcoS1QlCPdMN.jpeg",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://cdn-avatars.huggingface.co/v1/production/uploads/666b9ef5e6c60b6fc4156675/av6g3oEeQQ4Rpa_IBFfaT.jpeg",
      "bytes": 16388,
      "reused": false,
      "local_path": "attachments/c1930feaaf3c/images/body-76968de34da236d52240.jpeg",
      "src": "https://cdn-avatars.huggingface.co/v1/production/uploads/666b9ef5e6c60b6fc4156675/av6g3oEeQQ4Rpa_IBFfaT.jpeg",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://cdn-avatars.huggingface.co/v1/production/uploads/6640bbd0220cfa8cbfdce080/wiAHUu5ewawyipNs0YFBR.png",
      "bytes": 4508,
      "reused": false,
      "local_path": "attachments/c1930feaaf3c/images/body-f215cef2141effa79fc7.png",
      "src": "https://cdn-avatars.huggingface.co/v1/production/uploads/6640bbd0220cfa8cbfdce080/wiAHUu5ewawyipNs0YFBR.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://cdn-avatars.huggingface.co/v1/production/uploads/6422eab8e2029ade06eeee2c/Gai3BHr2WJ0YuhdumqQ_z.png",
      "bytes": 12688,
      "reused": false,
      "local_path": "attachments/c1930feaaf3c/images/body-c95688a65d5caf45a799.png",
      "src": "https://cdn-avatars.huggingface.co/v1/production/uploads/6422eab8e2029ade06eeee2c/Gai3BHr2WJ0YuhdumqQ_z.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/kv-cache/speed_improved.png",
      "bytes": 176853,
      "reused": true,
      "local_path": "attachments/c1930feaaf3c/images/005.png",
      "src": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/kv-cache/speed_improved.png",
      "alt": "bar plot showcasing improvement in generation speed",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/kv-cache/autoregression.png",
      "bytes": 70938,
      "reused": true,
      "local_path": "attachments/c1930feaaf3c/images/006.png",
      "src": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/kv-cache/autoregression.png",
      "alt": "diagram for autoregression",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://huggingface.co/blog/assets/grpo-with-trl-ifstruct/thumbnail.png",
      "bytes": 241101,
      "reused": false,
      "local_path": "attachments/c1930feaaf3c/images/body-e5c56040668ba6c9b737.png",
      "src": "https://huggingface.co/blog/assets/grpo-with-trl-ifstruct/thumbnail.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://huggingface.co/avatars/33af1a607a8bf7780b98d6f70ea19999.svg",
      "bytes": 397,
      "reused": false,
      "local_path": "attachments/c1930feaaf3c/images/body-dabb94fe94eb9c9ae71f.svg",
      "src": "https://huggingface.co/avatars/33af1a607a8bf7780b98d6f70ea19999.svg",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://cdn-avatars.huggingface.co/v1/production/uploads/62d648291fa3e4e7ae3fa6e8/oatOwf8Xqe5eDbCSuYqCd.png",
      "bytes": 4308,
      "reused": false,
      "local_path": "attachments/c1930feaaf3c/images/body-2f7cd9aaae2da43f9642.png",
      "src": "https://cdn-avatars.huggingface.co/v1/production/uploads/62d648291fa3e4e7ae3fa6e8/oatOwf8Xqe5eDbCSuYqCd.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://cdn-avatars.huggingface.co/v1/production/uploads/61929226ded356549e20c5da/ONUjP2S5fUWd07BiFXm0i.jpeg",
      "bytes": 3442,
      "reused": false,
      "local_path": "attachments/c1930feaaf3c/images/body-280c63c0d63ec39783a0.jpeg",
      "src": "https://cdn-avatars.huggingface.co/v1/production/uploads/61929226ded356549e20c5da/ONUjP2S5fUWd07BiFXm0i.jpeg",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://huggingface.co/blog/assets/train-sentence-transformers/st-hf-thumbnail.png",
      "bytes": 129194,
      "reused": false,
      "local_path": "attachments/c1930feaaf3c/images/body-3f59e95de0cdb130c49c.png",
      "src": "https://huggingface.co/blog/assets/train-sentence-transformers/st-hf-thumbnail.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://cdn-avatars.huggingface.co/v1/production/uploads/6317233cc92fd6fee317e030/cJHSvvimr1kqgQfHOjO5n.png",
      "bytes": 5138,
      "reused": false,
      "local_path": "attachments/c1930feaaf3c/images/body-4846b3d76d1e92e2d185.png",
      "src": "https://cdn-avatars.huggingface.co/v1/production/uploads/6317233cc92fd6fee317e030/cJHSvvimr1kqgQfHOjO5n.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://cdn-avatars.huggingface.co/v1/production/uploads/1636283009799-noauth.jpeg",
      "bytes": 3408,
      "reused": false,
      "local_path": "attachments/c1930feaaf3c/images/body-0e19363b1455b19d368e.jpeg",
      "src": "https://cdn-avatars.huggingface.co/v1/production/uploads/1636283009799-noauth.jpeg",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://cdn-avatars.huggingface.co/v1/production/uploads/6527e89a8808d80ccff88b7a/CuGNmF1Et8KMQ0mCd1NEJ.jpeg",
      "bytes": 7400,
      "reused": false,
      "local_path": "attachments/c1930feaaf3c/images/body-0ecd71cc794086a67918.jpeg",
      "src": "https://cdn-avatars.huggingface.co/v1/production/uploads/6527e89a8808d80ccff88b7a/CuGNmF1Et8KMQ0mCd1NEJ.jpeg",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://huggingface.co/avatars/34eec3c5d1ecd4ef7b629bb06ffe4559.svg",
      "bytes": 1281,
      "reused": false,
      "local_path": "attachments/c1930feaaf3c/images/body-7b3893ad5519a27173c3.svg",
      "src": "https://huggingface.co/avatars/34eec3c5d1ecd4ef7b629bb06ffe4559.svg",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://cdn-uploads.huggingface.co/production/uploads/6527e89a8808d80ccff88b7a/KJ2zbZd_I2w8c9psrEbJm.png",
      "bytes": 1275697,
      "reused": true,
      "local_path": "attachments/c1930feaaf3c/images/007.png",
      "src": "https://cdn-uploads.huggingface.co/production/uploads/6527e89a8808d80ccff88b7a/KJ2zbZd_I2w8c9psrEbJm.png",
      "alt": "image.png",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://cdn-uploads.huggingface.co/production/uploads/6527e89a8808d80ccff88b7a/u3DQovEcCxNJLjBdnHMdE.png",
      "bytes": 1715017,
      "reused": true,
      "local_path": "attachments/c1930feaaf3c/images/008.png",
      "src": "https://cdn-uploads.huggingface.co/production/uploads/6527e89a8808d80ccff88b7a/u3DQovEcCxNJLjBdnHMdE.png",
      "alt": "image.png",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://cdn-avatars.huggingface.co/v1/production/uploads/1654090481550-60f2e021adf471cbdf8bb660.jpeg",
      "bytes": 5778,
      "reused": false,
      "local_path": "attachments/c1930feaaf3c/images/body-780add96fe550a8eca74.jpeg",
      "src": "https://cdn-avatars.huggingface.co/v1/production/uploads/1654090481550-60f2e021adf471cbdf8bb660.jpeg",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://cdn-avatars.huggingface.co/v1/production/uploads/1669189789447-629f3b18ee05727ce328ccbe.jpeg",
      "bytes": 5898,
      "reused": true,
      "local_path": "attachments/c1930feaaf3c/images/001.jpeg",
      "src": "https://cdn-avatars.huggingface.co/v1/production/uploads/1669189789447-629f3b18ee05727ce328ccbe.jpeg",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://cdn-avatars.huggingface.co/v1/production/uploads/638a1f3258afc73b9c64a761/A6lqaOhd6Rt4bdIkdYOB6.png",
      "bytes": 5510,
      "reused": false,
      "local_path": "attachments/c1930feaaf3c/images/body-afa8c9344b18cd2b0c42.png",
      "src": "https://cdn-avatars.huggingface.co/v1/production/uploads/638a1f3258afc73b9c64a761/A6lqaOhd6Rt4bdIkdYOB6.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://cdn-avatars.huggingface.co/v1/production/uploads/64a52413fa840dd18c2f76fe/5PD-BHnKm2aQWFZhE7P2I.jpeg",
      "bytes": 17650,
      "reused": false,
      "local_path": "attachments/c1930feaaf3c/images/body-cb9f5ac3c9611248c60b.jpeg",
      "src": "https://cdn-avatars.huggingface.co/v1/production/uploads/64a52413fa840dd18c2f76fe/5PD-BHnKm2aQWFZhE7P2I.jpeg",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://cdn-avatars.huggingface.co/v1/production/uploads/62627ed702cd5952e013c60d/MkCYKQmBhcpLh7oZbic_1.jpeg",
      "bytes": 8702,
      "reused": false,
      "local_path": "attachments/c1930feaaf3c/images/body-25e1b0fbb091767967da.jpeg",
      "src": "https://cdn-avatars.huggingface.co/v1/production/uploads/62627ed702cd5952e013c60d/MkCYKQmBhcpLh7oZbic_1.jpeg",
      "alt": "",
      "downloaded": true
    }
  ],
  "audio": [],
  "videos": [
    {
      "url": "https://cdn-uploads.huggingface.co/production/uploads/6527e89a8808d80ccff88b7a/DP2zDJTAU-yHrxVRh5GUt.mp4",
      "src": "https://cdn-uploads.huggingface.co/production/uploads/6527e89a8808d80ccff88b7a/DP2zDJTAU-yHrxVRh5GUt.mp4",
      "status": "ok",
      "kind": "direct_video",
      "bytes": 661899,
      "reused": false,
      "content_type": "video/mp4",
      "valid": true,
      "height": 1080,
      "width": 1920,
      "duration": "15.000000",
      "local_path": "attachments/c1930feaaf3c/videos/repair-48ecd461466844c4ca16.mp4"
    }
  ],
  "links": [
    {
      "href": "https://huggingface.co/blog",
      "label": "Back to Articles",
      "type": "a"
    },
    {
      "href": "https://github.com/huggingface/blog/blob/main/kv-cache.md",
      "label": "Update on GitHub",
      "type": "a"
    },
    {
      "href": "https://huggingface.co/login?next=%2Fblog%2Fkv-cache",
      "label": "Upvote 121",
      "type": "a"
    },
    {
      "href": "https://huggingface.co/ariG23498",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://huggingface.co/lusxvr",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://huggingface.co/andito",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://huggingface.co/samuellimabraz",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://huggingface.co/John6666",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://huggingface.co/MElHuseyni",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://huggingface.co/kashif",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://huggingface.co/pcuenq",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://github.com/huggingface/nanoVLM",
      "label": "nanoVLM",
      "type": "a"
    },
    {
      "href": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/kv-cache/speed_improved.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/kv-cache/autoregression.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://huggingface.co/blog/grpo-with-trl-ifstruct",
      "label": "nlp guide community Fine-tuning a 350M Model for Better Structured Outputs in 100 GRPO Steps 149 September 3, 2026",
      "type": "a"
    },
    {
      "href": "https://huggingface.co/blog/train-multi-vector-encoder",
      "label": "nlp guide community Training and Finetuning Multi-Vector Embedding Models with Sentence Transformers 145 August 26, 2026",
      "type": "a"
    },
    {
      "href": "https://huggingface.co/ritwikraha",
      "label": "ritwikraha",
      "type": "a"
    },
    {
      "href": "https://huggingface.co/not-lain",
      "label": "not-lain",
      "type": "a"
    },
    {
      "href": "https://cdn-uploads.huggingface.co/production/uploads/6527e89a8808d80ccff88b7a/DP2zDJTAU-yHrxVRh5GUt.mp4",
      "label": "",
      "type": "video"
    },
    {
      "href": "https://huggingface.co/AmarnathReddy0201",
      "label": "AmarnathReddy0201",
      "type": "a"
    },
    {
      "href": "https://www.linkedin.com/posts/johko_pruna-ai-increasing-the-efficiency-of-image-activity-7343971567862710272-OGcP?utm_source=share&utm_medium=member_desktop&rcm=ACoAADHJ074BHxxPkGGXTCvene-FdU4B_MJqkMo",
      "label": "LINKEDIN_POST",
      "type": "a"
    },
    {
      "href": "https://cdn-uploads.huggingface.co/production/uploads/6527e89a8808d80ccff88b7a/KJ2zbZd_I2w8c9psrEbJm.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://cdn-uploads.huggingface.co/production/uploads/6527e89a8808d80ccff88b7a/u3DQovEcCxNJLjBdnHMdE.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://huggingface.co/join?next=%2Fblog%2Fkv-cache",
      "label": "Sign up",
      "type": "a"
    },
    {
      "href": "https://huggingface.co/manu",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://huggingface.co/shaikatasif",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://huggingface.co/byrayhana",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://huggingface.co/Siarhei",
      "label": "",
      "type": "a"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:53:27Z",
  "source_url": "https://huggingface.co/blog/kv-cache",
  "papers": [],
  "actual_video_files": [],
  "body_chars": 12829,
  "expected_body_images": 24,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/c1930feaaf3c/pages/source.html",
  "local_html": "attachments/c1930feaaf3c/pages/article.html",
  "body_base_path": "attachments/c1930feaaf3c/pages/",
  "body_markdown_path": "attachments/c1930feaaf3c/pages/article.md"
}