{
  "source_file": "0930",
  "source_platform": "Substack",
  "author": "Sebastian Raschka / Ahead of AI",
  "title_listed": "Understanding Parameter-Efficient Finetuning of Large Language Models",
  "keywords": "PEFT,LoRA,微调",
  "domain": "大模型算法",
  "content_type": "深度技术解析",
  "is_paid_flag": "否",
  "url": "https://magazine.sebastianraschka.com/p/understanding-parameter-efficient",
  "publish_date": "2023-05",
  "corpus_stage": "预训练/SFT",
  "notes": "参数高效微调(PEFT)讲解",
  "id": "9b20e80b676f",
  "collected_at": "2026-10-01T20:32:50",
  "title": "Understanding Parameter-Efficient LLM Finetuning: Prompt Tuning And Prefix Tuning",
  "body_markdown": "Last week, I shared an overview of different Large Language Model (LLM) finetuning techniques. In a series of short articles, I am planning to discuss a selection of the most relevant techniques one by one.\nFinetuning Large Language Models\nSebastian Raschka\n·\nApril 22, 2023\nRead full story\nLet's start with a selection of\nparameter-efficient finetuning\ntechniques concerning prompt modifications.\nA selection of parameter-efficient finetuning techniques covered in this article.\nPrompt Tuning\nThe original concept of prompt tuning refers to techniques that vary the input prompt to achieve better modeling results. For example, suppose we are interested in translating an English sentence into German. We can ask the model in various different ways, as illustrated below.\nAn example of hard prompt tuning, that is, rearranging the input to get better outputs.\nNow, this concept illustrated above is referred to as\nhard\nprompt tuning since we directly change the discrete input tokens, which are not differentiable.",
  "metadata": {
    "title": "Understanding Parameter-Efficient LLM Finetuning: Prompt Tuning And Prefix Tuning",
    "description": "Explore prompt & prefix tuning: efficient methods for model finetuning! While soft prompt tuning adjusts input embeddings, prefix tuning modifies transformer blocks. Both techniques offer parameter savings over full finetuning.",
    "keywords": null,
    "author": "Sebastian Raschka, PhD",
    "og:type": "article",
    "og:title": "Understanding Parameter-Efficient LLM Finetuning: Prompt Tuning And Prefix Tuning",
    "og:description": "Last week, I shared an overview of different Large Language Model (LLM) finetuning techniques.",
    "og:image": "https://substackcdn.com/image/fetch/$s_!V7dG!,w_1200,h_675,c_fill,f_jpg,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0ee889b8-314f-4e7b-aaa1-c9928d0b62b2_1724x1322.png",
    "og:url": "https://magazine.sebastianraschka.com/p/understanding-parameter-efficient",
    "twitter:title": "Understanding Parameter-Efficient LLM Finetuning: Prompt Tuning And Prefix Tuning",
    "twitter:description": "Last week, I shared an overview of different Large Language Model (LLM) finetuning techniques.",
    "twitter:image": "https://substackcdn.com/image/fetch/$s_!TzF-!,f_auto,q_auto:best,fl_progressive:steep/https%3A%2F%2Fsebastianraschka.substack.com%2Fapi%2Fv1%2Fpost_preview%2F117433391%2Ftwitter.jpg%3Fversion%3D4",
    "twitter:card": "summary_large_image",
    "article:modified_time": "2025-03-07T20:54:32.153Z"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [],
  "images": [
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!lJ3t!,w_140,h_140,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa534808e-9284-47eb-9b4d-3e3aedc0b5da_1180x842.png",
      "bytes": 2893,
      "reused": true,
      "local_path": "attachments/9b20e80b676f/images/027.png",
      "src": "https://substackcdn.com/image/fetch/$s_!lJ3t!,w_140,h_140,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa534808e-9284-47eb-9b4d-3e3aedc0b5da_1180x842.png",
      "alt": "Finetuning Large Language Models",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!mNEc!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff3624f88-e572-4dd6-b06a-b5e5442aec27_1746x718.png",
      "bytes": 44180,
      "reused": true,
      "local_path": "attachments/9b20e80b676f/images/036.png",
      "src": "https://substackcdn.com/image/fetch/$s_!mNEc!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff3624f88-e572-4dd6-b06a-b5e5442aec27_1746x718.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!U1xT!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe1eeee00-21d3-40c6-ad51-a1489c62678a_1582x330.png",
      "bytes": 53318,
      "reused": false,
      "local_path": "attachments/9b20e80b676f/images/body-378555f8c0e3a374d353.png",
      "src": "https://substackcdn.com/image/fetch/$s_!U1xT!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe1eeee00-21d3-40c6-ad51-a1489c62678a_1582x330.png",
      "alt": "",
      "downloaded": true
    }
  ],
  "audio": [],
  "videos": [],
  "links": [
    {
      "href": "https://magazine.sebastianraschka.com/p/finetuning-large-language-models",
      "label": "Finetuning Large Language Models Sebastian Raschka · April 22, 2023 Read full story",
      "type": "a"
    },
    {
      "href": "https://substack.com/profile/27393275-sebastian-raschka",
      "label": "Sebastian Raschka",
      "type": "a"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:47:24Z",
  "source_url": "https://magazine.sebastianraschka.com/p/understanding-parameter-efficient",
  "papers": [],
  "actual_video_files": [],
  "body_chars": 1015,
  "expected_body_images": 3,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/9b20e80b676f/pages/source.html",
  "local_html": "attachments/9b20e80b676f/pages/article.html",
  "body_base_path": "attachments/9b20e80b676f/pages/",
  "body_markdown_path": "attachments/9b20e80b676f/pages/article.md"
}