{
  "source_file": "0930",
  "source_platform": "个人博客",
  "author": "Philipp Schmid",
  "title_listed": "FSDP + QLoRA Llama 3",
  "keywords": "FSDP,QLoRA,Llama 3",
  "domain": "大模型算法",
  "content_type": "教程",
  "is_paid_flag": "否",
  "url": "https://www.philschmid.de/fsdp-qlora-llama3",
  "publish_date": "2024-04",
  "corpus_stage": "预训练/SFT",
  "notes": "用FSDP+QLoRA微调Llama 3",
  "id": "2068a07ff1d4",
  "collected_at": "2026-10-02T14:29:13",
  "title": "Efficiently fine-tune Llama 3 with PyTorch FSDP and Q-Lora",
  "body_markdown": "Efficiently fine-tune Llama 3 with PyTorch FSDP and Q-Lora\nApril 22, 2024\n11\nminute read\nView Code\nOpen LLMs like Meta\nLlama 3\n, Mistral AI\nMistral\n&\nMixtral\nmodels or AI21\nJamba\nare now OpenAI competitors. However, most of the time you need to fine-tune the model on your data to unlock the full potential of the model. Fine-tuning smaller LLMs, like Mistral became very accessible on a single GPU by using Q-Lora. But efficiently fine-tuning bigger models like Llama 3 70b or Mixtral stayed a challenge until now.\nThis blog post walks you thorugh how to fine-tune a Llama 3 using PyTorch FSDP and Q-Lora with the help of Hugging Face\nTRL\n,\nTransformers\n,\npeft\n&\ndatasets\n. In addition to FSDP we will use\nFlash Attention v2 through the Pytorch SDPA\nimplementation.\nSetup development environment\nCreate and prepare the dataset\nFine-tune the LLM with PyTorch FSDP, Q-Lora and SDPA\nTest Model and run Inference\nNote: This blog was created and validated on NVIDIA H100 and NVIDIA A10G GPUs. The configurations and code is optimized for 4xA10G GPUs each with 24GB of Memory. I hope this keeps the example as accessible as possible for most people. If you have access to more compute you can make changes to the config (yaml) at step 3.\nFSDP + Q-Lora Background\nIn a collaboration between\nAnswer.AI\n, Tim Dettmers\nQ-Lora creator\nand\nHugging Face\n, we are proud to announce to share the support of Q-Lora and PyTorch FSDP (Fully Sharded Data Parallel). FSDP and Q-Lora allows you now to fine-tune Llama 2 70b or Mixtral 8x7B on 2x consumer GPUs (24GB). If you want to learn more about the background of this collaboration take a look at\nYou can now train a 70b language model at home\n. Hugging Face PEFT is were the magic happens for this happens, read more about it in the\nPEFT documentation\n.\nPyTorch FSDP\nis a data/model parallelism technique that shards model across GPUs, reducing memory requirements and enabling the training of larger models more efficiently​​​​​​.\nQ-LoRA is a fine-tuning method that leverages quantization and Low-Rank Adapters to efficiently reduced computational requirements and memory footprint.\n1. Setup development environment\nOur first step is to install Hugging Face Libraries and Pyroch, including trl, transformers and datasets. If you haven't heard of trl yet, don't worry. It is a new library on top of transformers and datasets, which makes it easier to fine-tune, rlhf, align open LLMs.\nPython\n# Install Pytorch for FSDP and FA/SDPA\n%pip install\n\"torch==2.2.2\"\ntensorboard\n# Install Hugging Face libraries\n%pip install\n--\nupgrade\n\"transformers==4.40.0\"\n\"datasets==2.18.0\"\n\"accelerate==0.29.3\"\n\"evaluate==0.4.1\"\n\"bitsandbytes==0.43.1\"\n\"huggingface_hub==0.22.2\"\n\"trl==0.8.6\"\n\"peft==0.10.0\"\nNext we need to login into Hugging Face to access the Llama 3 70b model. If you don't have an account yet and accepted the terms, you can create one\nhere\n.\nPython\n!\nhuggingface-cli login\n--\ntoken\n\"\"\n2. Create and prepare the dataset\nAfter our environment is set up, we can start creating and preparing our dataset. A fine-tuning dataset should have a diverse set of demonstrations of the task you want to solve. If you want to learn more about how to create a dataset, take a look at the\nHow to Fine-Tune LLMs in 2024 with Hugging Face\n.\nWe will use the\nHuggingFaceH4/no_robots\ndataset a high-quality dataset of 10,000 instructions and demonstrations created by skilled human annotators. This data can be used for supervised fine-tuning (SFT) to make language models follow instructions better. No Robots was modelled after the instruction dataset described in OpenAI's\nInstructGPT paper\n, and is comprised mostly of single-turn instructions.\nJSON\n{\n\"messages\"\n: [{\n\"role\"\n:\n\"system\"\n,\n\"content\"\n:\n\"You are...\"\n}, {\n\"role\"\n:\n\"user\"\n,\n\"content\"\n:\n\"...\"\n}, {\n\"role\"\n:\n\"assistant\"\n,\n\"content\"\n:\n\"...\"\n}]}\n{\n\"messages\"\n: [{\n\"role\"\n:\n\"system\"\n,\n\"content\"\n:\n\"You are...\"\n}, {\n\"role\"\n:\n\"user\"\n,\n\"content\"\n:\n\"...\"\n}, {\n\"role\"\n:\n\"assistant\"\n,\n\"content\"\n:\n\"...\"\n}]}\n{\n\"messages\"\n: [{\n\"role\"\n:\n\"system\"\n,\n\"content\"\n:\n\"You are...\"\n}, {\n\"role\"\n:\n\"user\"\n,\n\"content\"\n:\n\"...\"\n}, {\n\"role\"\n:\n\"assistant\"\n,\n\"content\"\n:\n\"...\"\n}]}\nThe\nno_robots\ndataset has 10,000 split into 9,500 training and 500 test examples. Some samples are not including a\nsystem\nmessage. We will load the dataset with the\ndatasets\nlibrary, add a missing\nsystem\nmessage and save them to separate json files.\nPython\nfrom\ndatasets\nimport\nload_dataset\n# Convert dataset to OAI messages\nsystem_message =\n\"\"\"You are Llama, an AI assistant created by Philipp to be helpful and honest. Your knowledge spans a wide range of topics, allowing you to engage in substantive conversations and provide analysis on complex subjects.\"\"\"\ndef\ncreate_conversation\n(\nsample\n):\nif\nsample[\n\"messages\"\n][\n0\n][\n\"role\"\n] ==\n\"system\"\n:\nreturn\nsample\nelse\n:\nsample[\n\"messages\"\n] = [{\n\"role\"\n:\n\"system\"\n,\n\"content\"\n: system_message}] + sample[\n\"messages\"\n]\nreturn\nsample\n# Load dataset from the hub\ndataset =\nload_dataset\n(\n\"HuggingFaceH4/no_robots\"\n)\n# Add system message to each conversation\ncolumns_to_remove = list(dataset[\n\"train\"\n].features)\ncolumns_to_remove.\nremove\n(\n\"messages\"\n)\ndataset = dataset.\nmap\n(create_conversation,\nremove_columns\n=columns_to_remove,\nbatched\n=\nFalse\n)\n# Filter out conversations which are corrupted with wrong turns, keep which have even number of turns after adding system message\ndataset[\n\"train\"\n] = dataset[\n\"train\"\n].\nfilter\n(\nlambda\nx\n: len(x[\n\"messages\"\n][\n1\n:]) %\n2\n==\n0\n)\ndataset[\n\"test\"\n] = dataset[\n\"test\"\n].\nfilter\n(\nlambda\nx\n: len(x[\n\"messages\"\n][\n1\n:]) %\n2\n==\n0\n)\n# save datasets to disk\ndataset[\n\"train\"\n].\nto_json\n(\n\"train_dataset.json\"\n,\norient\n=\n\"records\"\n,\nforce_ascii\n=\nFalse\n)\ndataset[\n\"test\"\n].\nto_json\n(\n\"test_dataset.json\"\n,\norient\n=\n\"records\"\n,\nforce_ascii\n=\nFalse\n)\n3. Fine-tune the LLM with PyTorch FSDP, Q-Lora and SDPA\nWe are now ready to fine-tune our model with PyTorch FSDP, Q-Lora and SDPA. Since we are running in a distributed setup, we need to use\ntorchrun\nand a python script to start the training.\nWe prepared a script\nrun_fsdp_qlora.py\nwhich will load the dataset from disk, prepare the model, tokenizer and start the training. It usees the\nSFTTrainer\nfrom\ntrl\nto fine-tune our model. The\nSFTTrainer\nmakes it straightfoward to supervise fine-tune open LLMs supporting:\nDataset formatting, including conversational and instruction format (✅ used)\nTraining on completions only, ignoring prompts (❌ not used)\nPacking datasets for more efficient training (✅ used)\nPEFT (parameter-efficient fine-tuning) support including Q-LoRA (✅ used)\nPreparing the model and tokenizer for conversational fine-tuning (❌ not used, see below)\nNote: We are using an Anthropic/Vicuna like Chat Template with\nUser:\nand\nAssistant:\nroles. This done because the special tokens in base Llama 3 (\n<|begin_of_text|>\nor\n<|reserved_special_token_XX|>\n) are not trained. Meaning if want would like to use them for the template we need to train them which requires more memory, since we need to update the embedding layer and lm_head. If you have access to more compute you can modify\nLLAMA_3_CHAT_TEMPLATE\nin the\nrun_fsdp_qlora.py\nscript.\nFor configuration we use the new\nTrlParser\n, that allows us to provide hyperparameters in a yaml file or overwrite the arguments from the config file by explicitly passing them to the CLI, e.g.\n--num_epochs 10\n. Below is the config file for fine-tuning Llama 3 70B on 4x A10G GPUs or 4x24GB GPUs.\nPython\n%%writefile llama_3_70b_fsdp_qlora.yaml\n# script parameters\nmodel_id:\n\"meta-llama/Meta-Llama-3-70b\"\n# Hugging Face model id\ndataset_path:\n\".\"\n# path to dataset\nmax_seq_len:\n3072\n# 2048              # max sequence length for model and packing of the dataset\n# training parameters\noutput_dir:\n\"./llama-3-70b-hf-no-robot\"\n# Temporary output directory for model checkpoints\nreport_to:\n\"tensorboard\"\n# report metrics to tensorboard\nlearning_rate:\n0.0002\n# learning rate 2e-4\nlr_scheduler_type:\n\"constant\"\n# learning rate scheduler\nnum_train_epochs:\n3\n# number of training epochs\nper_device_train_batch_size:\n1\n# batch size per device during training\nper_device_eval_batch_size:\n1\n# batch size for evaluation\ngradient_accumulation_steps:\n2\n# number of steps before performing a backward/update pass\noptim: adamw_torch\n# use torch adamw optimizer\nlogging_steps:\n10\n# log every 10 steps\nsave_strategy: epoch\n# save checkpoint every epoch\nevaluation_strategy: epoch\n# evaluate every epoch\nmax_grad_norm:\n0.3\n# max gradient norm\nwarmup_ratio:\n0.03\n# warmup ratio\nbf16: true\n# use bfloat16 precision\ntf32: true\n# use tf32 precision\ngradient_checkpointing: true\n# use gradient checkpointing to save memory\n# FSDP parameters: https://huggingface.co/docs/transformers/main/en/fsdp\nfsdp:\n\"full_shard auto_wrap offload\"\n# remove offload if enough GPU memory\nfsdp_config:\nbackward_prefetch:\n\"backward_pre\"\nforward_prefetch:\n\"false\"\nuse_orig_params:\n\"false\"\nNote: At the end of the training there will be a slight increase in GPU memory usage (~10%). This is due to the saving of the model correctly. Make sure to have enough memory left on your GPU to save the model.\nREF\nTo launch our training we will use\ntorchrun\nto keep the example flexible and easy to adjust to, e.g. Amazon SageMaker or Google Cloud Vertex AI. For\ntorchrun\nand FSDP we need to set the environment variable\nACCELERATE_USE_FSDP\nand\nFSDP_CPU_RAM_EFFICIENT_LOADING\nto tell transformers/accelerate to use FSDP and load the model in a memory-efficient way.\nNote: To NOT CPU offloading you need to change the value of\nfsdp\nand remove\noffload\n. This only works on > 40GB GPUs since it requires more memory.\nNow, lets launch the training with the following command:\nPython\n!\nACCELERATE_USE_FSDP\n=\n1\nFSDP_CPU_RAM_EFFICIENT_LOADING\n=\n1\ntorchrun\n--\nnproc_per_node=\n4\n./scripts/run_fsdp_qlora.py\n--\nconfig llama_3_70b_fsdp_qlora.yaml\nExpected Memory usage:\nFull-finetuning with FSDP needs ~16X80GB GPUs\nFSDP + LoRA needs ~8X80GB GPUs\nFSDP + Q-Lora needs ~2x40GB GPUs\nFSDP + Q-Lora + CPU offloading needs 4x24GB GPUs, with 22 GB/GPU and 127 GB CPU RAM with a sequence length of 3072 and a batch size of 1.\nThe training of Llama 3 70B with Flash Attention for 3 epochs with a dataset of 10k samples takes 45h on a\ng5.12xlarge\n. The instance costs\n5.67$/h\nwhich would result in a total cost of\n255.15$\n. This sounds expensive but allows you to fine-tune a Llama 3 70B on small GPU resources. If we scale up the training to 4x H100 GPUs, the training time will be reduced to ~1,25h. If we assume 1x H100 costs\n5-10$/h\nthe total cost would between\n25$-50$\n.\nWe can see a trade-off between accessibility and performance. If you have access to more/better compute you can reduce the training time and cost, but even with small resources you can fine-tune a Llama 3 70B. The cost/performance is different since for 4x A10G GPUs we need to offload the model to the CPU which reduces the overall flops.\nNote: During evaluation and testing of the blog post I noticed that ~40 max steps (80 samples stacked to 3k sequence length) are enough for first results. The training for 40 steps ~1h or ~$5.\nOptional: Merge LoRA adapter in to the original model\nWhen using QLoRA, we only train adapters and not the full model. This means when saving the model during training we only save the adapter weights and not the full model. If you want to save the full model, which makes it easier to use with Text Generation Inference you can merge the adapter weights into the model weights using the\nmerge_and_unload\nmethod and then save the model with the\nsave_pretrained\nmethod. This will save a default model, which can be used for inference.\nNote: You might require > 192GB CPU Memory.\nPython\n#### COMMENT IN TO MERGE PEFT AND BASE MODEL ####\n# from peft import AutoPeftModelForCausalLM\n# # Load PEFT model on CPU\n# model = AutoPeftModelForCausalLM.from_pretrained(\n#     args.output_dir,\n#     torch_dtype=torch.float16,\n#     low_cpu_mem_usage=True,\n# )\n# # Merge LoRA and base model and save\n# merged_model = model.merge_and_unload()\n# merged_model.save_pretrained(args.output_dir,safe_serialization=True, max_shard_size=\"2GB\")\n4. Test Model and run Inference\nAfter the training is done we want to evaluate and test our model. We will load different samples from the original dataset and evaluate the model manually. Evaluating Generative AI models is not a trivial task since 1 input can have multiple correct outputs. If you want to learn more about evaluating generative models, check out\nEvaluate LLMs and RAG a practical example using Langchain and Hugging Face\nblog post.\nPython\nimport\ntorch\nfrom\npeft\nimport\nAutoPeftModelForCausalLM\nfrom\ntransformers\nimport\nAutoTokenizer\npeft_model_id =\n\"./llama-3-70b-hf-no-robot\"\n# Load Model with PEFT adapter\nmodel = AutoPeftModelForCausalLM.\nfrom_pretrained\n(\npeft_model_id,\ntorch_dtype\n=torch.float16,\nquantization_config\n= {\n\"load_in_4bit\"\n:\nTrue\n},\ndevice_map\n=\n\"auto\"\n)\ntokenizer = AutoTokenizer.\nfrom_pretrained\n(peft_model_id)\nLet’s load our test dataset try to generate an instruction.\nPython\nfrom\ndatasets\nimport\nload_dataset\nfrom\nrandom\nimport\nrandint\n# Load our test dataset\neval_dataset =\nload_dataset\n(\n\"json\"\n,\ndata_files\n=\n\"test_dataset.json\"\n,\nsplit\n=\n\"train\"\n)\nrand_idx =\nrandint\n(\n0\n, len(eval_dataset))\nmessages = eval_dataset[rand_idx][\n\"messages\"\n][:\n2\n]\n# Test on sample\ninput_ids = tokenizer.\napply_chat_template\n(messages,\nadd_generation_prompt\n=\nTrue\n,\nreturn_tensors\n=\n\"pt\"\n).\nto\n(model.device)\noutputs = model.\ngenerate\n(\ninput_ids,\nmax_new_tokens\n=\n512\n,\neos_token_id\n= tokenizer.eos_token_id,\ndo_sample\n=\nTrue\n,\ntemperature\n=\n0.6\n,\ntop_p\n=\n0.9\n,\n)\nresponse = outputs[\n0\n][input_ids.shape[-\n1\n]:]\nprint(\nf\n\"**Query:**\n\\n\n{\neval_dataset[rand_idx][\n'messages'\n][\n1\n][\n'content'\n]\n}\n\\n\n\"\n)\nprint(\nf\n\"**Original Answer:**\n\\n\n{\neval_dataset[rand_idx][\n'messages'\n][\n2\n][\n'content'\n]\n}\n\\n\n\"\n)\nprint(\nf\n\"**Generated Answer:**\n\\n\n{\ntokenizer.\ndecode\n(response,\nskip_special_tokens\n=\nTrue\n)\n}\n\"\n)\n# **Query:**\n# How long was the Revolutionary War?\n# **Original Answer:**\n# The American Revolutionary War lasted just over seven years. The war started on April 19, 1775, and ended on September 3, 1783.\n# **Generated Answer:**\n# The Revolutionary War, also known as the American Revolution, was an 18th-century war fought between the Kingdom of Great Britain and the Thirteen Colonies. The war lasted from 1775 to 1783.\nThat looks pretty good! 🚀 Now, its your turn!\nIf you want to deploy your model into production check out\nDeploy the LLM for Production\n.",
  "metadata": {
    "title": "Efficiently fine-tune Llama 3 with PyTorch FSDP and Q-Lora",
    "description": "Learn how to fine-tune Llama 3 70b with PyTorch FSDP and Q-Lora using Hugging Face TRL, Transformers, PEFT and Datasets.",
    "keywords": null,
    "author": "Philipp Schmid",
    "og:title": "Efficiently fine-tune Llama 3 with PyTorch FSDP and Q-Lora",
    "og:description": "Learn how to fine-tune Llama 3 70b with PyTorch FSDP and Q-Lora using Hugging Face TRL, Transformers, PEFT and Datasets.",
    "og:url": "https://www.philschmid.de/fsdp-qlora-llama3",
    "og:image": "https://www.philschmid.de/static/blog/fsdp-qlora-llama3/thumbnail.jpg",
    "og:image:width": "1200",
    "og:image:height": "630",
    "og:image:alt": "Efficiently fine-tune Llama 3 with PyTorch FSDP and Q-Lora",
    "og:type": "article",
    "twitter:card": "summary_large_image",
    "twitter:title": "Efficiently fine-tune Llama 3 with PyTorch FSDP and Q-Lora",
    "twitter:description": "Learn how to fine-tune Llama 3 70b with PyTorch FSDP and Q-Lora using Hugging Face TRL, Transformers, PEFT and Datasets.",
    "twitter:image": "https://www.philschmid.de/static/blog/fsdp-qlora-llama3/thumbnail.jpg",
    "article:published_time": "2024-04-22"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [
    {
      "file": "attachments/2068a07ff1d4/code/body-000.txt",
      "chars": 295,
      "language": "text"
    },
    {
      "file": "attachments/2068a07ff1d4/code/body-001.txt",
      "chars": 33,
      "language": "text"
    },
    {
      "file": "attachments/2068a07ff1d4/code/body-002.txt",
      "chars": 410,
      "language": "text"
    },
    {
      "file": "attachments/2068a07ff1d4/code/body-003.txt",
      "chars": 1338,
      "language": "text"
    },
    {
      "file": "attachments/2068a07ff1d4/code/body-004.txt",
      "chars": 1728,
      "language": "text"
    },
    {
      "file": "attachments/2068a07ff1d4/code/body-005.txt",
      "chars": 148,
      "language": "text"
    },
    {
      "file": "attachments/2068a07ff1d4/code/body-006.txt",
      "chars": 439,
      "language": "text"
    },
    {
      "file": "attachments/2068a07ff1d4/code/body-007.txt",
      "chars": 394,
      "language": "text"
    },
    {
      "file": "attachments/2068a07ff1d4/code/body-008.txt",
      "chars": 1278,
      "language": "text"
    }
  ],
  "images": [],
  "audio": [],
  "videos": [],
  "links": [
    {
      "href": "https://github.com/philschmid/deep-learning-pytorch-huggingface/blob/main/training/fsdp-qlora-distributed-llama3.ipynb",
      "label": "View Code",
      "type": "a"
    },
    {
      "href": "https://huggingface.co/meta-llama/Meta-Llama-3-70b",
      "label": "Llama 3",
      "type": "a"
    },
    {
      "href": "https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2",
      "label": "Mistral",
      "type": "a"
    },
    {
      "href": "https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1",
      "label": "Mixtral",
      "type": "a"
    },
    {
      "href": "https://huggingface.co/ai21labs/Jamba-v0.1",
      "label": "Jamba",
      "type": "a"
    },
    {
      "href": "https://huggingface.co/docs/trl/index",
      "label": "TRL",
      "type": "a"
    },
    {
      "href": "https://huggingface.co/docs/transformers/index",
      "label": "Transformers",
      "type": "a"
    },
    {
      "href": "https://huggingface.co/docs/peft/index",
      "label": "peft",
      "type": "a"
    },
    {
      "href": "https://huggingface.co/docs/datasets/index",
      "label": "datasets",
      "type": "a"
    },
    {
      "href": "https://pytorch.org/blog/pytorch2-2/",
      "label": "Flash Attention v2 through the Pytorch SDPA",
      "type": "a"
    },
    {
      "href": "https://www.answer.ai/posts/2024-03-06-fsdp-qlora.html",
      "label": "Answer.AI",
      "type": "a"
    },
    {
      "href": "https://github.com/TimDettmers/bitsandbytes",
      "label": "Q-Lora creator",
      "type": "a"
    },
    {
      "href": "https://huggingface.co/",
      "label": "Hugging Face",
      "type": "a"
    },
    {
      "href": "https://huggingface.co/docs/peft/v0.10.0/en/accelerate/fsdp",
      "label": "PEFT documentation",
      "type": "a"
    },
    {
      "href": "https://pytorch.org/blog/introducing-pytorch-fully-sharded-data-parallel-api/",
      "label": "PyTorch FSDP",
      "type": "a"
    },
    {
      "href": "https://huggingface.co/join",
      "label": "here",
      "type": "a"
    },
    {
      "href": "https://www.philschmid.de/fine-tune-llms-in-2024-with-trl",
      "label": "How to Fine-Tune LLMs in 2024 with Hugging Face",
      "type": "a"
    },
    {
      "href": "https://huggingface.co/datasets/HuggingFaceH4/no_robots",
      "label": "HuggingFaceH4/no_robots",
      "type": "a"
    },
    {
      "href": "https://huggingface.co/papers/2203.02155",
      "label": "InstructGPT paper",
      "type": "a"
    },
    {
      "href": "https://github.com/philschmid/deep-learning-pytorch-huggingface/blob/main/training/scripts/run_fsdp_qlora.py",
      "label": "run_fsdp_qlora.py",
      "type": "a"
    },
    {
      "href": "https://huggingface.co/docs/trl/sft_trainer",
      "label": "SFTTrainer",
      "type": "a"
    },
    {
      "href": "https://www.philschmid.de/evaluate-llm",
      "label": "Evaluate LLMs and RAG a practical example using Langchain and Hugging Face",
      "type": "a"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:52:47Z",
  "source_url": "https://www.philschmid.de/fsdp-qlora-llama3",
  "papers": [],
  "actual_video_files": [],
  "body_chars": 14473,
  "expected_body_images": 0,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/2068a07ff1d4/pages/source.html",
  "local_html": "attachments/2068a07ff1d4/pages/article.html",
  "body_base_path": "attachments/2068a07ff1d4/pages/",
  "body_markdown_path": "attachments/2068a07ff1d4/pages/article.md"
}