{
  "source_file": "0930",
  "source_platform": "个人博客",
  "author": "Philipp Schmid",
  "title_listed": "Fine-tune Llama 7B on AWS Trainium",
  "keywords": "Llama,Trainium,训练",
  "domain": "大模型算法",
  "content_type": "教程",
  "is_paid_flag": "否",
  "url": "https://www.philschmid.de/fine-tune-llama-7b-trainium",
  "publish_date": "2023-12",
  "corpus_stage": "预训练/SFT",
  "notes": "在AWS Trainium上微调Llama 7B",
  "id": "6a5b394eb81a",
  "collected_at": "2026-10-02T14:29:14",
  "title": "Fine-tune Llama 7B on AWS Trainium",
  "body_markdown": "Fine-tune Llama 7B on AWS Trainium\nDecember 21, 2023\n12\nminute read\nView Code\nThis tutorial will teach how to fine-tune open LLMs like\nLlama 2\non AWS Trainium. In our example, we are going to leverage Hugging Face\nhttps://huggingface.co/docs/optimum-neuron/index\n,\nTransformers\nand\nhttps://huggingface.co/docs/datasets/index\n.\nYou will learn how to:\nSetup AWS environment\nLoad and process the dataset\nFine-tune Llama on AWS Trainium using the\nNeuronTrainer\nEvalaute and test fine-tuned Llama model\nQuick intro: AWS Trainium\nAWS Trainium (Trn1)\nis a purpose-built EC2 for deep learning (DL) training workloads. Trainium is the successor of\nAWS Inferentia\nfocused on high-performance training workloads. Trainium has been optimized for training natural language processing, computer vision, and recommender models used. The accelerator supports a wide range of data types, including FP32, TF32, BF16, FP16, UINT8, and configurable FP8.\nThe biggest Trainium instance, the\ntrn1.32xlarge\ncomes with over 500GB of memory, making it easy to fine-tune ~10B parameter models on a single instance. Below you will find an overview of the available instance types. More details\nhere\n:\ninstance size\naccelerators\naccelerator memory\nvCPU\nCPU Memory\nprice per hour\ntrn1.2xlarge\n1\n32\n8\n32\n$1.34\ntrn1.32xlarge\n16\n512\n128\n512\n$21.50\ntrn1n.32xlarge (2x bandwidth)\n16\n512\n128\n512\n$24.78\nNote: This tutorial was created on a trn1.32xlarge AWS EC2 Instance.\n1. Setup AWS environment\nIn this example, we will use the\ntrn1.32xlarge\ninstance on AWS with 16 Accelerator, including 32 Neuron Cores and the\nHugging Face Neuron Deep Learning AMI\n. The Hugging Face AMI comes with all important libraries, like Transformers, Datasets, Optimum and Neuron packages pre-installed this makes it super easy to get started, since there is no need for environment management.\nThis blog post doesn’t cover how to create the instance in detail. You can check out my previous blog about\n“Setting up AWS Trainium for Hugging Face Transformers”\n, which includes a step-by-step guide on setting up the environment.\nOnce the instance is up and running, we can ssh into it. But instead of developing inside a terminal we want to use a\nJupyter\nenvironment, which we can use for preparing our dataset and launching the training. For this, we need to add a port for forwarding in the\nssh\ncommand, which will tunnel our localhost traffic to the Trainium instance.\nBash\nPUBLIC_DNS\n=\n\"\"\n# IP address, e.g. ec2-3-80-....\nKEY_PATH\n=\n\"\"\n# local path to key, e.g. ssh/trn.pem\nssh\n-L\n8080:localhost:8080\n-i\n${\nKEY_NAME\n}\n.pem\nubuntu@\n$PUBLIC_DNS\nLets now pull the optimum repository with the\nexample notebook and scripts\n.\nBash\ngit\nclone\nhttps://github.com/huggingface/optimum-neuron.git\nNext we can change our directory to\nnotbooks/text-generation\nand launch the\njupyter\nenvironment.``\nBash\n# change directory\ncd\noptimum-neuron/notebooks/text-generation\n# launch jupyter\npython\n-m\nnotebook\n--allow-root\n--port=8080\nYou should see a familiar\njupyter\noutput with a URL to the notebook.\nhttp://localhost:8080/?token=8c1739aff1755bd7958c4cfccc8d08cb5da5234f61f129a9\nWe can click on it, and a\njupyter\nenvironment opens in our local browser. Open the notebook\nllama2-7b-fine-tuning.ipynb\nand lets get started.\nNote: We are going to use the Jupyter environment only for preparing the dataset and then\ntorchrun\nfor launching our training script for distributed training.\nIf you are going to use official Llama 2 checkpoint you need to login into our hugging face account, which has access to the model, to use your token for accessing the gated repository. We can do this by running the following command:\nNote: We also provide an ungated checkpoint.\nPython\n!\nhuggingface-cli login\n--\ntoken\nYOUR_TOKEN\n2. Load and prepare the dataset\nWe will use\nDolly\nan open source dataset of instruction-following records on categories outlined in the\nInstructGPT paper\n, including brainstorming, classification, closed QA, generation, information extraction, open QA, and summarization.\nPython\n{\n\"instruction\"\n:\n\"What is world of warcraft\"\n,\n\"context\"\n:\n\"\"\n,\n\"response\"\n:\n\"World of warcraft is a massive online multi player role playing game. It was released in 2004 by bizarre entertainment\"\n}\nTo load the\ndolly\ndataset, we use the\nload_dataset()\nmethod from the 🤗 Datasets library.\nPython\nfrom\ndatasets\nimport\nload_dataset\nfrom\nrandom\nimport\nrandrange\n# Load dataset from the hub\ndataset =\nload_dataset\n(\n\"databricks/databricks-dolly-15k\"\n,\nsplit\n=\n\"train\"\n)\nprint(\nf\n\"dataset size:\n{\nlen(dataset)\n}\n\"\n)\nprint(dataset[\nrandrange\n(len(dataset))])\n# dataset size: 15011\nTo instruct tune our model we need to convert our structured examples into a collection of tasks described via instructions. We define a\nformatting_function\nthat takes a sample and returns a string with our format instruction.\nPython\ndef\nformat_dolly\n(\nsample\n):\ninstruction =\nf\n\"### Instruction\n\\n\n{\nsample[\n'instruction'\n]\n}\n\"\ncontext =\nf\n\"### Context\n\\n\n{\nsample[\n'context'\n]\n}\n\"\nif\nlen(sample[\n\"context\"\n]) >\n0\nelse\nNone\nresponse =\nf\n\"### Answer\n\\n\n{\nsample[\n'response'\n]\n}\n\"\n# join all the parts together\nprompt =\n\"\n\\n\\n\n\"\n.\njoin\n([i\nfor\ni\nin\n[instruction, context, response]\nif\ni\nis\nnot\nNone\n])\nreturn\nprompt\nlets test our formatting function on a random example.\nPython\nfrom\nrandom\nimport\nrandrange\nprint(\nformat_dolly\n(dataset[\nrandrange\n(len(dataset))]))\nIn addition, to formatting our samples we also want to pack multiple samples to one sequence to have a more efficient training. This means that we are stacking multiple samples to one sequence and split them with an EOS Token. This makes the training more efficient. Packing/stacking samples can be done during training or before. We will do it before training to save time. We created a utility method\npack_dataset\nthat takes a dataset and a packing function and returns a packed dataset.\nPython\nfrom\ntransformers\nimport\nAutoTokenizer\n# Hugging Face model id\nmodel_id =\n\"philschmid/Llama-2-7b-hf\"\n# ungated\n# model_id = \"meta-llama/Llama-2-7b-hf\" # gated\ntokenizer = AutoTokenizer.\nfrom_pretrained\n(model_id)\nTo pack/stack our dataset we need to first tokenize it and then we can pack it with the\npack_dataset\nmethod. To prepare our dataset we will now:\nFormat our samples using the template method and add an EOS token at the end of each sample\nTokenize our dataset to convert it from text to tokens\nPack our dataset to 2048 tokens\nPython\nfrom\nrandom\nimport\nrandint\n# add utils method to path for loading dataset\nimport\nsys\nsys.path.\nappend\n(\n\"./scripts/utils\"\n)\n# make sure you change this to the correct path\nfrom\npack_dataset\nimport\npack_dataset\n# template dataset to add prompt to each sample\ndef\ntemplate_dataset\n(\nsample\n):\nsample[\n\"text\"\n] =\nf\n\"\n{\nformat_dolly\n(sample)\n}{\ntokenizer.eos_token\n}\n\"\nreturn\nsample\n# apply prompt template per sample\ndataset = dataset.\nmap\n(template_dataset,\nremove_columns\n=list(dataset.features))\n# print random sample\nprint(dataset[\nrandint\n(\n0\n, len(dataset))][\n\"text\"\n])\n# tokenize dataset\ndataset = dataset.\nmap\n(\nlambda\nsample\n:\ntokenizer\n(sample[\n\"text\"\n]),\nbatched\n=\nTrue\n,\nremove_columns\n=list(dataset.features)\n)\n# chunk dataset\nlm_dataset =\npack_dataset\n(dataset,\nchunk_length\n=\n2048\n)\n# We use 2048 as the maximum length for packing\nAfter we processed the datasets we are going save it to disk. You could also save it to S3 or the Hugging Face Hub for later use.\nNote: Packing and preprocessing your dataset can be run outside of the Trainium instance.\nPython\n# save train_dataset to disk\ndataset_path =\n\"tokenized_dolly\"\nlm_dataset.\nsave_to_disk\n(dataset_path)\n3. Fine-tune Llama on AWS Trainium using the\nNeuronTrainer\nNormally you would use the\nTrainer\nand\nTrainingArguments\nto fine-tune PyTorch-based transformer models.\nBut together with AWS, we have developed a\nNeuronTrainer\nto improve performance, robustness, and safety when training on Trainium instances. The\nNeuronTrainer\nis part of the\noptimum-neuron\nlibrary and can be used as a 1-to-1 replacement for the\nTrainer\n.\nWhen it comes to distributed training on AWS Trainium there is a few things we need to take care of. Since Llama is a big model it might not fit on a single accelerator, thats why we added support for different distributed training strategies to the\nNeuronTrainer\nincluding:\nZeRO-1\n: shards the optimizer state over multiple devices.\nTensor Parallelism\n: shards the model parameters along a given dimension on multiple devices, defined with\ntensor_parallel_size\nSequence parallelism\nshards the activations on the sequence axis outside of the tensor parallel regions. It is useful because it saves memory by sharding the activations.\nPipeline Parallelism\n:\ncoming soon\nWe prepared a\nrun_clm.py\n, which implements those distributed training strategies for you already. If you want to more about the details you can take a look at the\ndocumentation\n. When training models on AWS Accelerators we first need to compile our model with our training arguments.\nTo overcome this we added a\nmodel cache\n, which allows us to use precompiled models and configuration from Hugging Face Hub to skip the compilation step. But every change in the config, will lead to a new compilation, which could result in some cache misses.\nNote: If your configuration is not cached please open an issue on\nGithub\n, we are happy to include it.\nWe pre-compiled the config for our training already meaning you can either skip the cell below or rerun it will only take a few minutes since it reuses the cached configuration.\nPython\n# precompilation command\n!\nMALLOC_ARENA_MAX\n=\n64\nneuron_parallel_compile torchrun\n--\nnproc_per_node=\n32\nscripts/run_clm.py \\\n--\nmodel_id {model_id} \\\n--\ndataset_path {dataset_path} \\\n--\nbf16\nTrue\n\\\n--\nlearning_rate\n5e-5\n\\\n--\noutput_dir dolly_llama \\\n--\noverwrite_output_dir\nTrue\n\\\n--\nper_device_train_batch_size\n1\n\\\n--\ngradient_checkpointing\nTrue\n\\\n--\ntensor_parallel_size\n8\n\\\n--\nmax_steps\n10\n\\\n--\nlogging_steps\n10\n\\\n--\ngradient_accumulation_steps\n16\nNote: Compiling without a cache can take ~40 minutes. It will also create dummy files in the\ndolly_llama_sharded\nduring compilation you we have to remove them afterwards. We also need to add\nMALLOC_ARENA_MAX=64\nto limit the CPU allocation to avoid potential crashes, don't remove it for now.\nPython\n# remove dummy artifacts which are created by the precompilation command\n!\nrm -rf dolly_llama\nAfter the compilation is done we can start our training with a similar command, we just need to remove the\nneuron_parallel_compile\n. We will use\ntorchrun\nto launch our training script.\ntorchrun\nis a tool that automatically distributes a PyTorch model across multiple accelerators. We can pass the number of accelerators as\nnproc_per_node\narguments alongside our hyperparameters.\nThe difference to the compilation command is that we changed from\nmax_steps=10\nto\nnum_train_epochs=3\n.\nLaunch the training, with the following command.\nPython\n!\nMALLOC_ARENA_MAX\n=\n64\ntorchrun\n--\nnproc_per_node=\n32\nscripts/run_clm.py \\\n--\nmodel_id {model_id} \\\n--\ndataset_path {dataset_path} \\\n--\nbf16\nTrue\n\\\n--\nlearning_rate\n5e-5\n\\\n--\noutput_dir dolly_llama \\\n--\noverwrite_output_dir\nTrue\n\\\n--\nskip_cache_push\nTrue\n\\\n--\nper_device_train_batch_size\n1\n\\\n--\ngradient_checkpointing\nTrue\n\\\n--\ntensor_parallel_size\n8\n\\\n--\nnum_train_epochs\n3\n\\\n--\nlogging_steps\n10\n\\\n--\ngradient_accumulation_steps\n16\nThats it, we successfully trained Llama 7B on AWS Trainium. The training took for 3 epochs on dolly (15k samples) took 43:24 minutes where the raw training time was only 31:46 minutes. This leads to a cost of ~$15.5 for the e2e training on the trn1.32xlarge instance. Not Bad!\nBut before we can share and test our model we need to consolidate our model. Since we used Tensor Parallelism during training, we need to consolidate the model weights before we can use it. Tensor Parallelism shards the model weights accross different workers, only sharded checkpoints will be saved during training.\nThe Optimum CLI provides a way of doing that very easily via the `optimum neuron consolidate`` command:\nPython\n!\noptimum-cli neuron consolidate dolly_llama/tensor_parallel_shards dolly_llama\nLets remove our \"sharded\" checkpoints as we have consolidated them already to safetensors.\nPython\n!\nrm -rf dolly_llama/tensor_parallel_shards\n4. Evalaute and test fine-tuned Llama model\nSimilar to training to be able to run inferece on AWS Trainium or AWS Inferentia2 we need to compile our model for the correct use. We will use our Trainium instance for the inference test, but we recommend customer to switch to Inferentia2 for inference.\nOptimum Neuron implements similar to Transformers AutoModel classes for easy inference use. We will use the\nNeuronModelForCausalLM\nclass to load our vanilla transformers checkpoint and convert it to neuron.\nPython\nfrom\noptimum.neuron\nimport\nNeuronModelForCausalLM\nfrom\ntransformers\nimport\nAutoTokenizer\ncompiler_args = {\n\"num_cores\"\n:\n2\n,\n\"auto_cast_type\"\n:\n'fp16'\n}\ninput_shapes = {\n\"batch_size\"\n:\n1\n,\n\"sequence_length\"\n:\n2048\n}\ntokenizer = AutoTokenizer.\nfrom_pretrained\n(\n\"dolly_llama\"\n)\nmodel = NeuronModelForCausalLM.\nfrom_pretrained\n(\n\"dolly_llama\"\n,\nexport\n=\nTrue\n,\n**compiler_args,\n**input_shapes)\nNote: Inference compilation can take ~25minutes. Luckily, you need to only run this onces. Since you can save the model afterwards. If you are going to run on Inferentia2 you need to recompile again. The compilation is parameter and hardware specific.\nPython\n# COMMENT IN if you want to save the compiled model\n# model.save_pretrained(\"compiled_dolly_llama\")\nWe can now test inference, but have to make sure we format our input to our prompt format we used for fine-tuning. Therefore we created a helper method, which accepts a\ndict\nwith our\ninstruction\nand optionally a\ncontext\n.\nPython\ndef\nformat_dolly_infernece\n(\nsample\n):\ninstruction =\nf\n\"### Instruction\n\\n\n{\nsample[\n'instruction'\n]\n}\n\"\ncontext =\nf\n\"### Context\n\\n\n{\nsample[\n'context'\n]\n}\n\"\nif\n\"context\"\nin\nsample\nelse\nNone\nresponse =\nf\n\"### Answer\n\\n\n\"\n# join all the parts together\nprompt =\n\"\n\\n\\n\n\"\n.\njoin\n([i\nfor\ni\nin\n[instruction, context, response]\nif\ni\nis\nnot\nNone\n])\nreturn\nprompt\ndef\ngenerate\n(\nsample\n):\nprompt =\nformat_dolly_infernece\n(sample)\ninputs =\ntokenizer\n(prompt,\nreturn_tensors\n=\n\"pt\"\n)\noutputs = model.\ngenerate\n(**inputs,\nmax_new_tokens\n=\n512\n,\ndo_sample\n=\nTrue\n,\ntemperature\n=\n0.9\n,\ntop_k\n=\n50\n,\ntop_p\n=\n0.9\n)\nreturn\ntokenizer.\ndecode\n(outputs[\n0\n],\nskip_special_tokens\n=\nFalse\n)[len(prompt):]\nLets test inference. First we test without a context.\nNote: Inference is not expected to be super fast on AWS Trainium using 2 cores. For Inference we recommend using Inferentia2.\nPython\nprompt = {\n\"instruction\"\n:\n\"Can you tell me something about AWS?\"\n}\nres =\ngenerate\n(prompt)\nprint(res)\nAWS stands for Amazon Web Services. AWS is a suite of remote computing services offered by Amazon. The most widely used of these include Amazon Elastic Compute Cloud (Amazon EC2), which provides resizable compute capacity in the cloud; Amazon Simple Storage Service (Amazon S3), which is an object storage service; and Amazon Elastic Block Store (Amazon EBS), which is designed to provide high performance, durable block storage volumes for use with AWS instances. AWS also provides other services, such as AWS Identity and Access Management (IAM), a service that enables organizations to control access to their AWS resources, and AWS Key Management Service (AWS KMS), which helps customers create and control the use of encryption keys.\nThat looks correct. Now, lets add some context, e.g. as you would do for RAG applications\nPython\nprompt = {\n\"instruction\"\n:\n\"How can train models on AWS Trainium?\"\n,\n\"context\"\n:\n\"🤗 Optimum Neuron is the interface between the 🤗 Transformers library and AWS Accelerators including [AWS Trainium](https://aws.amazon.com/machine-learning/trainium/?nc1=h_ls) and [AWS Inferentia](https://aws.amazon.com/machine-learning/inferentia/?nc1=h_ls). It provides a set of tools enabling easy model loading, training and inference on single- and multi-Accelerator settings for different downstream tasks.\"\n}\nres =\ngenerate\n(prompt)\nprint(res)\nYou can use the Optimum Neuron interface to train models on AWS Trainium.\nAwesome, our model also correctly uses the provided context. We are done. Congrats on fine-tuning Llama on AWS Trainium.\nThanks for reading! If you have any questions, feel free to contact me on\nTwitter\nor\nLinkedIn\n.",
  "metadata": {
    "title": "Fine-tune Llama 7B on AWS Trainium",
    "description": "In this blog post you will learn how to fine-tune Llama 7B on AWS Trainium using the Hugging Face Optimum Neuron library.",
    "keywords": null,
    "author": "Philipp Schmid",
    "og:title": "Fine-tune Llama 7B on AWS Trainium",
    "og:description": "In this blog post you will learn how to fine-tune Llama 7B on AWS Trainium using the Hugging Face Optimum Neuron library.",
    "og:url": "https://www.philschmid.de/fine-tune-llama-7b-trainium",
    "og:image": "https://www.philschmid.de/static/blog/fine-tune-llama-7b-trainium/thumbnail.jpg",
    "og:image:width": "1200",
    "og:image:height": "630",
    "og:image:alt": "Fine-tune Llama 7B on AWS Trainium",
    "og:type": "article",
    "twitter:card": "summary_large_image",
    "twitter:title": "Fine-tune Llama 7B on AWS Trainium",
    "twitter:description": "In this blog post you will learn how to fine-tune Llama 7B on AWS Trainium using the Hugging Face Optimum Neuron library.",
    "twitter:image": "https://www.philschmid.de/static/blog/fine-tune-llama-7b-trainium/thumbnail.jpg",
    "article:published_time": "2023-12-21"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [
    {
      "file": "attachments/6a5b394eb81a/code/body-000.txt",
      "chars": 163,
      "language": "text"
    },
    {
      "file": "attachments/6a5b394eb81a/code/body-001.txt",
      "chars": 59,
      "language": "text"
    },
    {
      "file": "attachments/6a5b394eb81a/code/body-002.txt",
      "chars": 123,
      "language": "text"
    },
    {
      "file": "attachments/6a5b394eb81a/code/body-003.txt",
      "chars": 41,
      "language": "text"
    },
    {
      "file": "attachments/6a5b394eb81a/code/body-004.txt",
      "chars": 201,
      "language": "text"
    },
    {
      "file": "attachments/6a5b394eb81a/code/body-005.txt",
      "chars": 270,
      "language": "text"
    },
    {
      "file": "attachments/6a5b394eb81a/code/body-006.txt",
      "chars": 371,
      "language": "text"
    },
    {
      "file": "attachments/6a5b394eb81a/code/body-007.txt",
      "chars": 84,
      "language": "text"
    },
    {
      "file": "attachments/6a5b394eb81a/code/body-008.txt",
      "chars": 214,
      "language": "text"
    },
    {
      "file": "attachments/6a5b394eb81a/code/body-009.txt",
      "chars": 824,
      "language": "text"
    },
    {
      "file": "attachments/6a5b394eb81a/code/body-010.txt",
      "chars": 99,
      "language": "text"
    },
    {
      "file": "attachments/6a5b394eb81a/code/body-011.txt",
      "chars": 445,
      "language": "text"
    },
    {
      "file": "attachments/6a5b394eb81a/code/body-012.txt",
      "chars": 92,
      "language": "text"
    },
    {
      "file": "attachments/6a5b394eb81a/code/body-013.txt",
      "chars": 428,
      "language": "text"
    },
    {
      "file": "attachments/6a5b394eb81a/code/body-014.txt",
      "chars": 78,
      "language": "text"
    },
    {
      "file": "attachments/6a5b394eb81a/code/body-015.txt",
      "chars": 42,
      "language": "text"
    },
    {
      "file": "attachments/6a5b394eb81a/code/body-016.txt",
      "chars": 409,
      "language": "text"
    },
    {
      "file": "attachments/6a5b394eb81a/code/body-017.txt",
      "chars": 99,
      "language": "text"
    },
    {
      "file": "attachments/6a5b394eb81a/code/body-018.txt",
      "chars": 793,
      "language": "text"
    },
    {
      "file": "attachments/6a5b394eb81a/code/body-019.txt",
      "chars": 104,
      "language": "text"
    },
    {
      "file": "attachments/6a5b394eb81a/code/body-020.txt",
      "chars": 533,
      "language": "text"
    }
  ],
  "images": [],
  "audio": [],
  "videos": [],
  "links": [
    {
      "href": "https://github.com/huggingface/optimum-neuron/blob/main/notebooks/text-generation/llama2-7b-fine-tuning.ipynb",
      "label": "View Code",
      "type": "a"
    },
    {
      "href": "https://huggingface.co/meta-llama/Llama-2-70b-hf",
      "label": "Llama 2",
      "type": "a"
    },
    {
      "href": "https://huggingface.co/docs/optimum-neuron/index",
      "label": "https://huggingface.co/docs/optimum-neuron/index",
      "type": "a"
    },
    {
      "href": "https://huggingface.co/docs/transformers/index",
      "label": "Transformers",
      "type": "a"
    },
    {
      "href": "https://huggingface.co/docs/datasets/index",
      "label": "https://huggingface.co/docs/datasets/index",
      "type": "a"
    },
    {
      "href": "https://aws.amazon.com/de/ec2/instance-types/trn1/",
      "label": "AWS Trainium (Trn1)",
      "type": "a"
    },
    {
      "href": "https://aws.amazon.com/ec2/instance-types/inf1/?nc1=h_ls",
      "label": "AWS Inferentia",
      "type": "a"
    },
    {
      "href": "https://aws.amazon.com/marketplace/pp/prodview-gr3e6yiscria2",
      "label": "Hugging Face Neuron Deep Learning AMI",
      "type": "a"
    },
    {
      "href": "https://www.philschmid.de/setup-aws-trainium",
      "label": "“Setting up AWS Trainium for Hugging Face Transformers”",
      "type": "a"
    },
    {
      "href": "https://github.com/huggingface/optimum-neuron/tree/main/notebooks/text-generation",
      "label": "example notebook and scripts",
      "type": "a"
    },
    {
      "href": "https://huggingface.co/datasets/databricks/databricks-dolly-15k",
      "label": "Dolly",
      "type": "a"
    },
    {
      "href": "https://arxiv.org/abs/2203.02155",
      "label": "InstructGPT paper",
      "type": "a"
    },
    {
      "href": "https://www.philschmid.de/scripts/utils/pack_dataset.py",
      "label": "pack_dataset",
      "type": "a"
    },
    {
      "href": "https://huggingface.co/docs/transformers/v4.19.4/en/main_classes/trainer",
      "label": "Trainer",
      "type": "a"
    },
    {
      "href": "https://awsdocs-neuron.readthedocs-hosted.com/en/latest/frameworks/torch/torch-neuronx/tutorials/training/zero1_gpt2.html",
      "label": "ZeRO-1",
      "type": "a"
    },
    {
      "href": "https://awsdocs-neuron.readthedocs-hosted.com/en/latest/libraries/neuronx-distributed/tensor_parallelism_overview.html",
      "label": "Tensor Parallelism",
      "type": "a"
    },
    {
      "href": "https://arxiv.org/pdf/2205.05198.pdf",
      "label": "Sequence parallelism",
      "type": "a"
    },
    {
      "href": "https://awsdocs-neuron.readthedocs-hosted.com/en/latest/libraries/neuronx-distributed/pipeline_parallelism_overview.html",
      "label": "Pipeline Parallelism",
      "type": "a"
    },
    {
      "href": "https://github.com/huggingface/optimum-neuron/blob/main/notebooks/text-generation/scripts/run_clm.py",
      "label": "run_clm.py",
      "type": "a"
    },
    {
      "href": "https://huggingface.co/docs/optimum-neuron/guides/distributed_training",
      "label": "documentation",
      "type": "a"
    },
    {
      "href": "https://huggingface.co/docs/optimum-neuron/guides/cache_system",
      "label": "model cache",
      "type": "a"
    },
    {
      "href": "https://github.com/huggingface/optimum-neuron/issues",
      "label": "Github",
      "type": "a"
    },
    {
      "href": "https://twitter.com/_philschmid",
      "label": "Twitter",
      "type": "a"
    },
    {
      "href": "https://www.linkedin.com/in/philipp-schmid-a6a2bb196/",
      "label": "LinkedIn",
      "type": "a"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:52:49Z",
  "source_url": "https://www.philschmid.de/fine-tune-llama-7b-trainium",
  "papers": [
    {
      "status": "ok",
      "url": "https://arxiv.org/pdf/2203.02155",
      "bytes": 1797405,
      "reused": true,
      "local_path": "papers/repair/4b8cf107df7118e0ffe8.pdf"
    },
    {
      "status": "ok",
      "url": "https://arxiv.org/pdf/2205.05198.pdf",
      "bytes": 3050278,
      "reused": false,
      "local_path": "papers/repair/f0d6c16a2cefb99e456e.pdf"
    }
  ],
  "actual_video_files": [],
  "body_chars": 16359,
  "expected_body_images": 0,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/6a5b394eb81a/pages/source.html",
  "local_html": "attachments/6a5b394eb81a/pages/article.html",
  "body_base_path": "attachments/6a5b394eb81a/pages/",
  "body_markdown_path": "attachments/6a5b394eb81a/pages/article.md"
}