{ "source_file": "0930", "source_platform": "个人博客", "author": "Philipp Schmid", "title_listed": "Fine-tune LLMs in 2025", "keywords": "微调,LLM,TRL", "domain": "大模型算法", "content_type": "教程", "is_paid_flag": "否", "url": "https://www.philschmid.de/fine-tune-llms-in-2025", "publish_date": "2024-12", "corpus_stage": "预训练/SFT", "notes": "2025年LLM微调指南", "id": "3223e7064646", "collected_at": "2026-10-02T14:29:12", "title": "How to fine-tune open LLMs in 2025 with Hugging Face", "body_markdown": "How to fine-tune open LLMs in 2025 with Hugging Face\nDecember 20, 2024\n18\nminute read\nView Code\nLarge Language Models (LLMs) continued their important role in 2024, with several major developments completely outperforming previous models. The focus continued to more smaller, more powerful models from companies like Meta, Qwen, or Google. These models not only became more powerful, but also more efficient. We got Llama models as small as 1B parameters outperforming Llama 2 13B.\nLLMs can now handle many tasks out-of-the-box through prompting, including chatbots, question answering, and summarization. However, for specialized applications requiring high accuracy or domain expertise, fine-tuning remains a powerful approach to achieve higher quality results than prompting alone, reduce costs by training smaller, more efficient models, and ensure reliability and consistency for specific use cases.\nContrary to last years guide\nHow to Fine-Tune LLMs in 2024 with Hugging Face\nthis guide focuses more on optimization, distributed training and being more customizable. This means support for different PEFT methods from Full-Finetuning to QLoRA and Spectrum, optimizations for faster and more efficient training, with\nFlash Attention\nor\nLiger Kernels\nand how to scale training to multiple GPUs using DeepSpeed.\nThis guide is created using a script rather than notebook. If you are compeltely new to fine-tuning LLMs, I recommend you to start with the\nHow to Fine-Tune LLMs in 2024 with Hugging Face\nguide and then come back to this guide.\nYou will learn how to:\nDefine a good use case for fine-tuning\nSetup the development environment\nCreate and prepare the dataset\nFine-tune the model using\ntrl\nand the\nSFTTrainer\nwith QLoRA as example\nTest and evaluate the model using GSM8K\nWhat is Qlora?\nQLoRA (Quantized Low-Rank Adaptation)\nenables efficient fine-tuning of LLMs using 4-bit quantization and minimal parameter updates, reducing resource needs but potentially impacting performance due to quantization trade-offs.\nWhat is Spectrum?\nSpectrum\nis a fine-tuning method that identifies the most informative layers of a LLM using Signal-to-Noise Ratio (SNR) analysis and selectively fine-tunes them, offering performance comparable to full fine-tuning with reduced resource usage, especially in distributed training setups.\nNote: This guide is designed for consumer GPUs (24GB+) like the NVIDIA RTX 4090/5090 or A10G, but can be adapted for larger systems.\n1. Define a good use case for fine-tuning\nOpen LLMs became more powerful and smaller in 2024. This often could mean fine-tuning might not be the first choice to solve your problem. Before you think about fine-tuning, you should always evaluate if prompting or already fine-tuned models can solve your problem. Create an evaluation setup and compare the performance of existing open models.\nHowever, fine-tuning can be particularly valuable in several scenarios. When you need to:\nConsistently improve performance on a specific set of tasks\nControl the style and format of model outputs (e.g., enforcing a company's tone of voice)\nTeach the model domain-specific knowledge or terminology\nReduce hallucinations for critical applications\nOptimize for latency by creating smaller, specialized models\nEnsure consistent adherence to specific guidelines or constraints\nAs an example, we are going to use the following use case:\nWe want to fine-tune a model, which can solve high-school math problems to teach students how to solve math problems.\nThis can be a good use case for fine-tuning, as it requires a lot of domain-specific knowledge about math and how to solve math problems.\nNote: This is a made-up example, as existing open models already can solve this task.\n2. Setup development environment\nOur first step is to install Hugging Face Libraries and Pyroch, including trl, transformers and datasets. If you haven't heard of trl yet, don't worry. It is a new library on top of transformers and datasets, which makes it easier to fine-tune, rlhf, align open LLMs.\nPython\n# Install Pytorch & other libraries\n%pip install\n\"torch==2.4.1\"\ntensorboard flash-attn\n\"liger-kernel==0.4.2\"\n\"setuptools<71.0.0\"\n\"deepspeed==0.15.4\"\nopenai\n\"lm-eval[api]==0.4.5\"\n# Install Hugging Face libraries\n%pip install\n--\nupgrade \\\n\"transformers==4.46.3\"\n\\\n\"datasets==3.1.0\"\n\\\n\"accelerate==1.1.1\"\n\\\n\"bitsandbytes==0.44.1\"\n\\\n\"trl==0.12.1\"\n\\\n\"peft==0.13.2\"\n\\\n\"lighteval==0.6.2\"\n\\\n\"hf-transfer==0.1.8\"\nWe will use the\nHugging Face Hub\nas a remote model versioning service. This means we will automatically push our model, logs and information to the Hub during training. You must register on the\nHugging Face\nfor this. After you have an account, we will use the\nlogin\nutil from the\nhuggingface_hub\npackage to log into our account and store our token (access key) on the disk.\nPython\nfrom\nhuggingface_hub\nimport\nlogin\nlogin\n(\ntoken\n=\n\"\"\n,\nadd_to_git_credential\n=\nTrue\n)\n# ADD YOUR TOKEN HERE\n3. Create and prepare the dataset\nOnce you've determined that fine-tuning is the right solution, you'll need a dataset. Most datasets are now created using automated synthetic workflows with LLMs, though several approaches exist:\nSynthetic Generation with LLMs\n: Most common approach using frameworks like\nDistilabel\nto generate high-quality synthetic data at scale\nExisting Datasets\n: Using public datasets from\nHugging Face Hub\nHuman Annotation\n: For highest quality but most expensive option\nThe\nLLM Datasets\nprovides an overview of high-quality datasets to fine-tune LLMs for all kind of purposes. For our example, we'll use\nOrca-Math\ndataset including 200,000 Math world problems.\nModern fine-tuning frameworks like\ntrl\nsupport standard formats:\nJSON\n// Conversation format\n{\n\"messages\"\n: [\n{\n\"role\"\n:\n\"system\"\n,\n\"content\"\n:\n\"You are...\"\n},\n{\n\"role\"\n:\n\"user\"\n,\n\"content\"\n:\n\"...\"\n},\n{\n\"role\"\n:\n\"assistant\"\n,\n\"content\"\n:\n\"...\"\n},\n]\n}\n// Instruction format\n{\n\"prompt\"\n:\n\"\"\n,\n\"completion\"\n:\n\"\"\n}\nNote: If you are interested in a guide on how to create high-quality datasets, let me know.\nTo prepare our datasets we will use the Datasets library and then convert it into the the conversational format, where we include the schema definition in the system message for our assistant. We'll then save the dataset as jsonl file, which we can then use to fine-tune our model.\nNote: This step can be different for your use case. For example, if you have already a dataset from, e.g. working with OpenAI, you can skip this step and go directly to the fine-tuning step.\nPython\nfrom\ndatasets\nimport\nload_dataset\n# Create system prompt\nsystem_message =\n\"\"\"Solve the given high school math problem by providing a clear explanation of each step leading to the final solution.\nProvide a detailed breakdown of your calculations, beginning with an explanation of the problem and describing how you derive each formula, value, or conclusion. Use logical steps that build upon one another, to arrive at the final answer in a systematic manner.\n# Steps\n1. **Understand the Problem**: Restate the given math problem and clearly identify the main question and any important given values.\n2. **Set Up**: Identify the key formulas or concepts that could help solve the problem (e.g., algebraic manipulation, geometry formulas, trigonometric identities).\n3. **Solve Step-by-Step**: Iteratively progress through each step of the math problem, justifying why each consecutive operation brings you closer to the solution.\n4. **Double Check**: If applicable, double check the work for accuracy and sense, and mention potential alternative approaches if any.\n5. **Final Answer**: Provide the numerical or algebraic solution clearly, accompanied by appropriate units if relevant.\n# Notes\n- Always clearly define any variable or term used.\n- Wherever applicable, include unit conversions or context to explain why each formula or step has been chosen.\n- Assume the level of mathematics is suitable for high school, and avoid overly advanced math techniques unless they are common at that level.\n\"\"\"\n# convert to messages\ndef\ncreate_conversation\n(\nsample\n):\nreturn\n{\n\"messages\"\n: [\n{\n\"role\"\n:\n\"system\"\n,\n\"content\"\n: system_message},\n{\n\"role\"\n:\n\"user\"\n,\n\"content\"\n: sample[\n\"question\"\n]},\n{\n\"role\"\n:\n\"assistant\"\n,\n\"content\"\n: sample[\n\"answer\"\n]}\n]\n}\n# Load dataset from the hub\ndataset =\nload_dataset\n(\n\"microsoft/orca-math-word-problems-200k\"\n,\nsplit\n=\n\"train\"\n)\n# Convert dataset to OAI messages\ndataset = dataset.\nmap\n(create_conversation,\nremove_columns\n=dataset.features,\nbatched\n=\nFalse\n)\nprint(dataset[\n345\n][\n\"messages\"\n])\n# save datasets to disk\ndataset.\nto_json\n(\n\"train_dataset.json\"\n,\norient\n=\n\"records\"\n)\n4. Fine-tune the model using\ntrl\nand the\nSFTTrainer\nwith QLoRA\nWe are now ready to fine-tune our model. We will use the\nSFTTrainer\nfrom\ntrl\nto fine-tune our model. The\nSFTTrainer\nmakes it straightfoward to supervise fine-tune open LLMs. The\nSFTTrainer\nis a subclass of the\nTrainer\nfrom the\ntransformers\nlibrary and supports all the same features, including logging, evaluation, and checkpointing, but adds additiional quality of life features, including:\nDataset formatting, including conversational and instruction format\nTraining on completions only, ignoring prompts\nPacking datasets for more efficient training\nPEFT (parameter-efficient fine-tuning) support including Q-LoRA, or Spectrum\nPreparing the model and tokenizer for conversational fine-tuning (e.g. adding special tokens)\ndistributed training with\naccelerate\nand FSDP/DeepSpeed\nWe prepared a\nrun_sft.py\nscripts, which supports providing a yaml configuration file to run the fine-tuning. This allows you to easily change the model, dataset, hyperparameters, and other settings. This is done by using the\nTrlParser\n, which parses the yaml file and converts it into the\nTrainingArguments\narguments. That way we can support Q-LoRA, Spectrum, and other PEFT methods with the same script. See Appendix A for execution examples for different models and PEFT methods and distributed training.\nQuestion: Why don't we use frameworks like\naxolotl\n?\nThat's a great question! Axolotl is a fantastic framework, it is used by many open source builders and is well tested. However, it is good to know how to do things manually. This will give you a better understanding of the inner workings and how it can be customized. Especially when you ran into an issue or want to extend the scripts and add new features.\nBefore we can start our training lets take a look at our\ntraining script\n. The script is kept very simple and is easy to understand. This should help you understand, customize and extend the script for your own use case. We define\ndataclasses\nfor our arguments. Every argument can then be provided either via the command line or by providing a yaml configuration file. That way we have better type safety and intellisense support.\nPython\n# ....\n@dataclass\nclass\nScriptArguments\n:\ndataset_id_or_path: str\n...\n# ....\nWe can customize behavior for different training methods and use them in our script with\nscript_args\n. The training script is separated by\n#######\nblocks for the different parts of the script. The main training function:\nLogs all hyperperparameters\nLoads the dataset from Hugging Face Hub or local disk\nLoads the tokenizer and model with our training strategy (e.g. Q-LoRA, Spectrum)\nInitializes the\nSFTTrainer\nStarts the training loop (optionally continue training from a checkpoint)\nSaves the model and optionally pushes it to the Hugging Face Hub\nBelow is an example recipe of how we can fine-tune a\nLlama-3.1-8B model with Q-LoRA\n.\nYAML\n# Model arguments\nmodel_name_or_path\n:\nMeta-Llama/Meta-Llama-3.1-8B\ntokenizer_name_or_path\n:\nMeta-Llama/Meta-Llama-3.1-8B-Instruct\nmodel_revision\n:\nmain\ntorch_dtype\n:\nbfloat16\nattn_implementation\n:\nflash_attention_2\nuse_liger\n:\ntrue\nbf16\n:\ntrue\ntf32\n:\ntrue\noutput_dir\n:\nruns/llama-3-1-8b-math-orca-qlora-10k-ep1\n# Dataset arguments\ndataset_id_or_path\n:\ntrain_dataset.json\nmax_seq_length\n:\n1024\npacking\n:\ntrue\n# LoRA arguments\nuse_peft\n:\ntrue\nload_in_4bit\n:\ntrue\nlora_target_modules\n:\n\"all-linear\"\n# important as we need to train the special tokens for the chat template of llama\nlora_modules_to_save\n: [\n\"lm_head\"\n,\n\"embed_tokens\"\n]\n# you might need to change this for qwen or other models\nlora_r\n:\n16\nlora_alpha\n:\n16\n# Training arguments\nnum_train_epochs\n:\n1\nper_device_train_batch_size\n:\n8\ngradient_accumulation_steps\n:\n2\ngradient_checkpointing\n:\ntrue\ngradient_checkpointing_kwargs\n:\nuse_reentrant\n:\nfalse\nlearning_rate\n:\n2.0e-4\nlr_scheduler_type\n:\nconstant\nwarmup_ratio\n:\n0.1\n# Logging arguments\nlogging_strategy\n:\nsteps\nlogging_steps\n:\n5\nreport_to\n:\n-\ntensorboard\nsave_strategy\n:\n\"epoch\"\nseed\n:\n42\n# Hugging Face Hub\npush_to_hub\n:\ntrue\n# hub_model_id: llama-3-1-8b-math-orca-qlora-10k-ep1 # if not defined same as output_dir\nhub_strategy\n:\nevery_save\nThis config works for single-GPU training and for multi-GPU training with DeepSpeed (see Appendix for full command). If you want to use Spectrum check the\nAppendix\nfor more information.\nPython\n!\npython scripts/run_sft.py\n--\nconfig receipes/llama-\n3\n-\n1\n-\n8b\n-qlora.yaml\nI ran several experiments with different optimization strategies, including Flash Attention, Liger Kernels, Q-Lora, and the Spectrum method to compare the time it takes to fine-tune a model. The results are summarized in the following table:\nModel\nTrain samples\nHardware\nMethod\ntrain sequence length\nper device batch size\ngradient accumulation\npacking\nFlash Attention\nLiger Kernels\nest. optimization steps\nest. train time\nLlama-3.1-8B\n10,000\n1x L4 24GB\nQ-LoRA\n1024\n1\n2\n❌\n❌\n❌\n5000\n~360 min\nLlama-3.1-8B\n10,000\n1x L4 24GB\nQ-LoRA\n1024\n2\n2\n✅\n❌\n❌\n1352\n~290 min\nLlama-3.1-8B\n10,000\n1x L4 24GB\nQ-LoRA\n1024\n2\n4\n✅\n✅\n❌\n676\n~220 min\nLlama-3.1-8B\n10,000\n1x L4 24GB\nQ-LoRA\n1024\n4\n4\n✅\n✅\n✅\n338\n~135 min\nLlama-3.1-8B\n10,000\n4x L4 24GB\nQ-LoRA\n1024\n8\n2\n✅\n✅\n✅\n84\n~33 min\nLlama-3.1-8B\n10,000\n8x L4 24GB\nQ-LoRA\n1024\n8\n2\n✅\n✅\n✅\n42\n~18 min\nLlama-3.1-8B\n10,000\n8x L4 24GB\nSpectrum (30%)\n1024\n8\n2\n✅\n✅\n✅\n42\n~21 min\nNotes:\nQ-Lora included training the embedding layer and the lm_head, as we use the Llama 3.1 chat template and in the base model the special tokens are not trained.\nFor distributed training Deepspeed (0.15.4) with ZeRO3 and Hugging Face Accelerate was used.\nSpectrum with 30% SNR layers took slightly longer than Q-Lora, but achieves 58% accuracy on GSM8K dataset, which is 4% higher than Q-Lora.\nUsing Q-LoRA only saves the trained adapter weights. If you want to use the model as standalone model, e.g. for inference you might want to merge the adapter and base model. This can be done using the following command:\nPython\n!\npython scripts/merge_adapter_weights.py\n--\npeft_model_id runs/llama-\n3\n-\n1\n-\n8b\n-math-orca-qlora-\n10k\n-ep1\n--\npush_to_hub\nTrue\n--\nrepository_id llama-\n3\n-\n1\n-\n8b\n-math-orca-qlora-\n10k\n-ep1-merged\n5. Test Model and run Inference\nAfter the training is done we want to evaluate and test our model. As we trained our model on solving math problems, we will evaluate the model on\nGSM8K\ndataset. GSM8K (Grade School Math 8K) is a dataset of 8.5K high quality linguistically diverse grade school math word problems. The dataset was created to support the task of question answering on basic mathematical problems that require multi-step reasoning.\nEvaluating Generative AI models is not a trivial task since 1 input can have multiple correct outputs. If you want to learn more about evaluating generative models, check out:\nEvaluate LLMs and RAG a practical example using Langchain and Hugging Face\n.\nEvaluate LLMs using Evaluation Harness and Hugging Face TGI/vLLM\nLLM Evaluation doesn't need to be complicated\nEvaluating Open LLMs with MixEval: The Closest Benchmark to LMSYS Chatbot Arena\nWe are going to use\nEvaluation Harness\nan open-source framework to evaluate language models on a wide range of tasks and benchmarks. The frameworks support evaluating models behind OpenAI compatible API endpoints, those can be locally or remotely. This super helpful as we can evaluate our model in the same environment we will use for production.\nWe are going to use\nText Generation Inference (TGI)\nfor testing and deploying our model. TGI is a purpose-built solution for deploying and serving Large Language Models (LLMs). TGI enables high-performance text generation using Tensor Parallelism and continous batching. If you are or want to use vLLM you can check the Appendix on how to start the inference server.\nNote: Make sure that you have enough GPU memory to run the container. Restart kernel to remove all allocated GPU memory from the notebook.\nWe will start the on 1 GPU detached. Meaning we can can continue to use the notebook while the container is running. If you have more GPUs you can change the\n--gpus\nand\n--num-shard\nflags to the number of GPUs.\nBash\n%%bash\nnum_gpus\n=\n1\nmodel_id\n=\nphilschmid/llama-3-1-8b-math-orca-spectrum-10k-ep1\n# replace with your model id\ndocker\nrun\n--name\ntgi\n--gpus\n${\nnum_gpus\n}\n-d\n-ti\n-p\n8080:80\n--shm-size=2GB\n\\\n-e\nHF_TOKEN=\n$(\ncat\n~/.cache/huggingface/token\n) \\\nghcr.io/huggingface/text-generation-inference:3.0.1\n\\\n--model-id\n${\nmodel_id\n} \\\n--num-shard\n${\nnum_gpus\n}\nOur container will now start in the background and download the model from Hugging Face Hub. We can check the logs to see the progress with\ndocker logs -f tgi\n.\nOnce our container is running we can send requests using the\nopenai\nor\nhuggingface_hub\nsdk. Here we ll use the\nopenai\nsdk to send a request to our inference server. If you don't have the\nopenai\nsdk installed you can install it using\npip install openai\n.\nPython\nfrom\nopenai\nimport\nOpenAI\n# create client\nclient =\nOpenAI\n(\nbase_url\n=\n\"http://localhost:8080/v1\"\n,\napi_key\n=\n\"-\"\n)\nsystem_message =\n\"\"\"Solve the given high school math problem by providing a clear explanation of each step leading to the final solution.\nProvide a detailed breakdown of your calculations, beginning with an explanation of the problem and describing how you derive each formula, value, or conclusion. Use logical steps that build upon one another, to arrive at the final answer in a systematic manner.\n# Steps\n1. **Understand the Problem**: Restate the given math problem and clearly identify the main question and any important given values.\n2. **Set Up**: Identify the key formulas or concepts that could help solve the problem (e.g., algebraic manipulation, geometry formulas, trigonometric identities).\n3. **Solve Step-by-Step**: Iteratively progress through each step of the math problem, justifying why each consecutive operation brings you closer to the solution.\n4. **Double Check**: If applicable, double check the work for accuracy and sense, and mention potential alternative approaches if any.\n5. **Final Answer**: Provide the numerical or algebraic solution clearly, accompanied by appropriate units if relevant.\n# Notes\n- Always clearly define any variable or term used.\n- Wherever applicable, include unit conversions or context to explain why each formula or step has been chosen.\n- Assume the level of mathematics is suitable for high school, and avoid overly advanced math techniques unless they are common at that level.\n\"\"\"\nmessages = [\n{\n\"role\"\n:\n\"system\"\n,\n\"content\"\n: system_message},\n{\n\"role\"\n:\n\"user\"\n,\n\"content\"\n:\n\"Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May?\"\n},\n]\nexpected_answer =\n\"72\"\n# Take a random sample from the dataset and remove the last message and send it to the model\nresponse = client.chat.completions.\ncreate\n(\nmodel\n=\n\"orca\"\n,\nmessages\n=messages,\nstream\n=\nFalse\n,\n# no streaming\nmax_tokens\n=\n256\n,\n)\nresponse = response.choices[\n0\n].message.content\n# Print results\nprint(\nf\n\"Query:\n\\n\n{\nmessages[\n1\n][\n'content'\n]\n}\n\"\n)\nprint(\nf\n\"Original Answer:\n\\n\n{\nexpected_answer\n}\n\"\n)\nprint(\nf\n\"Generated Answer:\n\\n\n{\nresponse\n}\n\"\n)\nAwesome that looks great! Now we can evaluate our model with the\nEvaluation Harness\n.\nNote: Make sure to change the model id to your fine-tuned model.\nPython\n!\nlm_eval\n--\nmodel local-chat-completions \\\n--\ntasks gsm8k_cot \\\n--\nmodel_args model=philschmid/llama-\n3\n-\n1\n-\n8b\n-math-orca-spectrum-\n10k\n-ep1,base_url=http://localhost:\n8080\n/v1/chat/completions,num_concurrent=\n8\n,max_retries=\n3\n,tokenized_requests=\nFalse\n\\\n--\napply_chat_template \\\n--\nfewshot_as_multiturn\nWow, 54% accuracy with only using 10k samples is pretty good! We successfully validated that our model can solve math problems. Now, don't forget to stop your container once you are done.\nPython\n!\ndocker stop tgi\n!\ndocker rm tgi\nConclusion\nThis guide provides the foundation for fine-tuning LLMs in 2025. The modular training scripts and configurations make it easy to adapt to your specific use case, whether you're training on a single GPU or scaling across multiple nodes.\nIf you encounter issues, have questions, or want to contribute improvements to the training pipeline, please open a PR on the repository.\nAppendix\nThe Appendix contains additional commands and documentation on how to run distributed training, inference and how to use Spectrum.\nDistributed Training\nDeepspeed + Q-LoRA\nNote: change the\nnum_processes\nto the number of GPUs you want to use.\nBash\naccelerate\nlaunch\n--config_file\nconfigs/accelerate_configs/deepspeed_zero3.yaml\n--num_processes\n8\nscripts/run_sft.py\n--config\nreceipes/llama-3-1-8b-qlora.yaml\nInference\nvLLM\nNote: Replace the model id with your fine-tuned model.\nBash\ndocker\nrun\n--runtime\nnvidia\n--gpus\nall\n\\\n-p\n8000:8000\n\\\nvllm/vllm-openai\n--model\nphilschmid/llama-3-1-8b-math-orca-qlora-10k-ep1-merged\nSpectrum\nSpectrum uses Signal-to-Noise Ratio (SNR) analysis to select the most useful layers for fine-tuning. It provides scripts and pre-run scanned for different models. If your model isn't scanned it will prompt you for the batch size to use for scanning. Batch size of 4 for 70b models requires 8xH100. But popular models like Llama 3.1 8B are already scanned. You can find the scanned models\nhere\n.\nThe script will generate a yaml configuration file in the\nmodel_snr_results\nwith the name of the model and the top-percent, e.g. for\nmeta-llama/Llama-3.1-8B\nand\n30\nit will generate it at\nsnr_results_meta-llama-Meta-Llama-3.1-8B_unfrozenparameters_30percent.yaml\n.\n--model-name\n: Specify the local model path or the Hugging Face repository.\n--top-percent\n: Specify the top percentage of SNR layers you want to retrieve.\nBash\n# clone spectrum\ngit\nclone\nhttps://github.com/cognitivecomputations/spectrum.git\ncd\nspectrum\n# generate yaml configuration\npython3\nspectrum.py\n--model-name\nmeta-llama/Meta-Llama-3.1-8B\n--top-percent\n30\n# Top 30% SNR layers saved to snr_results_meta-llama-Meta-Llama-3.1-8B_unfrozenparameters_30percent.yaml\ncd\n..\nAfter the yaml configuration is generated we can use it to fine-tune our model. We need to define the yaml configuration file in our train config yaml file and provide the path to the yaml file as\nspectrum_config_path\n. Take a look at\nreceipes/llama-3-1-8b-spectrum.yaml\nfor an example.\nThen we can start the training with the following command for single GPU training:\nBash\nCUDA_VISIBLE_DEVICES\n=\n0\npython\nscripts/run_sft.py\n--config\nreceipes/llama-3-1-8b-spectrum.yaml\nNote: Spectrum requires a more memory than Q-Lora. According to the paper ~30-50GB on a single GPU.\nFor multi-GPU training with FSDP and Deepspeed you can use the following command:\nBash\naccelerate\nlaunch\n--config_file\nconfigs/accelerate_configs/deepspeed_zero3.yaml\n--num_processes\n8\nscripts/run_sft.py\n--config\nreceipes/llama-3-1-8b-spectrum.yaml\nNote: Training on 8x L4 GPUs with Spectrum takes ~21 minutes. Q-Lora on the same config took 18 minutes.\nResults:\nSpectrum model trained for 1 epoch with 30% SNR layers on GSM8K dataset achieved 58% accuracy, which is 4% higher than Q-Lora.\nSpectrum model trained for 3 epochs with 30% SNR layers on GSM8K dataset achieved 60% accuracy.", "metadata": { "title": "How to fine-tune open LLMs in 2025 with Hugging Face", "description": "The only guide you need to fine-tune open LLMs in 2025, including QLoRA, Spectrum, Flash Attention, Liger Kernels and more.", "keywords": null, "author": "Philipp Schmid", "og:title": "How to fine-tune open LLMs in 2025 with Hugging Face", "og:description": "The only guide you need to fine-tune open LLMs in 2025, including QLoRA, Spectrum, Flash Attention, Liger Kernels and more.", "og:url": "https://www.philschmid.de/fine-tune-llms-in-2025", "og:image": "https://www.philschmid.de/static/blog/fine-tune-llms-in-2025/thumbnail.jpg", "og:image:width": "1200", "og:image:height": "630", "og:image:alt": "How to fine-tune open LLMs in 2025 with Hugging Face", "og:type": "article", "twitter:card": "summary_large_image", "twitter:title": "How to fine-tune open LLMs in 2025 with Hugging Face", "twitter:description": "The only guide you need to fine-tune open LLMs in 2025, including QLoRA, Spectrum, Flash Attention, Liger Kernels and more.", "twitter:image": "https://www.philschmid.de/static/blog/fine-tune-llms-in-2025/thumbnail.jpg", "article:published_time": "2024-12-20" }, "paywall": { "listed_paid": false, "detected": false, "marker": "", "public_only": false, "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙" }, "crawl": { "success": true, "status_code": 200, "error": "" }, "code_blocks": [ { "file": "attachments/3223e7064646/code/body-000.txt", "chars": 421, "language": "text" }, { "file": "attachments/3223e7064646/code/body-001.txt", "chars": 101, "language": "text" }, { "file": "attachments/3223e7064646/code/body-002.txt", "chars": 288, "language": "text" }, { "file": "attachments/3223e7064646/code/body-003.txt", "chars": 2129, "language": "text" }, { "file": "attachments/3223e7064646/code/body-004.txt", "chars": 85, "language": "text" }, { "file": "attachments/3223e7064646/code/body-005.txt", "chars": 1232, "language": "text" }, { "file": "attachments/3223e7064646/code/body-006.txt", "chars": 68, "language": "text" }, { "file": "attachments/3223e7064646/code/body-007.txt", "chars": 177, "language": "text" }, { "file": "attachments/3223e7064646/code/body-008.txt", "chars": 347, "language": "text" }, { "file": "attachments/3223e7064646/code/body-009.txt", "chars": 2263, "language": "text" }, { "file": "attachments/3223e7064646/code/body-010.txt", "chars": 295, "language": "text" }, { "file": "attachments/3223e7064646/code/body-011.txt", "chars": 31, "language": "text" }, { "file": "attachments/3223e7064646/code/body-012.txt", "chars": 158, "language": "text" }, { "file": "attachments/3223e7064646/code/body-013.txt", "chars": 144, "language": "text" }, { "file": "attachments/3223e7064646/code/body-014.txt", "chars": 312, "language": "text" }, { "file": "attachments/3223e7064646/code/body-015.txt", "chars": 93, "language": "text" }, { "file": "attachments/3223e7064646/code/body-016.txt", "chars": 161, "language": "text" } ], "images": [], "audio": [], "videos": [], "links": [ { "href": "https://github.com/philschmid/deep-learning-pytorch-huggingface/blob/main/training/fine-tune-llms-in-2025.ipynb", "label": "View Code", "type": "a" }, { "href": "https://www.philschmid.de/fine-tune-llms-in-2024-with-trl", "label": "How to Fine-Tune LLMs in 2024 with Hugging Face", "type": "a" }, { "href": "https://github.com/Dao-AILab/flash-attention", "label": "Flash Attention", "type": "a" }, { "href": "https://github.com/linkedin/Liger-Kernel", "label": "Liger Kernels", "type": "a" }, { "href": "https://huggingface.co/papers/2305.14314", "label": "QLoRA (Quantized Low-Rank Adaptation)", "type": "a" }, { "href": "https://huggingface.co/papers/2406.06623", "label": "Spectrum", "type": "a" }, { "href": "https://huggingface.co/models", "label": "Hugging Face Hub", "type": "a" }, { "href": "https://huggingface.co/join", "label": "Hugging Face", "type": "a" }, { "href": "https://distilabel.argilla.io/", "label": "Distilabel", "type": "a" }, { "href": "https://huggingface.co/datasets", "label": "Hugging Face Hub", "type": "a" }, { "href": "https://github.com/mlabonne/llm-datasets", "label": "LLM Datasets", "type": "a" }, { "href": "https://huggingface.co/datasets/microsoft/orca-math-word-problems-200k", "label": "Orca-Math", "type": "a" }, { "href": "https://huggingface.co/docs/trl/sft_trainer", "label": "SFTTrainer", "type": "a" }, { "href": "https://github.com/philschmid/deep-learning-pytorch-huggingface/blob/main/training/scripts/run_sft.py", "label": "run_sft.py", "type": "a" }, { "href": "https://github.com/axolotl-ai-cloud/axolotl", "label": "axolotl", "type": "a" }, { "href": "https://github.com/philschmid/deep-learning-pytorch-huggingface/blob/main/training/receipes/llama-3-1-8b-qlora.yaml", "label": "Llama-3.1-8B model with Q-LoRA", "type": "a" }, { "href": "https://www.philschmid.de/Appendix", "label": "Appendix", "type": "a" }, { "href": "https://huggingface.co/datasets/openai/gsm8k", "label": "GSM8K", "type": "a" }, { "href": "https://www.philschmid.de/evaluate-llm", "label": "Evaluate LLMs and RAG a practical example using Langchain and Hugging Face", "type": "a" }, { "href": "https://www.philschmid.de/evaluate-llms-with-lm-eval-and-tgi-vllm", "label": "Evaluate LLMs using Evaluation Harness and Hugging Face TGI/vLLM", "type": "a" }, { "href": "https://www.philschmid.de/llm-evaluation", "label": "LLM Evaluation doesn't need to be complicated", "type": "a" }, { "href": "https://www.philschmid.de/evaluate-llm-mixeval", "label": "Evaluating Open LLMs with MixEval: The Closest Benchmark to LMSYS Chatbot Arena", "type": "a" }, { "href": "https://github.com/EleutherAI/lm-evaluation-harness", "label": "Evaluation Harness", "type": "a" }, { "href": "https://github.com/huggingface/text-generation-inference", "label": "Text Generation Inference (TGI)", "type": "a" }, { "href": "https://github.com/cognitivecomputations/spectrum/tree/main/model_snr_results", "label": "here", "type": "a" }, { "href": "https://github.com/philschmid/deep-learning-pytorch-huggingface/blob/main/training/receipes/llama-3-1-8b-spectrum.yaml", "label": "receipes/llama-3-1-8b-spectrum.yaml", "type": "a" } ], "repair_version": 2, "repaired_at": "2026-10-14T10:52:47Z", "source_url": "https://www.philschmid.de/fine-tune-llms-in-2025", "papers": [], "actual_video_files": [], "body_chars": 23816, "expected_body_images": 0, "discovery_status": "ok", "delivery_status": "ok", "missing_reason": "", "raw_html": "attachments/3223e7064646/pages/source.html", "local_html": "attachments/3223e7064646/pages/article.html", "body_base_path": "attachments/3223e7064646/pages/", "body_markdown_path": "attachments/3223e7064646/pages/article.md" }