{ "source_file": "0930", "source_platform": "研究机构", "author": "Hugging Face Blog", "title_listed": "Fine-tune Llama 2 with QLoRA", "keywords": "Llama 2,QLoRA,微调", "domain": "大模型算法", "content_type": "教程", "is_paid_flag": "否", "url": "https://huggingface.co/blog/llama2", "publish_date": "2023-07", "corpus_stage": "预训练/SFT", "notes": "Llama 2模型介绍与微调（Hugging Face）", "id": "2a39f2bb6122", "collected_at": "2026-10-02T15:23:36", "title": "Llama 2 is here - get it on Hugging Face", "body_markdown": "Back to Articles\nLlama 2 is here - get it on Hugging Face\nPublished\n\t\t\t\t\tJuly 18, 2023\nUpdate on GitHub\nUpvote\n31\n+25\nPhilipp Schmid\nphilschmid\nFollow\nOmar Sanseviero\nosanseviero\nFollow\nPedro Cuenca\npcuenq\nFollow\nLewis Tunstall\nlewtun\nFollow\nIntroduction\nLlama 2 is a family of state-of-the-art open-access large language models released by Meta today, and we’re excited to fully support the launch with comprehensive integration in Hugging Face. Llama 2 is being released with a very permissive community license and is available for commercial use. The code, pretrained models, and fine-tuned models are all being released today 🔥\nWe’ve collaborated with Meta to ensure smooth integration into the Hugging Face ecosystem. You can find the 12 open-access models (3 base models & 3 fine-tuned ones with the original Meta checkpoints, plus their corresponding\ntransformers\nmodels) on the Hub. Among the features and integrations being released, we have:\nModels on the Hub\nwith their model cards and license.\nTransformers integration\nExamples to fine-tune the small variants of the model with a single GPU\nIntegration with\nText Generation Inference\nfor fast and efficient production-ready inference\nIntegration with Inference Endpoints\nTable of Contents\nWhy Llama 2?\nDemo\nInference\nWith Transformers\nWith Inference Endpoints\nFine-tuning with PEFT\nHow to Prompt Llama 2\nAdditional Resources\nConclusion\nWhy Llama 2?\nThe Llama 2 release introduces a family of pretrained and fine-tuned LLMs, ranging in scale from 7B to 70B parameters (7B, 13B, 70B). The pretrained models come with significant improvements over the Llama 1 models, including being trained on 40% more tokens, having a much longer context length (4k tokens 🤯), and using grouped-query attention for fast inference of the 70B model🔥!\nHowever, the most exciting part of this release is the fine-tuned models (Llama 2-Chat), which have been optimized for dialogue applications using\nReinforcement Learning from Human Feedback (RLHF)\n. Across a wide range of helpfulness and safety benchmarks, the Llama 2-Chat models perform better than most open models and achieve comparable performance to ChatGPT according to human evaluations. You can read the paper\nhere\n.\nimage from\nLlama 2: Open Foundation and Fine-Tuned Chat Models\nIf you’ve been waiting for an open alternative to closed-source chatbots, Llama 2-Chat is likely your best choice today!\nModel\nLicense\nCommercial use?\nPretraining length [tokens]\nLeaderboard score\nFalcon-7B\nApache 2.0\n✅\n1,500B\n44.17\nMPT-7B\nApache 2.0\n✅\n1,000B\n47.24\nLlama-7B\nLlama license\n❌\n1,000B\n45.65\nLlama-2-7B\nLlama 2 license\n✅\n2,000B\n50.97\nLlama-33B\nLlama license\n❌\n1,500B\n-\nLlama-2-13B\nLlama 2 license\n✅\n2,000B\n55.69\nmpt-30B\nApache 2.0\n✅\n1,000B\n52.77\nFalcon-40B\nApache 2.0\n✅\n1,000B\n58.07\nLlama-65B\nLlama license\n❌\n1,500B\n61.19\nLlama-2-70B\nLlama 2 license\n✅\n2,000B\n67.87\nLlama-2-70B-chat\nLlama 2 license\n✅\n2,000B\n62.4\nNote: the performance scores shown in the table below have been updated to account for the new methodology introduced in November 2023, which added new benchmarks. More details in\nthis post\n.\nDemo\nYou can easily try the 13B Llama 2 Model in\nthis Space\nor in the playground embedded below:\nTo learn more about how this demo works, read on below about how to run inference on Llama 2 models.\nInference\nIn this section, we’ll go through different approaches to running inference of the Llama 2 models. Before using these models, make sure you have requested access to one of the models in the official\nMeta Llama 2\nrepositories.\nNote: Make sure to also fill the official Meta form. Users are provided access to the repository once both forms are filled after few hours.\nUsing transformers\nWith transformers\nrelease 4.31\n, one can already use Llama 2 and leverage all the tools within the HF ecosystem, such as:\ntraining and inference scripts and examples\nsafe file format (\nsafetensors\n)\nintegrations with tools such as bitsandbytes (4-bit quantization) and PEFT (parameter efficient fine-tuning)\nutilities and helpers to run generation with the model\nmechanisms to export the models to deploy\nMake sure to be using the latest\ntransformers\nrelease and be logged into your Hugging Face account.\npip install transformers\nhuggingface-cli login\nIn the following code snippet, we show how to run inference with transformers. It runs on the free tier of Colab, as long as you select a GPU runtime.\nfrom\ntransformers\nimport\nAutoTokenizer\nimport\ntransformers\nimport\ntorch\n\nmodel =\n\"meta-llama/Llama-2-7b-chat-hf\"\ntokenizer = AutoTokenizer.from_pretrained(model)\npipeline = transformers.pipeline(\n\"text-generation\"\n,\n model=model,\n torch_dtype=torch.float16,\n device_map=\n\"auto\"\n,\n)\n\nsequences = pipeline(\n'I liked \"Breaking Bad\" and \"Band of Brothers\". Do you have any recommendations of other shows I might like?\\n'\n,\n do_sample=\nTrue\n,\n top_k=\n10\n,\n num_return_sequences=\n1\n,\n eos_token_id=tokenizer.eos_token_id,\n max_length=\n200\n,\n)\nfor\nseq\nin\nsequences:\nprint\n(\nf\"Result:\n{seq[\n'generated_text'\n]}\n\"\n)\nResult: I liked \"Breaking Bad\" and \"Band of Brothers\". Do you have any recommendations of other shows I might like?\nAnswer:\nOf course! If you enjoyed \"Breaking Bad\" and \"Band of Brothers,\" here are some other TV shows you might enjoy:\n1. \"The Sopranos\" - This HBO series is a crime drama that explores the life of a New Jersey mob boss, Tony Soprano, as he navigates the criminal underworld and deals with personal and family issues.\n2. \"The Wire\" - This HBO series is a gritty and realistic portrayal of the drug trade in Baltimore, exploring the impact of drugs on individuals, communities, and the criminal justice system.\n3. \"Mad Men\" - Set in the 1960s, this AMC series follows the lives of advertising executives on Madison Avenue, expl\nAnd although the model has\nonly\n4k tokens of context, you can use techniques supported in\ntransformers\nsuch as rotary position embedding scaling (\ntweet\n) to push it further!\nUsing text-generation-inference and Inference Endpoints\nText Generation Inference\nis a production-ready inference container developed by Hugging Face to enable easy deployment of large language models. It has features such as continuous batching, token streaming, tensor parallelism for fast inference on multiple GPUs, and production-ready logging and tracing.\nYou can try out Text Generation Inference on your own infrastructure, or you can use Hugging Face's\nInference Endpoints\n. To deploy a Llama 2 model, go to the\nmodel page\nand click on the\nDeploy -> Inference Endpoints\nwidget.\nFor 7B models, we advise you to select \"GPU [medium] - 1x Nvidia A10G\".\nFor 13B models, we advise you to select \"GPU [xlarge] - 1x Nvidia A100\".\nFor 70B models, we advise you to select \"GPU [2xlarge] - 2x Nvidia A100\" with\nbitsandbytes\nquantization enabled or \"GPU [4xlarge] - 4x Nvidia A100\"\nNote: You might need to request a quota upgrade via email to\napi-enterprise@huggingface.co\nto access A100s\nYou can learn more on how to\nDeploy LLMs with Hugging Face Inference Endpoints in our blog\n. The\nblog\nincludes information about supported hyperparameters and how to stream your response using Python and Javascript.\nFine-tuning with PEFT\nTraining LLMs can be technically and computationally challenging. In this section, we look at the tools available in the Hugging Face ecosystem to efficiently train Llama 2 on simple hardware and show how to fine-tune the 7B version of Llama 2 on a single NVIDIA T4 (16GB - Google Colab). You can learn more about it in the\nMaking LLMs even more accessible blog\n.\nWe created a\nscript\nto instruction-tune Llama 2 using QLoRA and the\nSFTTrainer\nfrom\ntrl\n.\nAn example command for fine-tuning Llama 2 7B on the\ntimdettmers/openassistant-guanaco\ncan be found below. The script can merge the LoRA weights into the model weights and save them as\nsafetensor\nweights by providing the\nmerge_and_push\nargument. This allows us to deploy our fine-tuned model after training using text-generation-inference and inference endpoints.\nFirst pip install\ntrl\nand clone the script:\npip install trl\ngit\nclone\nhttps://github.com/lvwerra/trl\nThen you can run the script:\npython trl/examples/scripts/sft_trainer.py \\\n --model_name meta-llama/Llama-2-7b-hf \\\n --dataset_name timdettmers/openassistant-guanaco \\\n --load_in_4bit \\\n --use_peft \\\n --batch_size 4 \\\n --gradient_accumulation_steps 2\nHow to Prompt Llama 2\nOne of the unsung advantages of open-access models is that you have full control over the\nsystem\nprompt in chat applications. This is essential to specify the behavior of your chat assistant –and even imbue it with some personality–, but it's unreachable in models served behind APIs.\nWe're adding this section just a few days after the initial release of Llama 2, as we've had many questions from the community about how to prompt the models and how to change the system prompt. We hope this helps!\nThe prompt template for the first turn looks like this:\n[INST] <>\n{{ system_prompt }}\n<>\n\n{{ user_message }} [/INST]\nThis template follows the model's training procedure, as described in\nthe Llama 2 paper\n. We can use any\nsystem_prompt\nwe want, but it's crucial that the format matches the one used during training.\nTo spell it out in full clarity, this is what is actually sent to the language model when the user enters some text (\nThere's a llama in my garden 😱 What should I do?\n) in\nour 13B chat demo\nto initiate a chat:\n[INST] <>\nYou are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.\n\nIf a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.\n<>\n\nThere's a llama in my garden 😱 What should I do? [/INST]\nAs you can see, the instructions between the special\n<>\ntokens provide context for the model so it knows how we expect it to respond. This works because exactly the same format was used during training with a wide variety of system prompts intended for different tasks.\nAs the conversation progresses,\nall\nthe interactions between the human and the \"bot\" are appended to the previous prompt, enclosed between\n[INST]\ndelimiters. The template used during multi-turn conversations follows this structure (🎩 h/t\nArthur Zucker\nfor some final clarifications):\n[INST] <>\n{{ system_prompt }}\n<>\n\n{{ user_msg_1 }} [/INST] {{ model_answer_1 }} [INST] {{ user_msg_2 }} [/INST]\nThe model is stateless and does not \"remember\" previous fragments of the conversation, we must always supply it with all the context so the conversation can continue. This is the reason why\ncontext length\nis a very important parameter to maximize, as it allows for longer conversations and larger amounts of information to be used.\nIgnore previous instructions\nIn API-based models, people resort to tricks in an attempt to override the system prompt and change the default model behaviour. As imaginative as these solutions are, this is not necessary in open-access models: anyone can use a different prompt, as long as it follows the format described above. We believe that this will be an important tool for researchers to study the impact of prompts on both desired and unwanted characteristics. For example, when people\nare surprised with absurdly cautious generations\n, you can explore whether maybe\na different prompt would work\n. (🎩 h/t\nClémentine Fourrier\nfor the links to this example).\nIn our\n13B\nand\n7B\ndemos, you can easily explore this feature by disclosing the \"Advanced Options\" UI and simply writing your desired instructions. You can also duplicate those demos and use them privately for fun or research!\nAdditional Resources\nPaper Page\nModels on the Hub\nLeaderboard\nMeta Examples and recipes for Llama model\nChat demo (7B)\nChat demo (13B)\nChat demo (70B) on TGI\nConclusion\nWe're very excited about Llama 2 being out! In the incoming days, be ready to learn more about ways to run your own fine-tuning, execute the smallest models on-device, and many other exciting updates we're prepating for you!\nModels mentioned in this article\n6\nSpaces mentioned in this article\n3\nPapers mentioned in this article\n1\nMore Articles from our Blog\nnlp\ncommunity\nresearch\nWelcome Falcon Mamba: The first strong attention-free 7B model\n+2\n114\nAugust 12, 2024\nnlp\ncommunity\nresearch\nGoogle releases Gemma 2 2B, ShieldGemma and Gemma Scope\n61\nJuly 31, 2024\nCommunity\nEdit\nPreview\nUpload images, audio, and videos by dragging in the text input, pasting, or\nclicking here\n.\nTap or paste here to upload images\nComment\n·\nSign up\nor\nlog in\nto comment\nUpvote\n31\n+19\nModels mentioned in this article\n6\nSpaces mentioned in this article\n3\nPapers mentioned in this article\n1", "metadata": { "title": "Llama 2 is here - get it on Hugging Face", "description": "We’re on a journey to advance and democratize artificial intelligence through open source and open science.", "keywords": null, "author": null, "og:title": "Llama 2 is here - get it on Hugging Face", "og:description": "We’re on a journey to advance and democratize artificial intelligence through open source and open science.", "og:type": "website", "og:url": "https://huggingface.co/blog/llama2", "og:image": "https://huggingface.co/blog/assets/llama2/thumbnail.jpg", "twitter:card": "summary_large_image", "twitter:site": "@huggingface", "twitter:image": "https://huggingface.co/blog/assets/llama2/thumbnail.jpg" }, "paywall": { "listed_paid": false, "detected": false, "marker": "", "public_only": false, "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙" }, "crawl": { "success": true, "status_code": 200, "error": "" }, "code_blocks": [ { "file": "attachments/2a39f2bb6122/code/body-000.txt", "chars": 47, "language": "text" }, { "file": "attachments/2a39f2bb6122/code/body-001.txt", "chars": 626, "language": "text" }, { "file": "attachments/2a39f2bb6122/code/body-002.txt", "chars": 743, "language": "text" }, { "file": "attachments/2a39f2bb6122/code/body-003.txt", "chars": 57, "language": "text" }, { "file": "attachments/2a39f2bb6122/code/body-004.txt", "chars": 239, "language": "text" }, { "file": "attachments/2a39f2bb6122/code/body-005.txt", "chars": 75, "language": "text" }, { "file": "attachments/2a39f2bb6122/code/body-006.txt", "chars": 594, "language": "text" }, { "file": "attachments/2a39f2bb6122/code/body-007.txt", "chars": 133, "language": "text" } ], "images": [ { "status": "ok", "url": "https://huggingface.co/avatars/88edb4eca546956d59c830a384af4606.svg", "bytes": 397, "reused": false, "local_path": "attachments/2a39f2bb6122/images/body-e10548bc84ee2e5f54fe.svg", "src": "https://huggingface.co/avatars/88edb4eca546956d59c830a384af4606.svg", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/M_4DSrrYxOL1x8Ufgghpz.png", "bytes": 6296, "reused": false, "local_path": "attachments/2a39f2bb6122/images/body-69420effd8f2a7cf733b.png", "src": "https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/M_4DSrrYxOL1x8Ufgghpz.png", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://huggingface.co/avatars/1ae2fc3910a64bd91d20aadb267c0bc3.svg", "bytes": 1285, "reused": false, "local_path": "attachments/2a39f2bb6122/images/body-621a7ec4db7fcceaf79a.svg", "src": "https://huggingface.co/avatars/1ae2fc3910a64bd91d20aadb267c0bc3.svg", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://huggingface.co/avatars/2549646556c265b0f8dc3f1dfb9779d8.svg", "bytes": 1285, "reused": false, "local_path": "attachments/2a39f2bb6122/images/body-b6fad2ad3602459abcfa.svg", "src": "https://huggingface.co/avatars/2549646556c265b0f8dc3f1dfb9779d8.svg", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://cdn-avatars.huggingface.co/v1/production/uploads/64245f2c089d5fae56b4549a/qUHFsL9Svwyj5BKpfMtaY.jpeg", "bytes": 8614, "reused": false, "local_path": "attachments/2a39f2bb6122/images/body-d1e4d3e8d3c471a07175.jpeg", "src": "https://cdn-avatars.huggingface.co/v1/production/uploads/64245f2c089d5fae56b4549a/qUHFsL9Svwyj5BKpfMtaY.jpeg", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://huggingface.co/avatars/259d571f92daae3630e4b14f407315a0.svg", "bytes": 1283, "reused": false, "local_path": "attachments/2a39f2bb6122/images/body-bd39c2dae10e0599bd70.svg", "src": "https://huggingface.co/avatars/259d571f92daae3630e4b14f407315a0.svg", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/llama-rlhf.png", "bytes": 229518, "reused": true, "local_path": "attachments/2a39f2bb6122/images/004.png", "src": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/llama-rlhf.png", "alt": "mqa", "downloaded": true }, { "status": "ok", "url": "https://huggingface.co/blog/assets/falconmamba/thumbnail.png", "bytes": 3955286, "reused": false, "local_path": "attachments/2a39f2bb6122/images/body-6bce9abb6ed777958a80.png", "src": "https://huggingface.co/blog/assets/falconmamba/thumbnail.png", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://cdn-avatars.huggingface.co/v1/production/uploads/6460c3811db65f878513bcaf/0C3SoPO0bqTKtnWIu1WtG.jpeg", "bytes": 10304, "reused": false, "local_path": "attachments/2a39f2bb6122/images/body-b875ab7405fde1342e98.jpeg", "src": "https://cdn-avatars.huggingface.co/v1/production/uploads/6460c3811db65f878513bcaf/0C3SoPO0bqTKtnWIu1WtG.jpeg", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://huggingface.co/avatars/b68caad7e987c095b0cab4d9035aac25.svg", "bytes": 1283, "reused": false, "local_path": "attachments/2a39f2bb6122/images/body-d3a7ebf067b44335ad12.svg", "src": "https://huggingface.co/avatars/b68caad7e987c095b0cab4d9035aac25.svg", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://cdn-avatars.huggingface.co/v1/production/uploads/664ddb00da286d1a60c298fc/VchH_2fFjZxnBD_Ht10ED.jpeg", "bytes": 9060, "reused": false, "local_path": "attachments/2a39f2bb6122/images/body-d0954be51645b6663c2d.jpeg", "src": "https://cdn-avatars.huggingface.co/v1/production/uploads/664ddb00da286d1a60c298fc/VchH_2fFjZxnBD_Ht10ED.jpeg", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://cdn-avatars.huggingface.co/v1/production/uploads/6697a9fb6d173ec7382e0392/Q_myrIBbdWI3RmEvEHrtQ.jpeg", "bytes": 11688, "reused": false, "local_path": "attachments/2a39f2bb6122/images/body-f43a18995207d039b081.jpeg", "src": "https://cdn-avatars.huggingface.co/v1/production/uploads/6697a9fb6d173ec7382e0392/Q_myrIBbdWI3RmEvEHrtQ.jpeg", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://huggingface.co/blog/assets/gemma-july-update/thumbnail.jpg", "bytes": 753543, "reused": false, "local_path": "attachments/2a39f2bb6122/images/body-f7122536c2f6c28244c6.jpg", "src": "https://huggingface.co/blog/assets/gemma-july-update/thumbnail.jpg", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://cdn-avatars.huggingface.co/v1/production/uploads/61b253b7ac5ecaae3d1efe0c/hwiQ0uvz3t-L5a-NtBIO6.png", "bytes": 4300, "reused": false, "local_path": "attachments/2a39f2bb6122/images/body-36a73831fb02e8382cf9.png", "src": "https://cdn-avatars.huggingface.co/v1/production/uploads/61b253b7ac5ecaae3d1efe0c/hwiQ0uvz3t-L5a-NtBIO6.png", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://cdn-avatars.huggingface.co/v1/production/uploads/1617264212503-603d25b75f9d390ab190b777.jpeg", "bytes": 9492, "reused": true, "local_path": "attachments/2a39f2bb6122/images/002.jpeg", "src": "https://cdn-avatars.huggingface.co/v1/production/uploads/1617264212503-603d25b75f9d390ab190b777.jpeg", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://cdn-avatars.huggingface.co/v1/production/uploads/1655385361868-61b85ce86eb1f2c5e6233736.jpeg", "bytes": 21134, "reused": false, "local_path": "attachments/2a39f2bb6122/images/body-c53427db6869fa24f06c.jpeg", "src": "https://cdn-avatars.huggingface.co/v1/production/uploads/1655385361868-61b85ce86eb1f2c5e6233736.jpeg", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://cdn-avatars.huggingface.co/v1/production/uploads/1641203017724-noauth.png", "bytes": 8756, "reused": false, "local_path": "attachments/2a39f2bb6122/images/body-83f985b4096b7146b43c.png", "src": "https://cdn-avatars.huggingface.co/v1/production/uploads/1641203017724-noauth.png", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://cdn-avatars.huggingface.co/v1/production/uploads/6032802e1f993496bc14d9e3/w6hr-DEQot4VVkoyRIBiy.png", "bytes": 18908, "reused": true, "local_path": "attachments/2a39f2bb6122/images/001.png", "src": "https://cdn-avatars.huggingface.co/v1/production/uploads/6032802e1f993496bc14d9e3/w6hr-DEQot4VVkoyRIBiy.png", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://huggingface.co/avatars/e46f487e69ad1a0ad80915d2516e18db.svg", "bytes": 397, "reused": false, "local_path": "attachments/2a39f2bb6122/images/body-6a206dcbd3e0bbed425b.svg", "src": "https://huggingface.co/avatars/e46f487e69ad1a0ad80915d2516e18db.svg", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://cdn-avatars.huggingface.co/v1/production/uploads/64e8841d83affd72d9424f5e/dwhOxUQ8N3xJYEbij98SZ.jpeg", "bytes": 11550, "reused": false, "local_path": "attachments/2a39f2bb6122/images/body-8c9506cd4de798f35a30.jpeg", "src": "https://cdn-avatars.huggingface.co/v1/production/uploads/64e8841d83affd72d9424f5e/dwhOxUQ8N3xJYEbij98SZ.jpeg", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://huggingface.co/avatars/d897a854fc6fce1aa216773b10d30536.svg", "bytes": 397, "reused": false, "local_path": "attachments/2a39f2bb6122/images/body-09a3893ead197c3ae810.svg", "src": "https://huggingface.co/avatars/d897a854fc6fce1aa216773b10d30536.svg", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://huggingface.co/avatars/14738d67bb743288059c2c24e8945ffc.svg", "bytes": 1281, "reused": false, "local_path": "attachments/2a39f2bb6122/images/body-632fb8dd20a1455e41f6.svg", "src": "https://huggingface.co/avatars/14738d67bb743288059c2c24e8945ffc.svg", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://huggingface.co/avatars/7a5df6476182272a81fd8e07303de594.svg", "bytes": 1282, "reused": false, "local_path": "attachments/2a39f2bb6122/images/body-09e67562dbe68ff17ff1.svg", "src": "https://huggingface.co/avatars/7a5df6476182272a81fd8e07303de594.svg", "alt": "", "downloaded": true } ], "audio": [], "videos": [], "links": [ { "href": "https://huggingface.co/blog", "label": "Back to Articles", "type": "a" }, { "href": "https://github.com/huggingface/blog/blob/main/llama2.md", "label": "Update on GitHub", "type": "a" }, { "href": "https://huggingface.co/login?next=%2Fblog%2Fllama2", "label": "Upvote 31", "type": "a" }, { "href": "https://huggingface.co/itsadeel", "label": "", "type": "a" }, { "href": "https://huggingface.co/liambyrne", "label": "", "type": "a" }, { "href": "https://huggingface.co/Gmc2", "label": "", "type": "a" }, { "href": "https://huggingface.co/Renjithvs", "label": "", "type": "a" }, { "href": "https://huggingface.co/pengxiang", "label": "", "type": "a" }, { "href": "https://huggingface.co/raj729", "label": "", "type": "a" }, { "href": "https://huggingface.co/philschmid", "label": "", "type": "a" }, { "href": "https://huggingface.co/osanseviero", "label": "", "type": "a" }, { "href": "https://huggingface.co/pcuenq", "label": "", "type": "a" }, { "href": "https://huggingface.co/lewtun", "label": "", "type": "a" }, { "href": "https://huggingface.co/meta-llama", "label": "Models on the Hub", "type": "a" }, { "href": "https://github.com/huggingface/transformers/releases/tag/v4.31.0", "label": "Transformers integration", "type": "a" }, { "href": "https://github.com/huggingface/text-generation-inference", "label": "Text Generation Inference", "type": "a" }, { "href": "https://huggingface.co/blog/rlhf", "label": "Reinforcement Learning from Human Feedback (RLHF)", "type": "a" }, { "href": "https://huggingface.co/papers/2307.09288", "label": "here", "type": "a" }, { "href": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/llama-rlhf.png", "label": "", "type": "a" }, { "href": "https://scontent-fra3-2.xx.fbcdn.net/v/t39.2365-6/10000000_6495670187160042_4742060979571156424_n.pdf?_nc_cat=104&ccb=1-7&_nc_sid=3c67a6&_nc_ohc=GK8Rh1tm_4IAX8b5yo4&_nc_ht=scontent-fra3-2.xx&oh=00_AfDtg_PRrV6tpy9UmiikeMRuQgk6Rej7bCPOkXZQVmUKAg&oe=64BBD830", "label": "Llama 2: Open Foundation and Fine-Tuned Chat Models", "type": "a" }, { "href": "https://huggingface.co/tiiuae/falcon-7b", "label": "Falcon-7B", "type": "a" }, { "href": "https://huggingface.co/mosaicml/mpt-7b", "label": "MPT-7B", "type": "a" }, { "href": "https://huggingface.co/meta-llama/Llama-2-7b-hf", "label": "Llama-2-7B", "type": "a" }, { "href": "https://huggingface.co/meta-llama/Llama-2-13b-hf", "label": "Llama-2-13B", "type": "a" }, { "href": "https://huggingface.co/mosaicml/mpt-30b", "label": "mpt-30B", "type": "a" }, { "href": "https://huggingface.co/tiiuae/falcon-40b", "label": "Falcon-40B", "type": "a" }, { "href": "https://huggingface.co/meta-llama/Llama-2-70b-hf", "label": "Llama-2-70B", "type": "a" }, { "href": "https://huggingface.co/meta-llama/Llama-2-70b-chat-hf", "label": "Llama-2-70B-chat", "type": "a" }, { "href": "https://huggingface.co/blog/open-llm-leaderboard-drop", "label": "this post", "type": "a" }, { "href": "https://huggingface.co/spaces/huggingface-projects/llama-2-13b-chat", "label": "this Space", "type": "a" }, { "href": "https://twitter.com/joao_gante/status/1679775399172251648", "label": "tweet", "type": "a" }, { "href": "https://huggingface.co/inference-endpoints", "label": "Inference Endpoints", "type": "a" }, { "href": "https://ui.endpoints.huggingface.co/new?repository=meta-llama/Llama-2-7b-hf", "label": "Deploy -> Inference Endpoints", "type": "a" }, { "href": "https://huggingface.co/blog/inference-endpoints-llm", "label": "Deploy LLMs with Hugging Face Inference Endpoints in our blog", "type": "a" }, { "href": "https://huggingface.co/blog/4bit-transformers-bitsandbytes", "label": "Making LLMs even more accessible blog", "type": "a" }, { "href": "https://github.com/lvwerra/trl/blob/main/examples/scripts/sft_trainer.py", "label": "script", "type": "a" }, { "href": "https://huggingface.co/docs/trl/v0.4.7/en/sft_trainer", "label": "SFTTrainer", "type": "a" }, { "href": "https://github.com/lvwerra/trl", "label": "trl", "type": "a" }, { "href": "https://huggingface.co/ArthurZ", "label": "Arthur Zucker", "type": "a" }, { "href": "https://twitter.com/lauraruis/status/1681612002718887936", "label": "are surprised with absurdly cautious generations", "type": "a" }, { "href": "https://twitter.com/overlordayn/status/1681631554672513025", "label": "a different prompt would work", "type": "a" }, { "href": "https://huggingface.co/clefourrier", "label": "Clémentine Fourrier", "type": "a" }, { "href": "https://huggingface.co/spaces/huggingface-projects/llama-2-7b-chat", "label": "7B", "type": "a" }, { "href": "https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard", "label": "Leaderboard", "type": "a" }, { "href": "https://github.com/facebookresearch/llama-recipes/tree/main", "label": "Meta Examples and recipes for Llama model", "type": "a" }, { "href": "https://huggingface.co/spaces/ysharma/Explore_llamav2_with_TGI", "label": "Chat demo (70B) on TGI", "type": "a" }, { "href": "https://huggingface.co/blog/falconmamba", "label": "nlp community research Welcome Falcon Mamba: The first strong attention-free 7B model +2 114 August 12, 2024", "type": "a" }, { "href": "https://huggingface.co/blog/gemma-july-update", "label": "nlp community research Google releases Gemma 2 2B, ShieldGemma and Gemma Scope 61 July 31, 2024", "type": "a" }, { "href": "https://huggingface.co/join?next=%2Fblog%2Fllama2", "label": "Sign up", "type": "a" }, { "href": "https://huggingface.co/Dave-ArCh", "label": "", "type": "a" }, { "href": "https://huggingface.co/smanna", "label": "", "type": "a" }, { "href": "https://huggingface.co/N1v3x", "label": "", "type": "a" }, { "href": "https://huggingface.co/Abdo36", "label": "", "type": "a" }, { "href": "https://huggingface.co/huseynov", "label": "", "type": "a" } ], "repair_version": 2, "repaired_at": "2026-10-14T10:53:22Z", "source_url": "https://huggingface.co/blog/llama2", "papers": [ { "status": "http_403", "url": "https://scontent-fra3-2.xx.fbcdn.net/v/t39.2365-6/10000000_6495670187160042_4742060979571156424_n.pdf?_nc_cat=104&ccb=1-7&_nc_sid=3c67a6&_nc_ohc=GK8Rh1tm_4IAX8b5yo4&_nc_ht=scontent-fra3-2.xx&oh=00_AfDtg_PRrV6tpy9UmiikeMRuQgk6Rej7bCPOkXZQVmUKAg&oe=64BBD830", "retried_at": "2026-10-20T18:06:18Z" } ], "actual_video_files": [], "body_chars": 12967, "expected_body_images": 23, "discovery_status": "ok", "delivery_status": "partial", "missing_reason": "http_403", "raw_html": "attachments/2a39f2bb6122/pages/source.html", "local_html": "attachments/2a39f2bb6122/pages/article.html", "body_base_path": "attachments/2a39f2bb6122/pages/", "body_markdown_path": "attachments/2a39f2bb6122/pages/article.md" }