0
0
ML Spring
Posts
Understanding LoRA: Low-rank Adaption of Large Language Models
Understanding LoRA: Low-rank Adaption of Large Language Models
The most powerful technique for finetuning LLMs! 🚀
Akshay Pachaar
November 13, 2023
LoRA (Low-Rank Adaptation) is one of the most powerful techniques when it comes to Fine-Tuning Large Language Models(LLMs).
Today I’ll clearly explain:
What is Lora❓
How does it works ❓
Followed by a hands-on coding tutorial❗️
But before we do that, let's set the stage with a brief overview of Fine-Tuning.
Fine-Tuning: The Traditional approach
Fine-tuning is a well-established method in machine learning where a pre-trained model is further trained (or "fine-tuned") on a specific task. This approach leverages the general knowledge the model has learned during its initial training (often on a large and diverse dataset) and adapts it to a particular use case.
Here’s a typical representation of traditional Fine-Tuning 👇
Subscribe to keep reading
This content is free, but you must be subscribed to ML Spring to continue reading.
Already a subscriber?
Sign in
.
Not now
Reply
Newest first
Add your comment
Login
Login
or
Subscribe
to participate
.