{
"source_file": "0930",
"source_platform": "Medium",
"author": "Ivo Bernardo",
"title_listed": "Setting Up Your Own Large Language Model",
"keywords": "",
"domain": "大模型算法",
"content_type": "深度技术解析",
"is_paid_flag": "否",
"url": "https://towardsdatascience.com/setting-up-your-own-large-language-model/",
"publish_date": "2026-07-04",
"corpus_stage": "",
"notes": "",
"id": "406b5167357c",
"collected_at": "2026-10-02T05:33:18",
"title": "Setting Up Your Own Large Language Model | Towards Data Science",
"body_markdown": "\nWe value your privacy\nWe use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking \"Accept All\", you consent to our use of cookies.\nCustomise Reject All Accept All\nCustomise Consent Preferences \nWe use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below.\nThe cookies that are categorised as \"Necessary\" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... Show more\nNecessaryAlways Active\nNecessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.\n * Cookie\nBCTempID\n * Duration\n10 minutes\n * Description\nNo description available.\n\n\n * Cookie\n__cf_bm\n * Duration\n1 hour\n * Description\nThis cookie, set by Cloudflare, is used to support Cloudflare Bot Management. \n\n\n * Cookie\nAWSALBCORS\n * Duration\n7 days\n * Description\nAmazon Web Services set this cookie for load balancing.\n\n\n * Cookie\n_cfuvid\n * Duration\nsession\n * Description\nCloudflare sets this cookie to track users across sessions to optimize user experience by maintaining session consistency and providing personalized services\n\n\n * Cookie\nli_gc\n * Duration\n6 months\n * Description\nLinkedin set this cookie for storing visitor's consent regarding using cookies for non-essential purposes.\n\n\n * Cookie\n__hssrc\n * Duration\nsession\n * Description\nThis cookie is set by Hubspot whenever it changes the session cookie. The __hssrc cookie set to 1 indicates that the user has restarted the browser, and if the cookie does not exist, it is assumed to be a new session.\n\n\n * Cookie\n__hssc\n * Duration\n1 hour\n * Description\nHubSpot sets this cookie to keep track of sessions and to determine if HubSpot should increment the session number and timestamps in the __hstc cookie.\n\n\n * Cookie\nwpEmojiSettingsSupports\n * Duration\nsession\n * Description\nWordPress sets this cookie when a user interacts with emojis on a WordPress site. It helps determine if the user's browser can display emojis properly.\n\n\n * Cookie\nBCSessionID\n * Duration\n1 year 1 month 4 days\n * Description\nBlueconic sets this cookie as a unique identifier for the BlueConic profile.\n\n\n * Cookie\n_octo\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nlogged_in\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\n__Secure-YEC\n * Duration\npast\n * Description\nYouTube sets this cookie to stores the user's video player preferences using embedded YouTube video\n\n\n * Cookie\n__eoi\n * Duration\n6 months\n * Description\nDescription is currently not available.\n\n\n * Cookie\nAWSALBTGCORS\n * Duration\n7 days\n * Description\nNo description available.\n\n\n * Cookie\nlogin-status-p\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nAWSALBTG\n * Duration\n7 days\n * Description\nNo description available.\n\n\n * Cookie\ncsrf_token\n * Duration\nsession\n * Description\nNo description available.\n\n\n * Cookie\ntoken_v2\n * Duration\n1 day\n * Description\nDescription is currently not available.\n\n\n * Cookie\nD\n * Duration\n1 year\n * Description\nDescription is currently not available.\n\n\n * Cookie\nPHPSESSID\n * Duration\nsession\n * Description\nThis cookie is native to PHP applications. The cookie stores and identifies a user's unique session ID to manage user sessions on the website. The cookie is a session cookie and will be deleted when all the browser windows are closed.\n\n\n * Cookie\nVISITOR_PRIVACY_METADATA\n * Duration\n6 months\n * Description\nYouTube sets this cookie to store the user's cookie consent state for the current domain. \n\n\n * Cookie\ncookietest\n * Duration\nsession\n * Description\nThe cookietest cookie is typically used to determine whether the user's browser accepts cookies, essential for website functionality and user experience.\n\n\n * Cookie\n__Host-airtable-session\n * Duration\n1 year\n * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n * Cookie\n__Host-airtable-session.sig\n * Duration\n1 year\n * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n * Cookie\nm\n * Duration\n1 year 1 month 4 days\n * Description\nStripe sets this cookie for fraud prevention purposes. It identifies the device used to access the website, allowing the website to be formatted accordingly.\n\n\n * Cookie\nBIGipServer*\n * Duration\nsession\n * Description\nMarketo sets this cookie to collect information about the user's online activity and build a profile about their interests to provide advertisements relevant to the user. \n\n\n * Cookie\n__cfruid\n * Duration\nsession\n * Description\nCloudflare sets this cookie to identify trusted web traffic.\n\n\n * Cookie\n_GRECAPTCHA\n * Duration\n6 months\n * Description\nGoogle Recaptcha service sets this cookie to identify bots to protect the website against malicious spam attacks.\n\n\n * Cookie\n__Secure-YNID\n * Duration\n6 months\n * Description\nGoogle cookie used to protect user security and prevent fraud, especially during the login process.\n\n\n * Cookie\ncookieyes-consent\n * Duration\n1 year\n * Description\nCookieYes sets this cookie to remember users' consent preferences so that their preferences are respected on subsequent visits to this site. It does not collect or store any personal information about the site visitors.\n\n\nFunctional\nFunctional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.\n * Cookie\nlidc\n * Duration\n1 day\n * Description\nLinkedIn sets the lidc cookie to facilitate data center selection.\n\n\n * Cookie\nbrw\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nbrwConsent\n * Duration\n5 minutes\n * Description\nDescription is currently not available.\n\n\n * Cookie\nWMF-Uniq\n * Duration\n1 year\n * Description\nDescription is currently not available.\n\n\n * Cookie\nloom_anon_comment\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nloom_referral_video\n * Duration\nsession\n * Description\nDescription is currently not available.\n\n\n * Cookie\nVISITOR_INFO1_LIVE\n * Duration\n6 months\n * Description\nA cookie set by YouTube to measure bandwidth that determines whether the user gets the new or old player interface.\n\n\n * Cookie\nyt-remote-connected-devices\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n * Cookie\nytidb::LAST_RESULT_ENTRY_KEY\n * Duration\nNever Expires\n * Description\nThe cookie ytidb::LAST_RESULT_ENTRY_KEY is used by YouTube to store the last search result entry that was clicked by the user. This information is used to improve the user experience by providing more relevant search results in the future.\n\n\n * Cookie\nyt-remote-device-id\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n * Cookie\nyt-remote-session-name\n * Duration\nsession\n * Description\nThe yt-remote-session-name cookie is used by YouTube to store the user's video player preferences using embedded YouTube video.\n\n\n * Cookie\nyt-remote-fast-check-period\n * Duration\nsession\n * Description\nThe yt-remote-fast-check-period cookie is used by YouTube to store the user's video player preferences for embedded YouTube videos.\n\n\n * Cookie\nyt-remote-session-app\n * Duration\nsession\n * Description\nThe yt-remote-session-app cookie is used by YouTube to store user preferences and information about the interface of the embedded YouTube video player.\n\n\n * Cookie\nyt-remote-cast-available\n * Duration\nsession\n * Description\nThe yt-remote-cast-available cookie is used to store the user's preferences regarding whether casting is available on their YouTube video player.\n\n\n * Cookie\nyt-remote-cast-installed\n * Duration\nsession\n * Description\nThe yt-remote-cast-installed cookie is used to store the user's video player preferences using embedded YouTube video.\n\n\n * Cookie\ncp_session\n * Duration\n3 months\n * Description\nCodepen sets this cookie for Help systems found in the website.\n\n\n * Cookie\nloid\n * Duration\n1 year 1 month 4 days\n * Description\nThis cookie is set by the Reddit. The cookie enables the sharing of content from the website onto the social media platform.\n\n\nAnalytics\nAnalytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.\n * Cookie\n__hstc\n * Duration\n6 months\n * Description\nHubspot set this main cookie for tracking visitors. It contains the domain, initial timestamp (first visit), last timestamp (last visit), current timestamp (this visit), and session number (increments for each subsequent session).\n\n\n * Cookie\nhubspotutk\n * Duration\n6 months\n * Description\nHubSpot sets this cookie to keep track of the visitors to the website. This cookie is passed to HubSpot on form submission and used when deduplicating contacts.\n\n\n * Cookie\n_ga\n * Duration\n1 year 1 month 4 days\n * Description\nGoogle Analytics sets this cookie to calculate visitor, session and campaign data and track site usage for the site's analytics report. The cookie stores information anonymously and assigns a randomly generated number to recognise unique visitors.\n\n\n * Cookie\n_ga_*\n * Duration\n1 year 1 month 4 days\n * Description\nGoogle Analytics sets this cookie to store and count page views.\n\n\n * Cookie\n__Host-psifi.analyticsTrace\n * Duration\n6 hours\n * Description\nDescription is currently not available.\n\n\n * Cookie\n__Host-psifi.analyticsTraceV2\n * Duration\n6 hours\n * Description\nDescription is currently not available.\n\n\n * Cookie\n_gh_sess\n * Duration\nsession\n * Description\nGitHub sets this cookie for temporary application and framework state between pages like what step the user is on in a multiple step form.\n\n\n * Cookie\nYSC\n * Duration\nsession\n * Description\nYSC cookie is set by Youtube and is used to track the views of embedded videos on Youtube pages.\n\n\n * Cookie\najs_anonymous_id\n * Duration\n1 year\n * Description\nThis cookie is set by Segment to count the number of people who visit a certain site by tracking if they have visited before.\n\n\n * Cookie\nvuid\n * Duration\n1 year 1 month 4 days\n * Description\nVimeo installs this cookie to collect tracking information by setting a unique ID to embed videos on the website. \n\n\nPerformance\nPerformance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors.\n * Cookie\nAWSALB\n * Duration\n7 days\n * Description\nAWSALB is an application load balancer cookie set by Amazon Web Services to map the session to the target.\n\n\n * Cookie\nacq\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nacq.sig\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nptc\n * Duration\n2 years\n * Description\nNo description available.\n\n\nAdvertisement\nAdvertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns.\n * Cookie\nmuc_ads\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to collect user behaviour and interaction data to optimize the website.\n\n\n * Cookie\nguest_id_marketing\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n * Cookie\nguest_id_ads\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n * Cookie\npersonalization_id\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to integrate and share features for social media and also store information about how the user uses the website, for tracking and targeting.\n\n\n * Cookie\nguest_id\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor. It registers if a user is signed in to the Twitter platform and collects information about ad preferences.\n\n\n * Cookie\nbcookie\n * Duration\n1 year\n * Description\nLinkedIn sets this cookie from LinkedIn share buttons and ad tags to recognize browser IDs.\n\n\n * Cookie\n__Secure-ROLLOUT_TOKEN\n * Duration\n6 months\n * Description\nYouTube sets this cookie to manage feature rollout and experimentation. It helps Google control which new features or interface changes are shown to users as part of testing and staged rollouts, ensuring consistent experience for a given user during an experiment.\n\n\n * Cookie\nyt.innertube::nextId\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n * Cookie\nyt.innertube::requests\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n * Cookie\nsession_tracker\n * Duration\nsession\n * Description\nThis cookie is set by the Reddit. This cookie is used to identify trusted web traffic. It also helps in adverstising on the website.\n\n\n * Cookie\nedgebucket\n * Duration\nsession\n * Description\nReddit sets this cookie to save the information about a log-on Reddit user, for the purpose of advertisement recommendations and updating the content.\n\n\n * Cookie\ndid\n * Duration\n1 year\n * Description\nArbor sets this cookie to show targeted ads to site visitors.This cookie expires after 2 months or 1 year.\n\n\nUncategorised\nOther uncategorised cookies are those that are being analysed and have not been classified into a category as yet.\nNo cookies to display.\nReject All Save My Preferences Accept All\nToggle Search[](https://towardsdatascience.com/)Toggle Mobile Navigation\n[](https://towardsdatascience.com/)\nPublish AI, ML & data-science insights to a global community of data professionals.\nSign in[Submit an Article](https://write.insightmediagroup.io)\n[Latest](https://towardsdatascience.com/latest/)[Editor's Picks](https://towardsdatascience.com/tag/editors-pick/)[Newsletter](https://towardsdatascience.com/tag/the-variable/)[ShipAI](https://towardsdatascience.com/shipai/)\n* * *\n[Write For TDS](https://towardsdatascience.com/submissions/)\n[LinkedIn](https://www.linkedin.com/company/towards-data-science/)[X](https://x.com/TDataScience)Toggle Search\n[Large Language Models](https://towardsdatascience.com/category/artificial-intelligence/large-language-models/)\n# Setting Up Your Own Large Language Model\nStill a long way to go, but the future is promising\n[Ivo Bernardo](https://towardsdatascience.com/author/ivopbernardo/)\nJuly 4, 202610 min read\n\nImage by Author\nYou’ve likely seen the headlines: frontier AI models are increasingly at risk of being locked behind strict export controls or mounting API costs.\nAs this technology embeds itself into our daily lives, the open-source movement isn’t just a philosophical preference, it is a necessary mechanism to keep AI in the hands of everyday users. We aren’t at parity yet; the proprietary models from the massive tech labs still hold a commanding lead in pure performance. But, we can hope that the gap is closing fast. Around the clock, an independent community of researchers and developers is pushing to ensure this technology is accessible to anyone with a computer. \n \nToday, the foundation for true democratization is already here: you can run a highly capable model entirely on your own laptop. For today’s experiment, I set out to find a large language model that can run entirely on my laptop — and use it for the simple tasks I’d normally hand off to a big lab model.\nWe’ll install [Qwen 3 8B](https://huggingface.co/Qwen/Qwen3-8B) on my MacBook Air, run it fully offline, and finally have a language model living on my own machine instead of a distant datacenter. The [Qwen](https://en.wikipedia.org/wiki/Qwen) family of models have been trained by Alibaba (the chinese company) and are fully open source, available on the internet for everyone to download. The model has 9 billion weights and takes up around 6gb of your RAM when loaded. \nWhat follows now is a practical, start-to-finish guide to running a proper local LLM on an Apple Silicon Mac and it includes the terminal commands you need. But before we open the terminal, we need to talk about why this is worth doing at all.\n···\nLearn this step by step with the interactive [AI Engineer](https://roadmap.sh/ai-engineer) roadmap.\n## Why Do This?\nMost of the time, cloud models are better and easier. I’m not going to pretend an 8-billion parameter model on a laptop beats frontier AI. It doesn’t and I will keep using the massive cloud models for heavy lifting.\nBut the constant pricing and sovereignity wars around AI may make open source and local models very relevant for a future where having access to the technology will make a huge difference. Every time you use Claude or ChatGPT, you are sending your data to some remote servers where the access can be blocked at any time.\n“ _Digital sovereignty_ ” is a grand phrase for a very ordinary desire: we may want to own the thing that reads our most sensitive thoughts, the same way you own a physical notebook or keep some cash at home.\nA local model answers that cleanly in the AI world. Once it’s downloaded, nothing leaves the machine. No API keys, no shifting terms of service, no quiet data retention policies. You can pull the Wi-Fi card out and it keeps working. For the highly sensitive part of your work, that alone may be worth the price of admission.\nPeople love to say local models are “ _democratizing_ ” AI. I want that to be true, but we aren’t there yet. Running this stack still assumes you own a €1,500 laptop with massive unified memory and you’re comfortable in a command line. That’s a narrow, lucky slice of the world.\nBut the _trajectory_ is democratizing. Two years ago, running a decent offline model required a dedicated workstation and serious technical pain. This weekend, it took me a couple of hours and 5 gigabytes of disk space.\nSo let’s install the thing.\n···\n## The Machine and the Specs\nI built this on a **MacBook Air M4** with **24 GB of unified memory** and about 235 GB of free storage. This was a fresh start: no Homebrew, no Python environment nightmares.\nThe number that actually matters here is the **24 GB**. Apple Silicon’s “unified memory” is the magic trick that makes Macs so exceptionally good at this. Because the CPU and GPU share the exact same memory pool, massive neural network weights don’t have to be sluggishly shuttled back and forth.\nAn 8B model takes up about 5 GB on disk and sits at roughly 6 GB in memory when loaded. On a 24 GB machine, that’s deeply comfortable. You could run a 14B model and still keep dozens of browser tabs open. (If you’re on an 8 GB Mac, stick to the 1.5B or 3B models and close your other apps).\n···\n## Why Ollama?\nThere are a dozen ways to run local AI, and most of them ask you to care about compiler flags and dependency trees. You shouldn’t have to.\n[Ollama](https://ollama.com/) is an open source framework and tool that just works. It’s a single binary that bundles a highly optimized model runner (`llama.cpp` using Apple's Metal for GPU acceleration), a Docker-style model registry, and a local HTTP API. You install it, you pull a model, and you talk to it. That’s it!\n···\n### Step 1: Install Ollama (No Homebrew Required)\nOllama ships as a standard macOS app in a zip file. The command-line interface (CLI) lives secretly inside the app bundle, so we can set it up entirely by hand.\nbash\n\n```\n# Download the Apple Silicon build\ncd ~/Downloads\ncurl -L -o Ollama-darwin.zip https://ollama.com/download/Ollama-darwin.zip\n# Unzip and move the app into your Applications folder\nunzip -o -q Ollama-darwin.zip\nmv Ollama.app /Applications/\n```\n\nIf you don’t know how to open the terminal, just go to your Mac applications and search for “terminal”:\n \n\n### Step 2: Put Ollama on Your PATH\nI didn’t want to fight with `sudo` permissions in `/usr/local/bin`, so I symlinked the bundled CLI into a local directory I own — this is just a handy shortcut to speed up the installation and spin up the LLM.\nbash\n\n```\n# Create a local bin directory and symlink the CLI\nmkdir -p ~/.local/bin\nln -sf /Applications/Ollama.app/Contents/Resources/ollama ~/.local/bin/ollama\n\n# Make it permanent in your zsh profile\necho 'export PATH=\"$HOME/.local/bin:$PATH\"' >> ~/.zshrc\n# Apply it to your current shell\nexport PATH=\"$HOME/.local/bin:$PATH\"\nollama --version\n```\n\n### Step 3: Start the Server\nOllama runs a lightweight background server to expose the API and manage your computer’s memory.\nbash\n\n```\n# Start the server and log output\nmkdir -p ~/.ollama/logs\nnohup ollama serve > ~/.ollama/logs/serve.log 2>&1 &\n\n# Ping it to check if it's alive\ncurl -s http://127.0.0.1:11434/api/version\n```\n\nIf the command above returns a “version”, ollama is set up!\n\nReturn of Ollama Version in Mac Terminal\n _Note: You can also just double-click the Ollama app in your Applications folder to run this server via your menu bar. I did it via terminal to see exactly what was happening under the hood._\n···\n### Step 4: Pull the Model\nWell this one is as easy as it gets:\nbash\n\n```\nollama pull qwen3:8b \nollama list\n```\n\nGo make a coffee. The download is about 5.2 GB.\nAfter running ollama list, you’ll see the model available for you:\n\nDownloaded LLM available Locally\n···\n### Step 5: Talk to the new digital Brain in your Computer\nYou have three distinct ways to interact with your new local model.\n**1. Interactive Chat (The Easiest)**\nbash\n\n```\nollama run qwen3:8b\n```\n\nRunning the following command will launch the interactive chat:\n\nInteractive Chat Window\nIn the default mode, the model will spill out the “thinking tokens”, something that is normally abstracted and hidden in most commercial tools.\nI’m going to start by asking my local model what it thinks about open source models:\n\nAnswer from the Local Model (Thinking Tokens)\nThe light grey text represents the model’s internal reasoning process. These models perform extensive calculation before generating a response, and for local models, this thinking phase accounts for a significant portion of the total time until the model spews out a response.\nAfter doing the thinking process, here is the answer from the model:\n\nAnswer from Local Model\nWas with most tools, these models also retain some context from previous interactions:\n\nNew question to Local Model\nThe model is outputting 5.7 tokens per second because I’m in battery saving mode. If I turn it down, we will probably see a value of 15–20 tokens per second.\n···\n**2. One-Shot Terminal Commands** \nTo interact with your local model, you can also provide the question outside of the interactive mode:\nbash\n\n```\nollama run qwen3:8b \"write a python script that tells me how many vowels a word has\"\n```\n\nHere’s the script that our local large language model built:\npython\n\n```\n```python\n# Prompt the user for a word\nword = input(\"Enter a word: \")\n\n# Define the set of vowels\nvowels = {'a', 'e', 'i', 'o', 'u'}\n\n# Initialize a counter\ncount = 0\n\n# Convert the word to lowercase and check each character\nfor char in word.lower():\n if char in vowels:\n count += 1\n\n# Output the result\nprint(f\"Number of vowels: {count}\")\n```\n\n···\n**3. The HTTP API (For Scripts and Apps)**\nCan you only use this within the terminal commands?\nOf course not! If you are comfortable with Python, you can build any local script using your local model:\npython\n\n```\nimport json, urllib.request\n\nreq = urllib.request.Request(\n \"http://127.0.0.1:11434/api/generate\",\n data=json.dumps({\n \"model\": \"qwen3:8b\",\n \"prompt\": \"Give me three uses for a local LLM.\",\n \"stream\": False,\n \"think\": False,\n }).encode(),\n headers={\"Content-Type\": \"application/json\"},\n)\nprint(json.loads(urllib.request.urlopen(req).read())[\"response\"])\n```\n\nHere is the answer from the model after running this Python script:\nmarkup\n\n```\nSure! Here are three common and practical uses for a **local LLM (Large Language Model)**:1. **Personalized Assistance and Productivity**A local LLM can act as a private AI assistant, helping with tasks like email drafting, scheduling, note-taking, and even coding. Since it runs locally, it maintains user privacy and doesn't rely on internet connectivity.2. **Content Creation and Language Processing**You can use a local LLM to generate creative content such as blog posts, stories, scripts, or marketing copy. It can also assist with language translation, grammar checking, and summarizing text.3. **Custom Applications and Integration**A local LLM can be integrated into custom applications or workflows, such as chatbots, customer support systems, or data analysis tools. This allows for tailored solutions without exposing sensitive data to external servers.Let me know if you'd like examples of how to implement these uses!\n```\n\nCool! You can now create your own applications with your own local model quite easily.\n···\n## Fine-Tuning the Experience — Taming the “Thinking” Tokens\nQwen 3 is a hybrid reasoning model. By default, it generates a verbose `...` block outlining its chain of thought before providing the actual answer. Sometimes you want to see the math but most of the time, you just want the answer quickly (and cut some time from waiting the output tokens from the thinking process).\nHere is how you bypass the reasoning pass:\n * **Disable it entirely:** `ollama run qwen3:8b --think=false`\n * **Run it, but hide it from the UI:** `ollama run qwen3:8b --hidethinking`\n * **In scripts:** Pass `\"think\": false` in your JSON payload.\n\n\n···\n## A Warning About Web Search\nModels are static up until their training data. That means that they can’t access data after they were trained, and companies have been relying on web search tools to augment the capability of the models. For example for our local model:\n\nLast day of training data of our Local Model\nBut, Ollama allows you to hand the model a web-search tool. This sounds incredible but there’s a catch.\nThe search itself executes on Ollama’s hosted cloud service. The moment you enable it, your prompts are being sent over the internet to fetch search results. The model stays local, but your queries do not. This may violate the principle of privacy you want to guarantee with the setup.\n···\n## Bonus: VS Code Integration\nThe ultimate endgame for me was getting an offline coding assistant. The cleanest, entirely free path for this is the **Continue.dev** extension.\n * Install VS Code and the Continue extension.\n * Open Continue’s configuration file at `~/.continue/config.yaml`.\n * Point it at your local Ollama server:\n\n\nyaml\n\n```\nname: Local Assistant\nversion: 1.0.0\nmodels:\n - name: Qwen3 8B (local)\n provider: ollama\n model: qwen3:8b\n roles:\n - chat\n - edit\n - apply\n - name: Qwen3 8B Autocomplete\n provider: ollama\n model: qwen3:8b\n roles:\n - autocomplete\n```\n\n**Pro-tip:** An 8B model is slightly too heavy for the split-second latency you want for inline code autocomplete. I highly recommend pulling a smaller model specifically for that task (`ollama pull qwen2.5-coder:1.5b-base`), mapping it to the `autocomplete` role, and letting Qwen3 8B handle the heavier `chat` tasks.\n···\n## What if I have a Windows Computer?\nAs I’m not on a windows for this tutorial, I haven’t tried it extensively. But the good news is that the Ollama package is available for Windows computers [here](https://ollama.com/download/windows). \nThe install process may differ a bit, but the logic behind using Ollama and pulling the models will be exactly the same.\n···\n## Where This Leaves Me\nMy total footprint for this project was 156 MB for the software and 5.2 GB for the model itself.\nI now have a highly capable language model living permanently on my hard drive. For public, complex work, I will still reach for the cloud. But for the drafts I don’t want ingested into training data, the offline flights, and the legally bound client documents? This intelligence is now on my computer.\nThis may be a bit too techy for most people still, but things are becoming more democratized. And it’s not just about availability. On the performance front, open-source models are improving at a staggering pace, delivering results that make the future of local AI look incredibly promising. For example, [GLM 5.2](https://ollama.com/library/glm-5.2) and [Qwen 3.7 Max](https://qwen.ai/blog?id=qwen3.7) are catching up to the big labs’ models performance:\n\nComparison of Models performance on Software Engineering Benchmark - Image by Author\nAs the technical floor keeps dropping, “owning your own AI” is going to stop being a luxury reserved for developers with expensive laptops. That is the version of AI democratization I actually believe in.\nGo give your laptop another brain this weekend and long live open source!\n## Register Now to Continue Reading\n### Unlock Towards Data Science with a Free Account\nAll articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more.\nRegister for freeSign in to your account\n* * *\n\nWritten By\n## [Ivo Bernardo](https://towardsdatascience.com/author/ivopbernardo/)\n[See all from Ivo Bernardo](https://towardsdatascience.com/author/ivopbernardo/)\n[Artificial Intelligence](https://towardsdatascience.com/tag/artificial-intelligence/)[Llm](https://towardsdatascience.com/tag/llm/)[Ollama](https://towardsdatascience.com/tag/ollama/)[Open Source](https://towardsdatascience.com/tag/open-source/)[Programming](https://towardsdatascience.com/tag/programming/)\nShare This Article\n * [](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Ftowardsdatascience.com%2Fsetting-up-your-own-large-language-model%2F&title=Setting%20Up%20Your%20Own%20Large%20Language%20Model)\n * [](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Ftowardsdatascience.com%2Fsetting-up-your-own-large-language-model%2F&title=Setting%20Up%20Your%20Own%20Large%20Language%20Model)\n * [](https://x.com/share?url=https%3A%2F%2Ftowardsdatascience.com%2Fsetting-up-your-own-large-language-model%2F&text=Setting%20Up%20Your%20Own%20Large%20Language%20Model)\n\n\nTowards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program.\n[Write for TDS](https://towardsdatascience.com/submissions/)\n## Related Articles\n * [](https://towardsdatascience.com/actor-critic-with-tensorflow-2-x-part-2of-2-b8ceb7e059db/)\n## [Actor-Critic With TensorFlow 2.x [Part 2of 2]](https://towardsdatascience.com/actor-critic-with-tensorflow-2-x-part-2of-2-b8ceb7e059db/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nImplementing A2C with/without multiple workers with TensorFlow 2.x\n[Abhishek Suran](https://towardsdatascience.com/author/abhisheksuran/)\nAugust 2, 20202 min read\n * [](https://towardsdatascience.com/escaping-the-sql-jungle/)\n## [Escaping the SQL Jungle](https://towardsdatascience.com/escaping-the-sql-jungle/)\n[Data Science](https://towardsdatascience.com/category/data-science/)\nBringing software engineering discipline to data transformations\n[Mike Huls](https://towardsdatascience.com/author/mikehuls/)\nMarch 21, 202611 min read\n * [](https://towardsdatascience.com/random-sampling-with-scipy-and-numpy-part-iii-8daa212ce554/)\n## [Random Sampling with SciPy and NumPy Part III](https://towardsdatascience.com/random-sampling-with-scipy-and-numpy-part-iii-8daa212ce554/)\n[Programming](https://towardsdatascience.com/category/programming/)\nImplementing custom distribution sampling in SciPy\n[Mark Jamison](https://towardsdatascience.com/author/markjamison03/)\nApril 22, 20222 min read\n * [](https://towardsdatascience.com/supervised-vs-unsupervised-machine-learning-ae895afc57f/)\n## [Supervised vs Unsupervised Machine Learning](https://towardsdatascience.com/supervised-vs-unsupervised-machine-learning-ae895afc57f/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nLooking at the Differences between Unsupervised and Supervised Learning\n[Marco Santos](https://towardsdatascience.com/author/marco-santos/)\nMay 25, 20206 min read\n * [](https://towardsdatascience.com/the-story-of-newly-found-friendship-ai-ml-and-the-pandemic-6ceb02376ebd/)\n## [The Story Of Newly-Found Friendship: AI, ML, and The Pandemic](https://towardsdatascience.com/the-story-of-newly-found-friendship-ai-ml-and-the-pandemic-6ceb02376ebd/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nThis article talks about how this pandemic has given the AI-ML duo a much-needed push and how this newly-found friendship is working out...\n[Praveen Mishra](https://towardsdatascience.com/author/praveen.mishra_41660/)\nOctober 7, 20209 min read\n * [](https://towardsdatascience.com/how-to-deploy-scikit-learn-models-to-azure-container-instances-a0a59d0d07a1/)\n## [How to Deploy Scikit-Learn Models to Azure Container Instances](https://towardsdatascience.com/how-to-deploy-scikit-learn-models-to-azure-container-instances-a0a59d0d07a1/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nProductionize your Scikit-Learn models with Azure Container Instances\n[Edwin Tan](https://towardsdatascience.com/author/edwin.tan/)\nJune 6, 202212 min read\n * [](https://towardsdatascience.com/how-i-built-and-deployed-an-app-in-2-days-with-lovable-supabase-and-netlify/)\n## [How I Built and Deployed an App in 2 days with Lovable, Supabase, and Netlify](https://towardsdatascience.com/how-i-built-and-deployed-an-app-in-2-days-with-lovable-supabase-and-netlify/)\n[Programming](https://towardsdatascience.com/category/programming/)\nAll ideas can be turned into action in a matter of time now.\n[Soner Yıldırım](https://towardsdatascience.com/author/sonery/)\nSeptember 18, 20257 min read\n * [](https://towardsdatascience.com/say-once-repeating-words-is-not-helping-ai-58f38035f66e/)\n## [Say Once! Repeating Words Is Not Helping AI](https://towardsdatascience.com/say-once-repeating-words-is-not-helping-ai-58f38035f66e/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nHow and why is repeating tokens harming LLMs? Why is this a problem?\n[Salvatore Raieli](https://towardsdatascience.com/author/salvatore-raieli/)\nJune 20, 202314 min read\n * [](https://towardsdatascience.com/from-chaos-to-clarity-streamlining-data-cleansing-using-large-language-models-a539fa0b2d90/)\n## [From Chaos to Clarity: Streamlining Data Cleansing Using Large Language Models](https://towardsdatascience.com/from-chaos-to-clarity-streamlining-data-cleansing-using-large-language-models-a539fa0b2d90/)\n[Large Language Models](https://towardsdatascience.com/category/artificial-intelligence/large-language-models/)\nCleaning up survey responses using OpenAI's GPT Model. Full Code with Github link.\n[Naresh Ram](https://towardsdatascience.com/author/naresh-ram/)\nJune 7, 202317 min read\n\n\n * [YouTube](https://www.youtube.com/c/TowardsDataScience)\n * [X](https://x.com/TDataScience)\n * [LinkedIn](https://www.linkedin.com/company/towards-data-science/)\n * [Threads](https://www.threads.net/@towardsdatascience)\n * [Bluesky](https://bsky.app/profile/towardsdatascience.com)\n\n\n[](https://towardsdatascience.com/)\nYour home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals.\nSubscribe to Our Newsletter[Write For TDS](https://towardsdatascience.com/submissions/)•[About](https://towardsdatascience.com/about-towards-data-science-d691af11cc2f/)•[Advertise](https://contact.towardsdatascience.com/advertise-with-towards-data-science)•[Privacy Policy](https://towardsdatascience.com/privacy-policy/)•[Terms of Use](https://towardsdatascience.com/website-terms-of-use/)\n© Insight Media Group, LLC 2026•[The New Stack](https://thenewstack.io)•[roadmap.sh](https://roadmap.sh)•[Creator Network](https://insightmediagroup.io/creator-network)\n#### Recommended Articles \nClose\n * [  How Relevance Models Foreshadowed Transformers for NLP ](https://towardsdatascience.com/how-relevance-models-foreshadowed-transformers-for-nlp/ \"How Relevance Models Foreshadowed Transformers for NLP\")\n * [  Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1 ](https://towardsdatascience.com/javascript-fatigue-you-dont-need-js-to-build-chatgpt/ \"Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1\")\n * [  The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs ](https://towardsdatascience.com/subset-sum-problem-solved-in-linear-time-for-dense-enough-inputs/ \"The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs\")\n * [  Does Your Company Have a Data Strategy? ](https://towardsdatascience.com/does-your-company-have-a-data-strategy-e8944dbec4ae/ \"Does Your Company Have a Data Strategy?\")\n\n\n",
"metadata": {
"title": "Setting Up Your Own Large Language Model | Towards Data Science",
"description": "Still a long way to go, but the future is promising",
"keywords": null,
"author": "Ivo Bernardo",
"og:locale": "en_US",
"og:type": "article",
"og:title": "Setting Up Your Own Large Language Model",
"og:description": "Still a long way to go, but the future is promising",
"og:url": "https://towardsdatascience.com/setting-up-your-own-large-language-model/",
"og:site_name": "Towards Data Science",
"og:image": "https://assets.insightmediagroup.io/media/wp-content/uploads/2026/07/image-34.jpg",
"og:image:width": "720",
"og:image:height": "421",
"og:image:type": "image/jpeg",
"twitter:card": "summary_large_image",
"twitter:site": "@TDataScience",
"twitter:creator": "@TDataScience",
"twitter:label1": "Written by",
"twitter:data1": "Ivo Bernardo",
"twitter:label2": "Est. reading time",
"twitter:data2": "10 min read",
"article:published_time": "2026-07-04T09:00:05.000Z",
"article:modified_time": "2026-07-04T09:00:05.000Z",
"article:author": "https://towardsdatascience.com/author/ivopbernardo/"
},
"paywall": {
"listed_paid": false,
"detected": false,
"marker": "",
"public_only": false,
"policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
},
"crawl": {
"success": true,
"status_code": 200,
"error": ""
},
"code_blocks": [
{
"language": "txt",
"file": "attachments/406b5167357c/code/block_000.txt",
"chars": 239
},
{
"language": "txt",
"file": "attachments/406b5167357c/code/block_001.txt",
"chars": 335
},
{
"language": "txt",
"file": "attachments/406b5167357c/code/block_002.txt",
"chars": 188
},
{
"language": "txt",
"file": "attachments/406b5167357c/code/block_003.txt",
"chars": 38
},
{
"language": "txt",
"file": "attachments/406b5167357c/code/block_004.txt",
"chars": 20
},
{
"language": "txt",
"file": "attachments/406b5167357c/code/block_005.txt",
"chars": 85
},
{
"language": "txt",
"file": "attachments/406b5167357c/code/block_006.txt",
"chars": 0
},
{
"language": "txt",
"file": "attachments/406b5167357c/code/block_007.txt",
"chars": 214
},
{
"language": "txt",
"file": "attachments/406b5167357c/code/block_008.txt",
"chars": 77
},
{
"language": "txt",
"file": "attachments/406b5167357c/code/block_009.txt",
"chars": 1886
}
],
"images": [
{
"status": "http_404",
"url": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607686/image?etag=1782161464448&width=500&height=500",
"src": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607686/image?etag=1782161464448&width=500&height=500",
"alt": "",
"downloaded": false,
"discovery": "saved_markdown",
"retried_at": "2026-10-20T17:54:23Z"
},
{
"status": "http_404",
"url": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500",
"src": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500",
"alt": "",
"downloaded": false,
"discovery": "saved_markdown",
"retried_at": "2026-10-20T17:54:23Z"
},
{
"status": "http_404",
"url": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500",
"src": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500",
"alt": "",
"downloaded": false,
"discovery": "saved_markdown",
"retried_at": "2026-10-20T17:54:23Z"
},
{
"status": "http_404",
"url": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/3660/image?etag=1780176246070&width=500&height=500",
"src": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/3660/image?etag=1780176246070&width=500&height=500",
"alt": "",
"downloaded": false,
"discovery": "saved_markdown",
"retried_at": "2026-10-20T17:54:23Z"
}
],
"audio": [],
"videos": [
{
"url": "https://www.youtube.com/c/TowardsDataScience",
"kind": "channel_reference",
"status": "reference_only"
}
],
"links": [
{
"type": "internal",
"href": "https://towardsdatascience.com/"
},
{
"type": "internal",
"href": "https://towardsdatascience.com/latest/"
},
{
"type": "internal",
"href": "https://towardsdatascience.com/tag/editors-pick/"
},
{
"type": "internal",
"href": "https://towardsdatascience.com/tag/the-variable/"
},
{
"type": "internal",
"href": "https://towardsdatascience.com/shipai/"
},
{
"type": "internal",
"href": "https://towardsdatascience.com/submissions/"
},
{
"type": "internal",
"href": "https://towardsdatascience.com/category/artificial-intelligence/large-language-models/"
},
{
"type": "internal",
"href": "https://towardsdatascience.com/author/ivopbernardo/"
},
{
"type": "internal",
"href": "https://towardsdatascience.com/tag/artificial-intelligence/"
},
{
"type": "internal",
"href": "https://towardsdatascience.com/tag/llm/"
},
{
"type": "internal",
"href": "https://towardsdatascience.com/tag/ollama/"
},
{
"type": "internal",
"href": "https://towardsdatascience.com/tag/open-source/"
},
{
"type": "internal",
"href": "https://towardsdatascience.com/tag/programming/"
},
{
"type": "internal",
"href": "https://towardsdatascience.com/actor-critic-with-tensorflow-2-x-part-2of-2-b8ceb7e059db/"
},
{
"type": "internal",
"href": "https://towardsdatascience.com/category/artificial-intelligence/"
},
{
"type": "internal",
"href": "https://towardsdatascience.com/author/abhisheksuran/"
},
{
"type": "internal",
"href": "https://towardsdatascience.com/escaping-the-sql-jungle/"
},
{
"type": "internal",
"href": "https://towardsdatascience.com/category/data-science/"
},
{
"type": "internal",
"href": "https://towardsdatascience.com/author/mikehuls/"
},
{
"type": "internal",
"href": "https://towardsdatascience.com/random-sampling-with-scipy-and-numpy-part-iii-8daa212ce554/"
},
{
"type": "external",
"href": "https://write.insightmediagroup.io"
},
{
"type": "external",
"href": "https://www.linkedin.com/company/towards-data-science/"
},
{
"type": "external",
"href": "https://x.com/TDataScience"
},
{
"type": "external",
"href": "https://huggingface.co/Qwen/Qwen3-8B"
},
{
"type": "external",
"href": "https://en.wikipedia.org/wiki/Qwen"
},
{
"type": "external",
"href": "https://roadmap.sh/ai-engineer"
},
{
"type": "external",
"href": "https://ollama.com/"
},
{
"type": "external",
"href": "https://ollama.com/download/windows"
},
{
"type": "external",
"href": "https://ollama.com/library/glm-5.2"
},
{
"type": "external",
"href": "https://qwen.ai/blog?id=qwen3.7"
},
{
"type": "external",
"href": "https://www.facebook.com/sharer/sharer.php?title=Setting+Up+Your+Own+Large+Language+Model&u=https%3A%2F%2Ftowardsdatascience.com%2Fsetting-up-your-own-large-language-model%2F"
},
{
"type": "external",
"href": "https://www.linkedin.com/shareArticle?mini=true&title=Setting+Up+Your+Own+Large+Language+Model&url=https%3A%2F%2Ftowardsdatascience.com%2Fsetting-up-your-own-large-language-model%2F"
},
{
"type": "external",
"href": "https://x.com/share?text=Setting+Up+Your+Own+Large+Language+Model&url=https%3A%2F%2Ftowardsdatascience.com%2Fsetting-up-your-own-large-language-model%2F"
},
{
"type": "external",
"href": "https://www.youtube.com/c/TowardsDataScience"
},
{
"type": "external",
"href": "https://www.threads.net/@towardsdatascience"
},
{
"type": "external",
"href": "https://bsky.app/profile/towardsdatascience.com"
},
{
"type": "external",
"href": "https://thenewstack.io"
},
{
"type": "external",
"href": "https://roadmap.sh"
},
{
"type": "external",
"href": "https://insightmediagroup.io/creator-network"
}
],
"repair_version": 2,
"repaired_at": "2026-10-20T17:28:31Z",
"source_url": "https://towardsdatascience.com/setting-up-your-own-large-language-model/",
"delivery_status": "unverified",
"discovery_status": "http_403",
"missing_reason": "源页面未重新核验:http_403；已按保留正文补采图片",
"repair_error": "",
"actual_video_files": [],
"expected_saved_body_images": 4,
"image_discovery": "saved_markdown",
"source_body_markdown": "\nWe value your privacy\nWe use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking \"Accept All\", you consent to our use of cookies.\nCustomise Reject All Accept All\nCustomise Consent Preferences \nWe use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below.\nThe cookies that are categorised as \"Necessary\" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... Show more\nNecessaryAlways Active\nNecessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.\n * Cookie\nBCTempID\n * Duration\n10 minutes\n * Description\nNo description available.\n\n\n * Cookie\n__cf_bm\n * Duration\n1 hour\n * Description\nThis cookie, set by Cloudflare, is used to support Cloudflare Bot Management. \n\n\n * Cookie\nAWSALBCORS\n * Duration\n7 days\n * Description\nAmazon Web Services set this cookie for load balancing.\n\n\n * Cookie\n_cfuvid\n * Duration\nsession\n * Description\nCloudflare sets this cookie to track users across sessions to optimize user experience by maintaining session consistency and providing personalized services\n\n\n * Cookie\nli_gc\n * Duration\n6 months\n * Description\nLinkedin set this cookie for storing visitor's consent regarding using cookies for non-essential purposes.\n\n\n * Cookie\n__hssrc\n * Duration\nsession\n * Description\nThis cookie is set by Hubspot whenever it changes the session cookie. The __hssrc cookie set to 1 indicates that the user has restarted the browser, and if the cookie does not exist, it is assumed to be a new session.\n\n\n * Cookie\n__hssc\n * Duration\n1 hour\n * Description\nHubSpot sets this cookie to keep track of sessions and to determine if HubSpot should increment the session number and timestamps in the __hstc cookie.\n\n\n * Cookie\nwpEmojiSettingsSupports\n * Duration\nsession\n * Description\nWordPress sets this cookie when a user interacts with emojis on a WordPress site. It helps determine if the user's browser can display emojis properly.\n\n\n * Cookie\nBCSessionID\n * Duration\n1 year 1 month 4 days\n * Description\nBlueconic sets this cookie as a unique identifier for the BlueConic profile.\n\n\n * Cookie\n_octo\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nlogged_in\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\n__Secure-YEC\n * Duration\npast\n * Description\nYouTube sets this cookie to stores the user's video player preferences using embedded YouTube video\n\n\n * Cookie\n__eoi\n * Duration\n6 months\n * Description\nDescription is currently not available.\n\n\n * Cookie\nAWSALBTGCORS\n * Duration\n7 days\n * Description\nNo description available.\n\n\n * Cookie\nlogin-status-p\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nAWSALBTG\n * Duration\n7 days\n * Description\nNo description available.\n\n\n * Cookie\ncsrf_token\n * Duration\nsession\n * Description\nNo description available.\n\n\n * Cookie\ntoken_v2\n * Duration\n1 day\n * Description\nDescription is currently not available.\n\n\n * Cookie\nD\n * Duration\n1 year\n * Description\nDescription is currently not available.\n\n\n * Cookie\nPHPSESSID\n * Duration\nsession\n * Description\nThis cookie is native to PHP applications. The cookie stores and identifies a user's unique session ID to manage user sessions on the website. The cookie is a session cookie and will be deleted when all the browser windows are closed.\n\n\n * Cookie\nVISITOR_PRIVACY_METADATA\n * Duration\n6 months\n * Description\nYouTube sets this cookie to store the user's cookie consent state for the current domain. \n\n\n * Cookie\ncookietest\n * Duration\nsession\n * Description\nThe cookietest cookie is typically used to determine whether the user's browser accepts cookies, essential for website functionality and user experience.\n\n\n * Cookie\n__Host-airtable-session\n * Duration\n1 year\n * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n * Cookie\n__Host-airtable-session.sig\n * Duration\n1 year\n * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n * Cookie\nm\n * Duration\n1 year 1 month 4 days\n * Description\nStripe sets this cookie for fraud prevention purposes. It identifies the device used to access the website, allowing the website to be formatted accordingly.\n\n\n * Cookie\nBIGipServer*\n * Duration\nsession\n * Description\nMarketo sets this cookie to collect information about the user's online activity and build a profile about their interests to provide advertisements relevant to the user. \n\n\n * Cookie\n__cfruid\n * Duration\nsession\n * Description\nCloudflare sets this cookie to identify trusted web traffic.\n\n\n * Cookie\n_GRECAPTCHA\n * Duration\n6 months\n * Description\nGoogle Recaptcha service sets this cookie to identify bots to protect the website against malicious spam attacks.\n\n\n * Cookie\n__Secure-YNID\n * Duration\n6 months\n * Description\nGoogle cookie used to protect user security and prevent fraud, especially during the login process.\n\n\n * Cookie\ncookieyes-consent\n * Duration\n1 year\n * Description\nCookieYes sets this cookie to remember users' consent preferences so that their preferences are respected on subsequent visits to this site. It does not collect or store any personal information about the site visitors.\n\n\nFunctional\nFunctional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.\n * Cookie\nlidc\n * Duration\n1 day\n * Description\nLinkedIn sets the lidc cookie to facilitate data center selection.\n\n\n * Cookie\nbrw\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nbrwConsent\n * Duration\n5 minutes\n * Description\nDescription is currently not available.\n\n\n * Cookie\nWMF-Uniq\n * Duration\n1 year\n * Description\nDescription is currently not available.\n\n\n * Cookie\nloom_anon_comment\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nloom_referral_video\n * Duration\nsession\n * Description\nDescription is currently not available.\n\n\n * Cookie\nVISITOR_INFO1_LIVE\n * Duration\n6 months\n * Description\nA cookie set by YouTube to measure bandwidth that determines whether the user gets the new or old player interface.\n\n\n * Cookie\nyt-remote-connected-devices\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n * Cookie\nytidb::LAST_RESULT_ENTRY_KEY\n * Duration\nNever Expires\n * Description\nThe cookie ytidb::LAST_RESULT_ENTRY_KEY is used by YouTube to store the last search result entry that was clicked by the user. This information is used to improve the user experience by providing more relevant search results in the future.\n\n\n * Cookie\nyt-remote-device-id\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n * Cookie\nyt-remote-session-name\n * Duration\nsession\n * Description\nThe yt-remote-session-name cookie is used by YouTube to store the user's video player preferences using embedded YouTube video.\n\n\n * Cookie\nyt-remote-fast-check-period\n * Duration\nsession\n * Description\nThe yt-remote-fast-check-period cookie is used by YouTube to store the user's video player preferences for embedded YouTube videos.\n\n\n * Cookie\nyt-remote-session-app\n * Duration\nsession\n * Description\nThe yt-remote-session-app cookie is used by YouTube to store user preferences and information about the interface of the embedded YouTube video player.\n\n\n * Cookie\nyt-remote-cast-available\n * Duration\nsession\n * Description\nThe yt-remote-cast-available cookie is used to store the user's preferences regarding whether casting is available on their YouTube video player.\n\n\n * Cookie\nyt-remote-cast-installed\n * Duration\nsession\n * Description\nThe yt-remote-cast-installed cookie is used to store the user's video player preferences using embedded YouTube video.\n\n\n * Cookie\ncp_session\n * Duration\n3 months\n * Description\nCodepen sets this cookie for Help systems found in the website.\n\n\n * Cookie\nloid\n * Duration\n1 year 1 month 4 days\n * Description\nThis cookie is set by the Reddit. The cookie enables the sharing of content from the website onto the social media platform.\n\n\nAnalytics\nAnalytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.\n * Cookie\n__hstc\n * Duration\n6 months\n * Description\nHubspot set this main cookie for tracking visitors. It contains the domain, initial timestamp (first visit), last timestamp (last visit), current timestamp (this visit), and session number (increments for each subsequent session).\n\n\n * Cookie\nhubspotutk\n * Duration\n6 months\n * Description\nHubSpot sets this cookie to keep track of the visitors to the website. This cookie is passed to HubSpot on form submission and used when deduplicating contacts.\n\n\n * Cookie\n_ga\n * Duration\n1 year 1 month 4 days\n * Description\nGoogle Analytics sets this cookie to calculate visitor, session and campaign data and track site usage for the site's analytics report. The cookie stores information anonymously and assigns a randomly generated number to recognise unique visitors.\n\n\n * Cookie\n_ga_*\n * Duration\n1 year 1 month 4 days\n * Description\nGoogle Analytics sets this cookie to store and count page views.\n\n\n * Cookie\n__Host-psifi.analyticsTrace\n * Duration\n6 hours\n * Description\nDescription is currently not available.\n\n\n * Cookie\n__Host-psifi.analyticsTraceV2\n * Duration\n6 hours\n * Description\nDescription is currently not available.\n\n\n * Cookie\n_gh_sess\n * Duration\nsession\n * Description\nGitHub sets this cookie for temporary application and framework state between pages like what step the user is on in a multiple step form.\n\n\n * Cookie\nYSC\n * Duration\nsession\n * Description\nYSC cookie is set by Youtube and is used to track the views of embedded videos on Youtube pages.\n\n\n * Cookie\najs_anonymous_id\n * Duration\n1 year\n * Description\nThis cookie is set by Segment to count the number of people who visit a certain site by tracking if they have visited before.\n\n\n * Cookie\nvuid\n * Duration\n1 year 1 month 4 days\n * Description\nVimeo installs this cookie to collect tracking information by setting a unique ID to embed videos on the website. \n\n\nPerformance\nPerformance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors.\n * Cookie\nAWSALB\n * Duration\n7 days\n * Description\nAWSALB is an application load balancer cookie set by Amazon Web Services to map the session to the target.\n\n\n * Cookie\nacq\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nacq.sig\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nptc\n * Duration\n2 years\n * Description\nNo description available.\n\n\nAdvertisement\nAdvertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns.\n * Cookie\nmuc_ads\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to collect user behaviour and interaction data to optimize the website.\n\n\n * Cookie\nguest_id_marketing\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n * Cookie\nguest_id_ads\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n * Cookie\npersonalization_id\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to integrate and share features for social media and also store information about how the user uses the website, for tracking and targeting.\n\n\n * Cookie\nguest_id\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor. It registers if a user is signed in to the Twitter platform and collects information about ad preferences.\n\n\n * Cookie\nbcookie\n * Duration\n1 year\n * Description\nLinkedIn sets this cookie from LinkedIn share buttons and ad tags to recognize browser IDs.\n\n\n * Cookie\n__Secure-ROLLOUT_TOKEN\n * Duration\n6 months\n * Description\nYouTube sets this cookie to manage feature rollout and experimentation. It helps Google control which new features or interface changes are shown to users as part of testing and staged rollouts, ensuring consistent experience for a given user during an experiment.\n\n\n * Cookie\nyt.innertube::nextId\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n * Cookie\nyt.innertube::requests\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n * Cookie\nsession_tracker\n * Duration\nsession\n * Description\nThis cookie is set by the Reddit. This cookie is used to identify trusted web traffic. It also helps in adverstising on the website.\n\n\n * Cookie\nedgebucket\n * Duration\nsession\n * Description\nReddit sets this cookie to save the information about a log-on Reddit user, for the purpose of advertisement recommendations and updating the content.\n\n\n * Cookie\ndid\n * Duration\n1 year\n * Description\nArbor sets this cookie to show targeted ads to site visitors.This cookie expires after 2 months or 1 year.\n\n\nUncategorised\nOther uncategorised cookies are those that are being analysed and have not been classified into a category as yet.\nNo cookies to display.\nReject All Save My Preferences Accept All\nToggle Search[](https://towardsdatascience.com/)Toggle Mobile Navigation\n[](https://towardsdatascience.com/)\nPublish AI, ML & data-science insights to a global community of data professionals.\nSign in[Submit an Article](https://write.insightmediagroup.io)\n[Latest](https://towardsdatascience.com/latest/)[Editor's Picks](https://towardsdatascience.com/tag/editors-pick/)[Newsletter](https://towardsdatascience.com/tag/the-variable/)[ShipAI](https://towardsdatascience.com/shipai/)\n* * *\n[Write For TDS](https://towardsdatascience.com/submissions/)\n[LinkedIn](https://www.linkedin.com/company/towards-data-science/)[X](https://x.com/TDataScience)Toggle Search\n[Large Language Models](https://towardsdatascience.com/category/artificial-intelligence/large-language-models/)\n# Setting Up Your Own Large Language Model\nStill a long way to go, but the future is promising\n[Ivo Bernardo](https://towardsdatascience.com/author/ivopbernardo/)\nJuly 4, 202610 min read\n\nImage by Author\nYou’ve likely seen the headlines: frontier AI models are increasingly at risk of being locked behind strict export controls or mounting API costs.\nAs this technology embeds itself into our daily lives, the open-source movement isn’t just a philosophical preference, it is a necessary mechanism to keep AI in the hands of everyday users. We aren’t at parity yet; the proprietary models from the massive tech labs still hold a commanding lead in pure performance. But, we can hope that the gap is closing fast. Around the clock, an independent community of researchers and developers is pushing to ensure this technology is accessible to anyone with a computer. \n \nToday, the foundation for true democratization is already here: you can run a highly capable model entirely on your own laptop. For today’s experiment, I set out to find a large language model that can run entirely on my laptop — and use it for the simple tasks I’d normally hand off to a big lab model.\nWe’ll install [Qwen 3 8B](https://huggingface.co/Qwen/Qwen3-8B) on my MacBook Air, run it fully offline, and finally have a language model living on my own machine instead of a distant datacenter. The [Qwen](https://en.wikipedia.org/wiki/Qwen) family of models have been trained by Alibaba (the chinese company) and are fully open source, available on the internet for everyone to download. The model has 9 billion weights and takes up around 6gb of your RAM when loaded. \nWhat follows now is a practical, start-to-finish guide to running a proper local LLM on an Apple Silicon Mac and it includes the terminal commands you need. But before we open the terminal, we need to talk about why this is worth doing at all.\n···\nLearn this step by step with the interactive [AI Engineer](https://roadmap.sh/ai-engineer) roadmap.\n## Why Do This?\nMost of the time, cloud models are better and easier. I’m not going to pretend an 8-billion parameter model on a laptop beats frontier AI. It doesn’t and I will keep using the massive cloud models for heavy lifting.\nBut the constant pricing and sovereignity wars around AI may make open source and local models very relevant for a future where having access to the technology will make a huge difference. Every time you use Claude or ChatGPT, you are sending your data to some remote servers where the access can be blocked at any time.\n“ _Digital sovereignty_ ” is a grand phrase for a very ordinary desire: we may want to own the thing that reads our most sensitive thoughts, the same way you own a physical notebook or keep some cash at home.\nA local model answers that cleanly in the AI world. Once it’s downloaded, nothing leaves the machine. No API keys, no shifting terms of service, no quiet data retention policies. You can pull the Wi-Fi card out and it keeps working. For the highly sensitive part of your work, that alone may be worth the price of admission.\nPeople love to say local models are “ _democratizing_ ” AI. I want that to be true, but we aren’t there yet. Running this stack still assumes you own a €1,500 laptop with massive unified memory and you’re comfortable in a command line. That’s a narrow, lucky slice of the world.\nBut the _trajectory_ is democratizing. Two years ago, running a decent offline model required a dedicated workstation and serious technical pain. This weekend, it took me a couple of hours and 5 gigabytes of disk space.\nSo let’s install the thing.\n···\n## The Machine and the Specs\nI built this on a **MacBook Air M4** with **24 GB of unified memory** and about 235 GB of free storage. This was a fresh start: no Homebrew, no Python environment nightmares.\nThe number that actually matters here is the **24 GB**. Apple Silicon’s “unified memory” is the magic trick that makes Macs so exceptionally good at this. Because the CPU and GPU share the exact same memory pool, massive neural network weights don’t have to be sluggishly shuttled back and forth.\nAn 8B model takes up about 5 GB on disk and sits at roughly 6 GB in memory when loaded. On a 24 GB machine, that’s deeply comfortable. You could run a 14B model and still keep dozens of browser tabs open. (If you’re on an 8 GB Mac, stick to the 1.5B or 3B models and close your other apps).\n···\n## Why Ollama?\nThere are a dozen ways to run local AI, and most of them ask you to care about compiler flags and dependency trees. You shouldn’t have to.\n[Ollama](https://ollama.com/) is an open source framework and tool that just works. It’s a single binary that bundles a highly optimized model runner (`llama.cpp` using Apple's Metal for GPU acceleration), a Docker-style model registry, and a local HTTP API. You install it, you pull a model, and you talk to it. That’s it!\n···\n### Step 1: Install Ollama (No Homebrew Required)\nOllama ships as a standard macOS app in a zip file. The command-line interface (CLI) lives secretly inside the app bundle, so we can set it up entirely by hand.\nbash\n\n```\n# Download the Apple Silicon build\ncd ~/Downloads\ncurl -L -o Ollama-darwin.zip https://ollama.com/download/Ollama-darwin.zip\n# Unzip and move the app into your Applications folder\nunzip -o -q Ollama-darwin.zip\nmv Ollama.app /Applications/\n```\n\nIf you don’t know how to open the terminal, just go to your Mac applications and search for “terminal”:\n \n\n### Step 2: Put Ollama on Your PATH\nI didn’t want to fight with `sudo` permissions in `/usr/local/bin`, so I symlinked the bundled CLI into a local directory I own — this is just a handy shortcut to speed up the installation and spin up the LLM.\nbash\n\n```\n# Create a local bin directory and symlink the CLI\nmkdir -p ~/.local/bin\nln -sf /Applications/Ollama.app/Contents/Resources/ollama ~/.local/bin/ollama\n\n# Make it permanent in your zsh profile\necho 'export PATH=\"$HOME/.local/bin:$PATH\"' >> ~/.zshrc\n# Apply it to your current shell\nexport PATH=\"$HOME/.local/bin:$PATH\"\nollama --version\n```\n\n### Step 3: Start the Server\nOllama runs a lightweight background server to expose the API and manage your computer’s memory.\nbash\n\n```\n# Start the server and log output\nmkdir -p ~/.ollama/logs\nnohup ollama serve > ~/.ollama/logs/serve.log 2>&1 &\n\n# Ping it to check if it's alive\ncurl -s http://127.0.0.1:11434/api/version\n```\n\nIf the command above returns a “version”, ollama is set up!\n\nReturn of Ollama Version in Mac Terminal\n _Note: You can also just double-click the Ollama app in your Applications folder to run this server via your menu bar. I did it via terminal to see exactly what was happening under the hood._\n···\n### Step 4: Pull the Model\nWell this one is as easy as it gets:\nbash\n\n```\nollama pull qwen3:8b \nollama list\n```\n\nGo make a coffee. The download is about 5.2 GB.\nAfter running ollama list, you’ll see the model available for you:\n\nDownloaded LLM available Locally\n···\n### Step 5: Talk to the new digital Brain in your Computer\nYou have three distinct ways to interact with your new local model.\n**1. Interactive Chat (The Easiest)**\nbash\n\n```\nollama run qwen3:8b\n```\n\nRunning the following command will launch the interactive chat:\n\nInteractive Chat Window\nIn the default mode, the model will spill out the “thinking tokens”, something that is normally abstracted and hidden in most commercial tools.\nI’m going to start by asking my local model what it thinks about open source models:\n\nAnswer from the Local Model (Thinking Tokens)\nThe light grey text represents the model’s internal reasoning process. These models perform extensive calculation before generating a response, and for local models, this thinking phase accounts for a significant portion of the total time until the model spews out a response.\nAfter doing the thinking process, here is the answer from the model:\n\nAnswer from Local Model\nWas with most tools, these models also retain some context from previous interactions:\n\nNew question to Local Model\nThe model is outputting 5.7 tokens per second because I’m in battery saving mode. If I turn it down, we will probably see a value of 15–20 tokens per second.\n···\n**2. One-Shot Terminal Commands** \nTo interact with your local model, you can also provide the question outside of the interactive mode:\nbash\n\n```\nollama run qwen3:8b \"write a python script that tells me how many vowels a word has\"\n```\n\nHere’s the script that our local large language model built:\npython\n\n```\n```python\n# Prompt the user for a word\nword = input(\"Enter a word: \")\n\n# Define the set of vowels\nvowels = {'a', 'e', 'i', 'o', 'u'}\n\n# Initialize a counter\ncount = 0\n\n# Convert the word to lowercase and check each character\nfor char in word.lower():\n if char in vowels:\n count += 1\n\n# Output the result\nprint(f\"Number of vowels: {count}\")\n```\n\n···\n**3. The HTTP API (For Scripts and Apps)**\nCan you only use this within the terminal commands?\nOf course not! If you are comfortable with Python, you can build any local script using your local model:\npython\n\n```\nimport json, urllib.request\n\nreq = urllib.request.Request(\n \"http://127.0.0.1:11434/api/generate\",\n data=json.dumps({\n \"model\": \"qwen3:8b\",\n \"prompt\": \"Give me three uses for a local LLM.\",\n \"stream\": False,\n \"think\": False,\n }).encode(),\n headers={\"Content-Type\": \"application/json\"},\n)\nprint(json.loads(urllib.request.urlopen(req).read())[\"response\"])\n```\n\nHere is the answer from the model after running this Python script:\nmarkup\n\n```\nSure! Here are three common and practical uses for a **local LLM (Large Language Model)**:1. **Personalized Assistance and Productivity**A local LLM can act as a private AI assistant, helping with tasks like email drafting, scheduling, note-taking, and even coding. Since it runs locally, it maintains user privacy and doesn't rely on internet connectivity.2. **Content Creation and Language Processing**You can use a local LLM to generate creative content such as blog posts, stories, scripts, or marketing copy. It can also assist with language translation, grammar checking, and summarizing text.3. **Custom Applications and Integration**A local LLM can be integrated into custom applications or workflows, such as chatbots, customer support systems, or data analysis tools. This allows for tailored solutions without exposing sensitive data to external servers.Let me know if you'd like examples of how to implement these uses!\n```\n\nCool! You can now create your own applications with your own local model quite easily.\n···\n## Fine-Tuning the Experience — Taming the “Thinking” Tokens\nQwen 3 is a hybrid reasoning model. By default, it generates a verbose `...` block outlining its chain of thought before providing the actual answer. Sometimes you want to see the math but most of the time, you just want the answer quickly (and cut some time from waiting the output tokens from the thinking process).\nHere is how you bypass the reasoning pass:\n * **Disable it entirely:** `ollama run qwen3:8b --think=false`\n * **Run it, but hide it from the UI:** `ollama run qwen3:8b --hidethinking`\n * **In scripts:** Pass `\"think\": false` in your JSON payload.\n\n\n···\n## A Warning About Web Search\nModels are static up until their training data. That means that they can’t access data after they were trained, and companies have been relying on web search tools to augment the capability of the models. For example for our local model:\n\nLast day of training data of our Local Model\nBut, Ollama allows you to hand the model a web-search tool. This sounds incredible but there’s a catch.\nThe search itself executes on Ollama’s hosted cloud service. The moment you enable it, your prompts are being sent over the internet to fetch search results. The model stays local, but your queries do not. This may violate the principle of privacy you want to guarantee with the setup.\n···\n## Bonus: VS Code Integration\nThe ultimate endgame for me was getting an offline coding assistant. The cleanest, entirely free path for this is the **Continue.dev** extension.\n * Install VS Code and the Continue extension.\n * Open Continue’s configuration file at `~/.continue/config.yaml`.\n * Point it at your local Ollama server:\n\n\nyaml\n\n```\nname: Local Assistant\nversion: 1.0.0\nmodels:\n - name: Qwen3 8B (local)\n provider: ollama\n model: qwen3:8b\n roles:\n - chat\n - edit\n - apply\n - name: Qwen3 8B Autocomplete\n provider: ollama\n model: qwen3:8b\n roles:\n - autocomplete\n```\n\n**Pro-tip:** An 8B model is slightly too heavy for the split-second latency you want for inline code autocomplete. I highly recommend pulling a smaller model specifically for that task (`ollama pull qwen2.5-coder:1.5b-base`), mapping it to the `autocomplete` role, and letting Qwen3 8B handle the heavier `chat` tasks.\n···\n## What if I have a Windows Computer?\nAs I’m not on a windows for this tutorial, I haven’t tried it extensively. But the good news is that the Ollama package is available for Windows computers [here](https://ollama.com/download/windows). \nThe install process may differ a bit, but the logic behind using Ollama and pulling the models will be exactly the same.\n···\n## Where This Leaves Me\nMy total footprint for this project was 156 MB for the software and 5.2 GB for the model itself.\nI now have a highly capable language model living permanently on my hard drive. For public, complex work, I will still reach for the cloud. But for the drafts I don’t want ingested into training data, the offline flights, and the legally bound client documents? This intelligence is now on my computer.\nThis may be a bit too techy for most people still, but things are becoming more democratized. And it’s not just about availability. On the performance front, open-source models are improving at a staggering pace, delivering results that make the future of local AI look incredibly promising. For example, [GLM 5.2](https://ollama.com/library/glm-5.2) and [Qwen 3.7 Max](https://qwen.ai/blog?id=qwen3.7) are catching up to the big labs’ models performance:\n\nComparison of Models performance on Software Engineering Benchmark - Image by Author\nAs the technical floor keeps dropping, “owning your own AI” is going to stop being a luxury reserved for developers with expensive laptops. That is the version of AI democratization I actually believe in.\nGo give your laptop another brain this weekend and long live open source!\n## Register Now to Continue Reading\n### Unlock Towards Data Science with a Free Account\nAll articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more.\nRegister for freeSign in to your account\n* * *\n\nWritten By\n## [Ivo Bernardo](https://towardsdatascience.com/author/ivopbernardo/)\n[See all from Ivo Bernardo](https://towardsdatascience.com/author/ivopbernardo/)\n[Artificial Intelligence](https://towardsdatascience.com/tag/artificial-intelligence/)[Llm](https://towardsdatascience.com/tag/llm/)[Ollama](https://towardsdatascience.com/tag/ollama/)[Open Source](https://towardsdatascience.com/tag/open-source/)[Programming](https://towardsdatascience.com/tag/programming/)\nShare This Article\n * [](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Ftowardsdatascience.com%2Fsetting-up-your-own-large-language-model%2F&title=Setting%20Up%20Your%20Own%20Large%20Language%20Model)\n * [](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Ftowardsdatascience.com%2Fsetting-up-your-own-large-language-model%2F&title=Setting%20Up%20Your%20Own%20Large%20Language%20Model)\n * [](https://x.com/share?url=https%3A%2F%2Ftowardsdatascience.com%2Fsetting-up-your-own-large-language-model%2F&text=Setting%20Up%20Your%20Own%20Large%20Language%20Model)\n\n\nTowards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program.\n[Write for TDS](https://towardsdatascience.com/submissions/)\n## Related Articles\n * [](https://towardsdatascience.com/actor-critic-with-tensorflow-2-x-part-2of-2-b8ceb7e059db/)\n## [Actor-Critic With TensorFlow 2.x [Part 2of 2]](https://towardsdatascience.com/actor-critic-with-tensorflow-2-x-part-2of-2-b8ceb7e059db/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nImplementing A2C with/without multiple workers with TensorFlow 2.x\n[Abhishek Suran](https://towardsdatascience.com/author/abhisheksuran/)\nAugust 2, 20202 min read\n * [](https://towardsdatascience.com/escaping-the-sql-jungle/)\n## [Escaping the SQL Jungle](https://towardsdatascience.com/escaping-the-sql-jungle/)\n[Data Science](https://towardsdatascience.com/category/data-science/)\nBringing software engineering discipline to data transformations\n[Mike Huls](https://towardsdatascience.com/author/mikehuls/)\nMarch 21, 202611 min read\n * [](https://towardsdatascience.com/random-sampling-with-scipy-and-numpy-part-iii-8daa212ce554/)\n## [Random Sampling with SciPy and NumPy Part III](https://towardsdatascience.com/random-sampling-with-scipy-and-numpy-part-iii-8daa212ce554/)\n[Programming](https://towardsdatascience.com/category/programming/)\nImplementing custom distribution sampling in SciPy\n[Mark Jamison](https://towardsdatascience.com/author/markjamison03/)\nApril 22, 20222 min read\n * [](https://towardsdatascience.com/supervised-vs-unsupervised-machine-learning-ae895afc57f/)\n## [Supervised vs Unsupervised Machine Learning](https://towardsdatascience.com/supervised-vs-unsupervised-machine-learning-ae895afc57f/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nLooking at the Differences between Unsupervised and Supervised Learning\n[Marco Santos](https://towardsdatascience.com/author/marco-santos/)\nMay 25, 20206 min read\n * [](https://towardsdatascience.com/the-story-of-newly-found-friendship-ai-ml-and-the-pandemic-6ceb02376ebd/)\n## [The Story Of Newly-Found Friendship: AI, ML, and The Pandemic](https://towardsdatascience.com/the-story-of-newly-found-friendship-ai-ml-and-the-pandemic-6ceb02376ebd/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nThis article talks about how this pandemic has given the AI-ML duo a much-needed push and how this newly-found friendship is working out...\n[Praveen Mishra](https://towardsdatascience.com/author/praveen.mishra_41660/)\nOctober 7, 20209 min read\n * [](https://towardsdatascience.com/how-to-deploy-scikit-learn-models-to-azure-container-instances-a0a59d0d07a1/)\n## [How to Deploy Scikit-Learn Models to Azure Container Instances](https://towardsdatascience.com/how-to-deploy-scikit-learn-models-to-azure-container-instances-a0a59d0d07a1/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nProductionize your Scikit-Learn models with Azure Container Instances\n[Edwin Tan](https://towardsdatascience.com/author/edwin.tan/)\nJune 6, 202212 min read\n * [](https://towardsdatascience.com/how-i-built-and-deployed-an-app-in-2-days-with-lovable-supabase-and-netlify/)\n## [How I Built and Deployed an App in 2 days with Lovable, Supabase, and Netlify](https://towardsdatascience.com/how-i-built-and-deployed-an-app-in-2-days-with-lovable-supabase-and-netlify/)\n[Programming](https://towardsdatascience.com/category/programming/)\nAll ideas can be turned into action in a matter of time now.\n[Soner Yıldırım](https://towardsdatascience.com/author/sonery/)\nSeptember 18, 20257 min read\n * [](https://towardsdatascience.com/say-once-repeating-words-is-not-helping-ai-58f38035f66e/)\n## [Say Once! Repeating Words Is Not Helping AI](https://towardsdatascience.com/say-once-repeating-words-is-not-helping-ai-58f38035f66e/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nHow and why is repeating tokens harming LLMs? Why is this a problem?\n[Salvatore Raieli](https://towardsdatascience.com/author/salvatore-raieli/)\nJune 20, 202314 min read\n * [](https://towardsdatascience.com/from-chaos-to-clarity-streamlining-data-cleansing-using-large-language-models-a539fa0b2d90/)\n## [From Chaos to Clarity: Streamlining Data Cleansing Using Large Language Models](https://towardsdatascience.com/from-chaos-to-clarity-streamlining-data-cleansing-using-large-language-models-a539fa0b2d90/)\n[Large Language Models](https://towardsdatascience.com/category/artificial-intelligence/large-language-models/)\nCleaning up survey responses using OpenAI's GPT Model. Full Code with Github link.\n[Naresh Ram](https://towardsdatascience.com/author/naresh-ram/)\nJune 7, 202317 min read\n\n\n * [YouTube](https://www.youtube.com/c/TowardsDataScience)\n * [X](https://x.com/TDataScience)\n * [LinkedIn](https://www.linkedin.com/company/towards-data-science/)\n * [Threads](https://www.threads.net/@towardsdatascience)\n * [Bluesky](https://bsky.app/profile/towardsdatascience.com)\n\n\n[](https://towardsdatascience.com/)\nYour home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals.\nSubscribe to Our Newsletter[Write For TDS](https://towardsdatascience.com/submissions/)•[About](https://towardsdatascience.com/about-towards-data-science-d691af11cc2f/)•[Advertise](https://contact.towardsdatascience.com/advertise-with-towards-data-science)•[Privacy Policy](https://towardsdatascience.com/privacy-policy/)•[Terms of Use](https://towardsdatascience.com/website-terms-of-use/)\n© Insight Media Group, LLC 2026•[The New Stack](https://thenewstack.io)•[roadmap.sh](https://roadmap.sh)•[Creator Network](https://insightmediagroup.io/creator-network)\n#### Recommended Articles \nClose\n * [  How Relevance Models Foreshadowed Transformers for NLP ](https://towardsdatascience.com/how-relevance-models-foreshadowed-transformers-for-nlp/ \"How Relevance Models Foreshadowed Transformers for NLP\")\n * [  Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1 ](https://towardsdatascience.com/javascript-fatigue-you-dont-need-js-to-build-chatgpt/ \"Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1\")\n * [  The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs ](https://towardsdatascience.com/subset-sum-problem-solved-in-linear-time-for-dense-enough-inputs/ \"The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs\")\n * [  Does Your Company Have a Data Strategy? ](https://towardsdatascience.com/does-your-company-have-a-data-strategy-e8944dbec4ae/ \"Does Your Company Have a Data Strategy?\")\n\n\n",
"body_base_path": "attachments/406b5167357c/pages/",
"body_markdown_path": "attachments/406b5167357c/pages/archived-body.md",
"local_html": "attachments/406b5167357c/pages/archived-body.html",
"body_source": "prior_capture"
}