{
  "source_file": "0930",
  "source_platform": "Dev.to",
  "author": "xbill",
  "title_listed": "Gemma 4 on an 2021 4 GB Laptop GPU: QAT Takes It From 9.5 GiB to 1.6",
  "keywords": "",
  "domain": "大模型算法",
  "content_type": "工程实践",
  "is_paid_flag": "否",
  "url": "https://dev.to/gde/gemma-4-on-an-old-4-gb-laptop-gpu-qat-takes-it-from-95-gib-to-16-b5l",
  "publish_date": "2026-09-10",
  "corpus_stage": "",
  "notes": "",
  "id": "741f25fcd08e",
  "collected_at": "2026-10-02T13:51:37",
  "title": "Gemma 4 on an 2021 4 GB Laptop GPU: QAT Takes It From 9.5 GiB to 1.6 - DEV Community",
  "body_markdown": "This article provides a step by step deployment guide for Gemma 4 E2B's quantization-aware-trained (QAT) checkpoint to a local, laptop hosted GPU enabled system — a much older Lenovo Yoga 9, on sale since January 2021, with a 4 GB GTX 1650 Ti. A suite of Python MCP tools is built to simplify management of the llama.cpp hosted deployment.\nhttps://github.com/xbill9/gemma4-dev/tree/main/local-llamacpp-1650ti-2b-q4_0\nGemma 4 E2B in bfloat16 is 9.5 GiB of weights. This laptop's GPU has 4 GiB. Even a plain int8 conversion would not fit.\nThe QAT release closes that gap: the same model, trained knowing it would be stored at 4 bits, in a 3.35 GB file of which only 1.31 GiB ever has to be on the GPU.\nIt loaded in 1618 MiB, left more than half the card free, and decodes at 73.75 tok/s.\nWhat is this project trying to Do?\nEvery other rig in this repository serves Gemma 4 from rented hardware: Cloud TPU, Compute Engine, EC2, Cloud Run. I wanted to know whether the same model would run on the laptop I already had — a 10th-generation Core i7 with a GTX 16-series GPU, no tensor cores and a 4 GB ceiling that no quota request can raise.\nIt does, and QAT is the reason. The rest of this article is how, and what the old hardware changes about running it.\nAt This Point You Should Have…\nAn NVIDIA GPU with a CUDA driver — here a GTX 1650 Ti with Max-Q Design, driver 615.71.09\nThe CUDA toolkit —\nnvcc\n13.3 here\nPython 3.10 or newer for the MCP server — 3.14.7 here, the system\npython3\n, no virtualenv\nThe repository cloned, and\nlocal-llamacpp-1650ti-2b-q4_0/\nas your working directory\nClaude Code, or any MCP client that speaks stdio\nThe Laptop\nMachine\nLenovo Yoga 9 15IMH5\nOn sale\nsince January 2021, when Windows Central reviewed it\nCPU\nIntel Core i7-10750H, 10th generation (Comet Lake), 12 threads\nRAM\n15 GiB, as\nfree\nreports it\nGPU\nGeForce GTX 1650 Ti with Max-Q Design, 4096 MiB\nGPU power\n40 W limit\nThe MCP server reports the GPU the same way:\ngpu_status\nEnter fullscreen mode\nExit fullscreen mode\n📡 **GPU** — `local-llamacpp-1650ti-2b-q4_0`\n\nNVIDIA GeForce GTX 1650 Ti with Max-Q Design, 7.5, 4096 MiB, 1606 MiB, 2127 MiB, 615.71.09\n\n⚠️  GTX 16-series (TU116/TU117): compute capability 7.5 but **no tensor cores**. Do not compare throughput against the T4-based `g4dn`/`g5g` rigs on the strength of a matching compute capability.\nEnter fullscreen mode\nExit fullscreen mode\nCompute capability 7.5 is a trap.\nA T4 is also 7.5 and has tensor cores; the GTX 16-series is a cut-down Turing die with the tensor cores removed. llama.cpp notices on its own at init:\nThe following devices will have suboptimal performance due to a lack of tensor cores:\n  Device 0: NVIDIA GeForce GTX 1650 Ti with Max-Q Design\nEnter fullscreen mode\nExit fullscreen mode\nWhy Can't the Normal Model Fit?\nThe sizes, from this repository's model reference:\nGemma 4 E2B as\nWeights\nFits 4096 MiB?\nbfloat16\n9.5 GiB\n❌ no\nint8\n~4.8 GiB\n❌ no\nQAT Q4_0 GGUF, whole file\n3.35 GB\nbarely, on paper\nQAT Q4_0 GGUF, GPU-resident part\n1.31 GiB\n✅ yes\nThe bf16 checkpoint is more than twice the card. Halving it to int8 is still larger than the card. Only a 4-bit model is in range, and that is where QAT comes in.\nWhat Is QAT, and Why Does It Matter Here?\nQuantization-aware training simulates quantization\nduring\ntraining rather than compressing a finished model afterwards. The 4-bit weights are not a post-hoc approximation of a bf16 model: the model was trained knowing it would be stored this way. Google's model card puts the goal plainly:\nThis model card is for the new versions of the Gemma 4 family optimized with Quantization-Aware Training (QAT), which allows preserving similar quality to bfloat16 while dramatically reducing the memory requirements to load the model.\nThat is the difference that matters on a 4 GB card.\nRounding the bf16 model down to 4 bits after the fact would fit too, but nothing in its training would have prepared it for that.\nQAT is the version of \"fits\" that was designed to keep the model's quality close to bf16 on the way there.\nGoogle ships the QAT weights four ways:\nArtifact\nWhat it is\nFor\n-qat-q4_0-unquantized\nQAT values in a half-precision container\ncustom compilation\n-qat-q4_0-gguf\nthe same values, packed Q4_0\nllama.cpp — this rig\n-qat-w4a16-ct\ncompressed-tensors\nvLLM\nmobile\non-device runtimes\nphones\nThe GGUF is not a lesser copy. This repository checked it: four norm tensors read out of the GGUF are bit-identical to the ones in the\n-unquantized\nrelease, so it is the same QAT model in its native packing.\nWhich QAT Checkpoint?\ngoogle/gemma-4-E2B-it-qat-q4_0-gguf\n, which llama.cpp opens directly. The agent reads it off disk:\nmodel_info\nEnter fullscreen mode\nExit fullscreen mode\n📡 **Model** — `local-llamacpp-1650ti-2b-q4_0`\n\n- **Name:** `google/gemma-4-E2B-it-qat-q4_0-gguf`\n- **Path:** `/home/xbill/models/gemma-4-E2B-it-qat-q4_0/gemma-4-E2B_q4_0-it.gguf`\n- **On disk:** 3.35 GB\n- **Quantization slot:** `q4_0` — but the dominant tensor type is **Q6_K**. Both embedding tensors are Q6_K (2.257 GB of 3.334 GB); only the ~1.08 GB transformer body is actually Q4_0.\n- **Resident on GPU:** ~1.31 GiB. `per_layer_token_embd` (1.93 GB, 58% of the file) is `TENSOR_READ_LAZY` and is served by GET_ROWS out of the mmap.\n\nRun `inspect_gguf.py` to re-derive the split from the artifact rather than trusting these numbers.\nEnter fullscreen mode\nExit fullscreen mode\nTwo lines in that output explain why a 3.35 GB file fits in far less than 3.35 GB.\nWhy Only 1.31 GiB Has to Be on the GPU\nA 3.35 GB file against 3724 MiB free reads as \"barely fits.\" It is better than that.\nmake info\nreads the tensor table out of the GGUF:\nmake info\nEnter fullscreen mode\nExit fullscreen mode\ngemma-4-E2B_q4_0-it.gguf\n  tensors:  541\n  total:    3.334 GB\n\n  largest tensors:\n      1926.8 MB  per_layer_token_embd.weight  Q6_K    [8960, 262144]  <- LAZY, host-resident\n       330.3 MB  token_embd.weight            Q6_K    [1536, 262144]\n        27.5 MB  per_layer_model_proj.weight  F16     [1536, 8960]\n        10.6 MB  blk.34.ffn_up.weight         Q4_0    [1536, 12288]\n\n  lazy (never on GPU):        1.927 GB  (58% of file)\n  must be resident:           1.407 GB = 1.31 GiB\n\n  by tensor type (the slot-5 token is q4_0; the file mostly is not):\n    Q6_K      2.257 GB  (67.7%)\n    Q4_0      1.048 GB  (31.4%)\n    F16       0.028 GB  ( 0.8%)\nEnter fullscreen mode\nExit fullscreen mode\nOne tensor is 58% of the file, and none of it has to be on the GPU.\nper_layer_token_embd\nis Gemma 4's per-layer embedding table — the \"E\" in E2B, and why the model has about 5B parameters in total but 2B effective. llama.cpp creates it with\nTENSOR_READ_LAZY\nand uses it as a row lookup, not a matmul, so the rows a token needs are read out of the memory-mapped file on the host.\nQAT and the lazy table stack. QAT puts the 35 transformer blocks at Q4_0, 1.05 GB where half precision would be 3.7 GB; the lazy table keeps the biggest remaining tensor off the card entirely.\nWhat decode actually streams per token drops from 4.514 GB at half precision to 1.407 GB, a 3.2x cut.\nTwo rules follow, and both are the opposite of what a memory-anxious setup would do:\nDo not lower\n-ngl\nto save memory.\nFull offload leaves over 2 GiB free. Lowering it moves real matmul weights to the CPU and saves nothing that was scarce.\nNever pass\n--no-mmap\n.\nLazy reading requires mmap. Turn it off and the 1.93 GB table has to be materialised, and a comfortable fit becomes a failure to load.\nStep 1 — Build llama.cpp for sm_75\nA standard CUDA build, pinned to this card's architecture:\ngrep\n-E\n\"^(GGML_CUDA|CMAKE_CUDA_ARCHITECTURES|CMAKE_BUILD_TYPE)[:=]\"\n~/llama.cpp/build/CMakeCache.txt\n~/llama.cpp/build/bin/llama-server\n--version\nEnter fullscreen mode\nExit fullscreen mode\nCMAKE_BUILD_TYPE:STRING=Release\nCMAKE_CUDA_ARCHITECTURES:UNINITIALIZED=75\nGGML_CUDA:BOOL=ON\nversion: 0.3.0-dev (build 1, commit 95ef7fc)\nEnter fullscreen mode\nExit fullscreen mode\nStep 2 — Configure\ntpu.env\nOne committed env file drives both the\nMakefile\nand the MCP server. It is called\ntpu.env\nfor consistency with its siblings; there is no TPU here.\ngrep\n-E\n\"^(MODEL_PATH|ENDPOINT|N_GPU_LAYERS|CONTEXT_SIZE|KV_CACHE_TYPE|FLASH_ATTENTION|THREADS|PARALLEL_SLOTS|METRICS)=\"\ntpu.env\nEnter fullscreen mode\nExit fullscreen mode\nMODEL_PATH=/home/xbill/models/gemma-4-E2B-it-qat-q4_0/gemma-4-E2B_q4_0-it.gguf\nENDPOINT=http://127.0.0.1:8080\nN_GPU_LAYERS=99\nCONTEXT_SIZE=8192\nKV_CACHE_TYPE=f16\nFLASH_ATTENTION=1\nMETRICS=1\nTHREADS=4\nPARALLEL_SLOTS=1\nEnter fullscreen mode\nExit fullscreen mode\nN_GPU_LAYERS=99\nis full offload. Every value was measured on this card rather than copied from a cloud rig.\nStep 3 — Serve It and Measure the Memory\nmake serve\nEnter fullscreen mode\nExit fullscreen mode\nmake serve\nruns in the foreground on purpose: this is a laptop, nothing is billed, and Ctrl-C is a complete teardown. From another shell:\nnvidia-smi\n--query-compute-apps\n=\npid,process_name,used_memory\n--format\n=\ncsv,noheader\nEnter fullscreen mode\nExit fullscreen mode\n37073, /home/xbill/llama.cpp/build/bin/llama-server, 1618 MiB\nEnter fullscreen mode\nExit fullscreen mode\n1618 MiB of 4096.\nWeights, an 8192-token context and CUDA's own overhead, with more than half the card to spare. Had the lazy table been resident the requirement would have been about 3.5 GB, and the model would have failed to load.\nThe agent confirms it is serving:\nmodel_server_status\nEnter fullscreen mode\nExit fullscreen mode\n✅ Serving at http://127.0.0.1:8080 (pid 83619). `/health` → 200.\nEnter fullscreen mode\nExit fullscreen mode\nThe endpoint is a fixed local address, so there is nothing to discover. An early version of this tool read a missing pid file as \"not running\" and reported ❌ against a healthy server; it now checks which process owns port 8080, and\n/health\nmakes the final call.\nWhere the 1618 MiB Goes\nThe allocation, as printed by the engine when a sibling rig loaded the same file on this same card through Ollama:\nterm\nMiB\nmodel buffer (weights)\n1341.78\nKV, 3 full-attention layers × 8192 cells\n48.00\nKV, 12 sliding-window layers × 1024 cells\n12.00\ncompute buffer\n122.52\nCUDA context + slack\n~94\nThe weights are the QAT body plus the resident embeddings. The KV cache is only 60 MiB, because llama.cpp caps Gemma 4's sliding-window layers at the 1024-token window regardless of context size.\nMy own derivation before the run predicted 1616 MiB — two MiB off — with 144 MiB of KV and ~130 of overhead.\nBoth of those terms were wrong, in opposite directions, and the total still matched.\nA total that agrees with the hardware does not confirm the terms; read the engine's allocation log.\nStep 4 — Ask the Model\nquery_model \"In one sentence, what is a TPU?\"\nEnter fullscreen mode\nExit fullscreen mode\n✅ **Reply**\n\nA TPU is a specialized hardware accelerator designed by Google specifically to speed up the computationally intensive operations required for training and running machine learning models and deep learning tasks.\n\n---\n_(plus 1167 chars of reasoning, suppressed)_\nprompt 25 tok · completion 308 tok · 70.5 tok/s\nEnter fullscreen mode\nExit fullscreen mode\nA 4-bit Gemma 4 answering on a laptop GPU with no tensor cores, through Claude Code, over MCP.\nA one-sentence answer cost 308 completion tokens\n, and most of them were thinking.\nThe Second Way to Get an Empty Reply\nGemma 4 emits a thinking block. llama.cpp routes it to\nreasoning_content\nand leaves\ncontent\nempty until the block closes, so a caller that reads only\ncontent\nwith a modest budget gets\n\"\"\nand concludes the server is broken.\nquery_model\ndefaults to\nmax_tokens=1024\n, and reports the empty case as what it is:\nquery_model \"Name three TPU generations.\" max_tokens=64\nEnter fullscreen mode\nExit fullscreen mode\n📡 **Reasoning only — no answer yet.** `finish_reason: length` after 64 tokens, all of them thinking.\n\nThis is Gemma 4 reasoning, not a broken server. Re-run with a larger `max_tokens` (currently 64).\nEnter fullscreen mode\nExit fullscreen mode\nThe same applies to benchmarks:\na 128-token generation limit on this model measures the thinking phase and nothing else.\nTuning: What Moved Decode?\nllama-bench -p 512 -n 128 -r 3\n, all layers on the GPU:\nconfig\nprefill t/s\ndecode t/s\n🥇\n-fa 1\n, f16 KV,\n-t 4\n340.33\n73.75\n🥈\n-fa 1\n, f16 KV,\n-t 6\n340.39\n73.74\n🥉\n-fa 1\n, f16 KV,\n-t 8\n338.49\n72.97\n-fa 0\n, f16 KV,\n-t 6\n338.74\n70.39\n-fa 1\n, f16 K, q8_0 V\n202.66\n65.55\n-fa 1\n, q8_0 K, f16 V\n239.09\n64.91\nFlash attention is a free +4.8% on decode\n(70.39 → 73.74). Adopted.\nThread count does not matter.\nDecode is GPU-bound.\nQuantizing the KV cache is a loss.\nq8_0\ncosts 11–12% of decode and 30–40% of prefill. With no tensor cores to hide the dequantization, and a KV cache of 60 MiB, it buys memory that was never scarce.\nQAT already did the quantizing that matters.\nGGML_CUDA_FORCE_MMQ\ndoes not pay\n, even though llama.cpp suggests it for this card: 337.61 / 73.04 against the stock build's 340.33 / 73.75.\nDoes Concurrency Help?\nFor decode, yes:\nllama-batched-bench\ngoes from 73.33 to 277.61 tok/s between one and 64 parallel sequences. But that is the decode phase in isolation. Through the HTTP endpoint with 512-token prompts, prefill runs one request at a time — time to first token doubles exactly with every doubling of clients — and aggregate output tops out near 48 tok/s, 1.47x from one client. With short prompts and long answers it reaches 2.6x at 16 clients, but every level between 1 and 16 flips between two speeds from trial to trial.\nFor one person on one laptop, concurrency one is the right setting\n, and that is what\nPARALLEL_SLOTS=1\ncarries.\nManaging It with MCP\nserver.py\nis a single-file MCP server with seven tools:\ngpu_status\n,\nmodel_info\n,\nstart_model_server\n,\nstop_model_server\n,\nmodel_server_status\n,\nquery_model\nand\nget_help\n. There is no provisioning tool, because there is nothing to provision. It registers under its directory name, so every tool reaches Claude Code as\nmcp__local-llamacpp-1650ti-2b-q4_0__<tool>\n.\nA stdio check confirms the handshake independently of any client:\n{\nprintf\n'%s\\n'\n\\\n'{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"initialize\",\"params\":{\"protocolVersion\":\"2025-06-18\",\"capabilities\":{},\"clientInfo\":{\"name\":\"probe\",\"version\":\"0\"}}}'\n\\\n'{\"jsonrpc\":\"2.0\",\"method\":\"notifications/initialized\"}'\n\\\n'{\"jsonrpc\":\"2.0\",\"id\":2,\"method\":\"tools/list\",\"params\":{}}'\n;\nsleep\n4\n;\n}\n\\\n| python3 server.py 2>/dev/null\nEnter fullscreen mode\nExit fullscreen mode\ninitialize OK: name='local-llamacpp-1650ti-2b-q4_0' version='' proto 2025-06-18\ntools/list OK: 7 tools -> get_help, gpu_status, model_info, model_server_status, query_model, start_model_server, stop_model_server\nEnter fullscreen mode\nExit fullscreen mode\nOne Thing Broke: MCP SDK 2.x\nMidway through, the server stopped loading. Another project on the machine needed the MCP Python SDK 2.x, all of these projects share one system Python, and 2.x renamed the class this server imports:\npython3\n-c\n\"from mcp.server.fastmcp import FastMCP\"\nEnter fullscreen mode\nExit fullscreen mode\nraise ModuleNotFoundError(_MESSAGE, name=__name__)\nModuleNotFoundError: No module named 'mcp.server.fastmcp'. This is mcp 2.x, where FastMCP was renamed to MCPServer (from mcp.server.mcpserver import MCPServer) and other APIs changed; see the migration guide at https://py.sdk.modelcontextprotocol.io/v2/migration/#fastmcp-renamed-to-mcpserver or pin 'mcp<2' to keep running v1 code.\nEnter fullscreen mode\nExit fullscreen mode\nPinning back was not an option with a shared interpreter. The fix was two lines, the requirement bound, and the test mock's module path:\n-from mcp.server.fastmcp import FastMCP\n+from mcp.server.mcpserver import MCPServer\n-mcp = FastMCP(MCP_SERVER_NAME)\n+mcp = MCPServer(MCP_SERVER_NAME)\n-mcp>=1.2.0,<2\n+mcp>=2\n-sys.modules[\"mcp.server.fastmcp\"] = _fastmcp_module\n+sys.modules[\"mcp.server.mcpserver\"] = _mcpserver_module\nEnter fullscreen mode\nExit fullscreen mode\nTouching\nstart_model_server\nfor the rename surfaced an older bug: the tool launched\nllama-server\nwithout\n-fa\n,\n-t\nor\n--parallel\n, so an MCP-started server came up with\n4 slots and 6 threads\ninstead of the measured configuration — and llama.cpp splits the context across slots. The argv now lives in one function, a test holds it to the\nMakefile\n, and the running server's real command line matches:\ntr\n'\\0'\n' '\n< /proc/\n$(\npidof llama-server\n)\n/cmdline\nEnter fullscreen mode\nExit fullscreen mode\n/home/xbill/llama.cpp/build/bin/llama-server -m /home/xbill/models/gemma-4-E2B-it-qat-q4_0/gemma-4-E2B_q4_0-it.gguf --host 127.0.0.1 --port 8080 -ngl 99 -c 8192 -ctk f16 -ctv f16 -fa 1 -t 4 --parallel 1 --metrics\nEnter fullscreen mode\nExit fullscreen mode\nmake lint\nmake\ntest\nEnter fullscreen mode\nExit fullscreen mode\nAll checks passed!\nlint OK\n----------------------------------------------------------------------\nRan 30 tests in 0.035s\n\nOK\nEnter fullscreen mode\nExit fullscreen mode\nAnd Price/Performance?\nThere is no price. The laptop was already on the desk: nothing is billed, nothing is reserved, and there is no idle cost to optimise.\nWhat QAT bought is capability, not a discount. Without it this GPU cannot hold Gemma 4 E2B at all. With it, the model takes 1618 MiB and a single conversation decodes at 73.75 tok/s.\nTeardown\n# make serve is in the foreground:\nCtrl-C\nEnter fullscreen mode\nExit fullscreen mode\nOr ask the agent for\nstop_model_server\n, which sends SIGTERM to whichever process owns port 8080, including one started by\nmake serve\n. Not run for this article; the server is still up.\nWhat This Does Not Cover\nOutput quality was not measured here.\nThe claim that QAT holds quality close to bfloat16 is Google's, from the model card; this article measured memory and speed, not accuracy, and never compared the QAT model's answers against a bf16 run.\nNo benchmark completed a task.\nEvery sweep generation hit its token cap inside Gemma 4's thinking block, so the throughput figures are real and the tasks are not.\nNothing here transfers to a T4.\nSame compute capability, different silicon.\nSummary\nThe goal of this article was to run Gemma 4 on a much older laptop whose GPU has 4 GB of memory. The key to the solution was Google's quantization-aware-trained checkpoint, packed as a GGUF, plus llama.cpp leaving its largest tensor in host memory. The measured results were:\nbf16 needs 9.5 GiB and int8 ~4.8 GiB; the QAT GGUF needs 1.31 GiB on the GPU.\nOnly the 4-bit QAT model fits this card, and it fits with room to spare.\nFull offload in 1618 MiB of 4096\n, with an 8192-token context and a KV cache of just 60 MiB.\n73.75 tok/s single-stream decode\non a GPU with no tensor cores; flash attention is worth 4.8%.\nQuantizing further is a loss:\na\nq8_0\nKV cache costs 11–12% of decode, because QAT already took the memory pressure away.\nAn MCP server manages the whole lifecycle\n, and moving it to SDK 2.x took a two-line change that also exposed an MCP-started server running the wrong configuration.\nScope: one Lenovo Yoga 9 15IMH5 (Core i7-10750H, 15 GiB RAM, GTX 1650 Ti Max-Q with 4096 MiB and a 40 W cap), llama.cpp\n95ef7fc\nbuilt for sm_75 with CUDA 13.3,\ngoogle/gemma-4-E2B-it-qat-q4_0-gguf\n, and the MCP server on mcp 2.2.0 under Python 3.14.7.\nllama-bench\nrows are three repeats; serving sweeps are three repeats per level with the prompt cache defeated. The memory split comes from a sibling rig running the same engine through Ollama on the same card. Model quality was not measured.\nThe strategy for using MCP for a local GPU deployment was validated with an incremental step by step approach.\nReferences\nlocal-llamacpp-1650ti-2b-q4_0 | GitHub\ngoogle/gemma-4-E2B-it-qat-q4_0-gguf | Hugging Face\nLenovo Yoga 9i 15 review, January 2021 | Windows Central\nQuantization-Aware Training for Gemma 4 | Google\nllama.cpp | GitHub\nMigration Guide: v1 to v2 | MCP Python SDK",
  "metadata": {
    "title": "Gemma 4 on an 2021 4 GB Laptop GPU: QAT Takes It From 9.5 GiB to 1.6 - DEV Community",
    "description": "Step-by-step: running Google's quantization-aware-trained Gemma 4 E2B on a 10th-gen Core i7 laptop with a 4 GB GTX 1650 Ti — why bf16 and int8 cannot fit, why the QAT GGUF does with room to spare, and managing it with an MCP server. Tagged with gemma, llamacpp, mcp, cuda.",
    "keywords": "gemma, llamacpp, mcp, cuda, software, coding, development, engineering, inclusive, community",
    "author": null,
    "og:type": "article",
    "og:url": "https://dev.to/gde/gemma-4-on-an-old-4-gb-laptop-gpu-qat-takes-it-from-95-gib-to-16-b5l",
    "og:title": "Gemma 4 on an 2021 4 GB Laptop GPU: QAT Takes It From 9.5 GiB to 1.6",
    "og:description": "Step-by-step: running Google's quantization-aware-trained Gemma 4 E2B on a 10th-gen Core i7 laptop with a 4 GB GTX 1650 Ti — why bf16 and int8 cannot fit, why the QAT GGUF does with room to spare, and managing it with an MCP server.",
    "og:site_name": "DEV Community",
    "og:image": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fraw.githubusercontent.com%2Fxbill9%2Fgemma4-dev%2Fmain%2Flocal-llamacpp-1650ti-2b-q4_0%2Fdocs%2Fdevto-cover.225eaada.jpg",
    "twitter:site": "@thepracticaldev",
    "twitter:creator": "@",
    "twitter:title": "Gemma 4 on an 2021 4 GB Laptop GPU: QAT Takes It From 9.5 GiB to 1.6",
    "twitter:description": "Step-by-step: running Google's quantization-aware-trained Gemma 4 E2B on a 10th-gen Core i7 laptop with a 4 GB GTX 1650 Ti — why bf16 and int8 cannot fit, why the QAT GGUF does with room to spare, and managing it with an MCP server.",
    "twitter:card": "summary_large_image",
    "twitter:widgets:new-embed-design": "on",
    "twitter:image:src": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fraw.githubusercontent.com%2Fxbill9%2Fgemma4-dev%2Fmain%2Flocal-llamacpp-1650ti-2b-q4_0%2Fdocs%2Fdevto-cover.225eaada.jpg"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [
    {
      "file": "attachments/741f25fcd08e/code/body-000.txt",
      "chars": 11,
      "language": "text"
    },
    {
      "file": "attachments/741f25fcd08e/code/body-001.txt",
      "chars": 333,
      "language": "text"
    },
    {
      "file": "attachments/741f25fcd08e/code/body-002.txt",
      "chars": 143,
      "language": "text"
    },
    {
      "file": "attachments/741f25fcd08e/code/body-003.txt",
      "chars": 11,
      "language": "text"
    },
    {
      "file": "attachments/741f25fcd08e/code/body-004.txt",
      "chars": 636,
      "language": "text"
    },
    {
      "file": "attachments/741f25fcd08e/code/body-005.txt",
      "chars": 10,
      "language": "text"
    },
    {
      "file": "attachments/741f25fcd08e/code/body-006.txt",
      "chars": 650,
      "language": "text"
    },
    {
      "file": "attachments/741f25fcd08e/code/body-007.txt",
      "chars": 147,
      "language": "text"
    },
    {
      "file": "attachments/741f25fcd08e/code/body-008.txt",
      "chars": 137,
      "language": "text"
    },
    {
      "file": "attachments/741f25fcd08e/code/body-009.txt",
      "chars": 129,
      "language": "text"
    },
    {
      "file": "attachments/741f25fcd08e/code/body-010.txt",
      "chars": 217,
      "language": "text"
    },
    {
      "file": "attachments/741f25fcd08e/code/body-011.txt",
      "chars": 11,
      "language": "text"
    },
    {
      "file": "attachments/741f25fcd08e/code/body-012.txt",
      "chars": 83,
      "language": "text"
    },
    {
      "file": "attachments/741f25fcd08e/code/body-013.txt",
      "chars": 62,
      "language": "text"
    },
    {
      "file": "attachments/741f25fcd08e/code/body-014.txt",
      "chars": 20,
      "language": "text"
    },
    {
      "file": "attachments/741f25fcd08e/code/body-015.txt",
      "chars": 65,
      "language": "text"
    },
    {
      "file": "attachments/741f25fcd08e/code/body-016.txt",
      "chars": 46,
      "language": "text"
    },
    {
      "file": "attachments/741f25fcd08e/code/body-017.txt",
      "chars": 323,
      "language": "text"
    },
    {
      "file": "attachments/741f25fcd08e/code/body-018.txt",
      "chars": 56,
      "language": "text"
    },
    {
      "file": "attachments/741f25fcd08e/code/body-019.txt",
      "chars": 200,
      "language": "text"
    },
    {
      "file": "attachments/741f25fcd08e/code/body-020.txt",
      "chars": 347,
      "language": "text"
    },
    {
      "file": "attachments/741f25fcd08e/code/body-021.txt",
      "chars": 212,
      "language": "text"
    },
    {
      "file": "attachments/741f25fcd08e/code/body-022.txt",
      "chars": 52,
      "language": "text"
    },
    {
      "file": "attachments/741f25fcd08e/code/body-023.txt",
      "chars": 388,
      "language": "text"
    },
    {
      "file": "attachments/741f25fcd08e/code/body-024.txt",
      "chars": 285,
      "language": "text"
    },
    {
      "file": "attachments/741f25fcd08e/code/body-025.txt",
      "chars": 50,
      "language": "text"
    },
    {
      "file": "attachments/741f25fcd08e/code/body-026.txt",
      "chars": 213,
      "language": "text"
    },
    {
      "file": "attachments/741f25fcd08e/code/body-027.txt",
      "chars": 20,
      "language": "text"
    },
    {
      "file": "attachments/741f25fcd08e/code/body-028.txt",
      "chars": 125,
      "language": "text"
    },
    {
      "file": "attachments/741f25fcd08e/code/body-029.txt",
      "chars": 42,
      "language": "text"
    }
  ],
  "images": [],
  "audio": [],
  "videos": [],
  "links": [
    {
      "href": "https://github.com/xbill9/gemma4-dev/tree/main/local-llamacpp-1650ti-2b-q4_0",
      "label": "https://github.com/xbill9/gemma4-dev/tree/main/local-llamacpp-1650ti-2b-q4_0",
      "type": "a"
    },
    {
      "href": "https://huggingface.co/google/gemma-4-E2B-it-qat-q4_0-gguf",
      "label": "google/gemma-4-E2B-it-qat-q4_0-gguf | Hugging Face",
      "type": "a"
    },
    {
      "href": "https://www.windowscentral.com/lenovo-yoga-9i-15",
      "label": "Lenovo Yoga 9i 15 review, January 2021 | Windows Central",
      "type": "a"
    },
    {
      "href": "https://blog.google/innovation-and-ai/technology/developers-tools/quantization-aware-training-gemma-4/",
      "label": "Quantization-Aware Training for Gemma 4 | Google",
      "type": "a"
    },
    {
      "href": "https://github.com/ggml-org/llama.cpp",
      "label": "llama.cpp | GitHub",
      "type": "a"
    },
    {
      "href": "https://py.sdk.modelcontextprotocol.io/v2/migration/",
      "label": "Migration Guide: v1 to v2 | MCP Python SDK",
      "type": "a"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:48:33Z",
  "source_url": "https://dev.to/gde/gemma-4-on-an-old-4-gb-laptop-gpu-qat-takes-it-from-95-gib-to-16-b5l",
  "papers": [],
  "actual_video_files": [],
  "body_chars": 19743,
  "expected_body_images": 0,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/741f25fcd08e/pages/source.html",
  "local_html": "attachments/741f25fcd08e/pages/article.html",
  "body_base_path": "attachments/741f25fcd08e/pages/",
  "body_markdown_path": "attachments/741f25fcd08e/pages/article.md"
}