{
  "source_file": "0930",
  "source_platform": "Dev.to",
  "author": "xbill",
  "title_listed": "A 4 GB Laptop GPU Beats a 12-Core CPU by 4.3x on Gemma 4",
  "keywords": "",
  "domain": "大模型算法",
  "content_type": "工程实践",
  "is_paid_flag": "否",
  "url": "https://dev.to/gde/a-4-gb-laptop-gpu-beats-a-12-core-cpu-by-43x-on-gemma-4-4150",
  "publish_date": "2026-09-16",
  "corpus_stage": "",
  "notes": "",
  "id": "10799e11da81",
  "collected_at": "2026-10-02T13:51:25",
  "title": "A 4 GB Laptop GPU Beats a 12-Core CPU by 4.3x on Gemma 4 - DEV Community",
  "body_markdown": "This article compares two ways of serving the same small language model on the same laptop: CPU-only, and on the 4 GB GTX 1650 Ti sitting in the same chassis. The payload is byte-identical on both arms and the command lines differ by a single flag. The card takes decode by\n4.3x\n.\nThe repository is at\nhttps://github.com/xbill9/gemma4-dev\nWhat Is Being Compared?\nOne machine, a 13th Gen Intel Core i7-1360P laptop with a GTX 1650 Ti (Max-Q) in it. Two arms:\nCPU arm\nGPU arm\nDevice\ni7-1360P, 12 cores / 16 threads\nGTX 1650 Ti Max-Q, 4096 MiB\nTopology\n4 SMT P-cores (0-7) + 8 E-cores (8-15)\nTU117, compute capability 7.5,\nno tensor cores\nSIMD / math\navx2\n,\navx_vnni\n,\nno AVX-512\nCUDA\nllama.cpp\nc6824a9\n,\nGGML_CUDA=OFF\nbuild\nc6824a9\n, CUDA build\nFlag that differs\n-ngl 0\n-ngl 99\nEverything else matches, and that is the whole exercise:\n-m\ngemma-4-E2B_q4_0-it.gguf\n--host\n127.0.0.1\n--port\n8080\n\\\n-ngl\n{\n0|99\n}\n-c\n8192\n-ctk\nf16\n-ctv\nf16\n-fa\n1\n-t\n4\n-tb\n8\n--parallel\n1\n--metrics\nEnter fullscreen mode\nExit fullscreen mode\nThe model is\ngoogle/gemma-4-E2B-it-qat-q4_0-gguf\n, a 3.35 GB quantization-aware GGUF. The two arms were run alternately against the same endpoint, CPU first, with a fixed 120 second cooldown between them.\nThe Result\nEight cells: four prompt lengths by two output lengths, three repeats each, concurrency 1. Decode is client-side inter-token rate measured off the SSE stream, which is the only decode statistic both arms can produce.\nin tok\nout tok\nCPU decode\nGPU decode\n🥇\nCPU TTFT ms\nGPU TTFT ms\n🥇\n94\n32\n17.61\n71.22\n4.04x\n1143\n385\n2.97x\n94\n128\n17.22\n71.20\n4.13x\n1219\n385\n3.17x\n516\n32\n16.16\n69.93\n4.33x\n5978\n1610\n3.71x\n516\n128\n16.39\n69.49\n4.24x\n5995\n1612\n3.72x\n998\n32\n15.81\n68.60\n4.34x\n11748\n3244\n3.62x\n998\n128\n16.10\n68.52\n4.26x\n11532\n3242\n3.56x\n1959\n32\n15.66\n67.61\n4.32x\n23926\n6522\n3.67x\n1959\n128\n15.80\n67.64\n4.28x\n23773\n6522\n3.65x\nMedians over the eight cells: decode\n4.27x\n, prefill\n3.63x\n, end-to-end\n3.81x\n.\nIs That Difference Real?\nWithin-cell spread over three repeats was\n6.29%\nworst case on the CPU arm and\n0.84%\non the GPU arm. The decode ratio spans 4.04x to 4.34x across every cell. The effect is an order of magnitude clear of the noise, which is the only reason it is worth reporting at all.\nDecode Is Flat, Prefill Is Linear\nLook down the decode columns rather than across them.\nCPU decode moves from 17.61 to 15.66 tok/s. GPU decode moves from 71.22 to 67.61. Across a\n21x range of prompt length\n, on both arms, decode barely moves. TTFT over the same range goes from 1143 ms to 23926 ms on the CPU and 385 ms to 6522 ms on the GPU — linear in the prompt.\nThat is the expected shape stated twice. Decode reads the whole model per token and is bandwidth-bound; prefill multiplies through the prompt and is compute-bound. An accelerator helps both, but it helps them for different reasons, and a benchmark that quotes one number hides that.\nThe decode\nratio\nalso climbs with context — 4.04x at 94 tokens, 4.34x at 998. The CPU arm loses ground as the KV cache grows and the GPU arm barely does.\nHow Does 3.35 GB Fit in a 4 GB Card?\nIt does not have to. With the model loaded and serving, the card holds\n1598 MiB\n.\nThe artifact is only 32% Q4_0. Both embedding tensors are Q6_K and account for 2.257 GB of the 3.334 GB of tensor bytes, and the largest of them —\nper_layer_token_embd\n, at 1.93 GB, 58% of the file — is created with\nTENSOR_READ_LAZY\nin llama.cpp's\nsrc/models/gemma4.cpp\nand served by\nGGML_OP_GET_ROWS\nstraight out of the mmap. A few rows are touched per token. It never goes to the card.\nWhat is resident is the ~1.08 GB Q4_0 transformer body, which is what decode reads every token, plus about 60 MiB of KV cache and the compute buffers.\nThe same mechanism costs something on the CPU arm instead of saving something.\nRssAnon\nthere is 1,202,152 kB, because llama.cpp repacks Q4_0 weights into an interleaved layout for its AVX2 kernels, copying the body out of the mmap into anonymous memory.\nSo the headline works because of the checkpoint's shape, not in spite of the card's size. A 4 GB GPU from 2021 is not too small for this model. It is roughly 2.5x larger than it needs to be.\nWhat Was Controlled\nThe two arms differ by one flag. Everything else was held equal deliberately, and checked rather than assumed:\nHow it was held\nEngine\none llama.cpp commit,\nc6824a9\n, built twice — CPU build and CUDA build\nBinary identity\nthe SHA-256 of each running executable is recorded in its report\nFlags\nidentical apart from\n-ngl\n, including\n-t 4 -tb 8\non both arms\nPrompts\none harness, byte-identical in both rigs, with device-neutral filler\nPrompt lengths\nevery paired cell matched exactly — 94, 516, 998 and 1959 tokens\nEndpoint\nthe same\n127.0.0.1:8080\n, one arm running at a time\nThat last row is the one worth dwelling on. Both arms serve the same model on the same port, so an HTTP response says nothing about which device produced it.\nThe device is therefore read from the running process rather than taken from a label\n—\n/proc/<pid>/exe\nfor the binary,\n/proc/<pid>/maps\nfor the ggml backends actually loaded,\n/proc/<pid>/cmdline\nfor the real\n-ngl\n— and that attestation is stamped into every report beside the numbers.\nmaps\nrather than\nldd\n, because llama.cpp\ndlopen\ns its backends: a CUDA backend can be absent from\nldd\noutput and present in the running process. The verdict needs a GPU backend mapped in\nand\nlayers assigned to it, since a CUDA build running\n-ngl 0\ncomputes on the CPU but is not a clean CPU arm — the device is initialised and large prefill batches can still land on it. The CPU arm hides the device from the process entirely rather than trusting\n-ngl 0\n.\nWhat Was Not Controlled\nRead the 4x as real and anything under about 7% as nothing.\nOrder and thermals.\nThe CPU arm ran first and saturated 12 cores. An\ni7-1360P and a Max-Q card share one thermal envelope, so the GPU arm started on\na warm package. The 120 second cooldown was sized, not measured. If this biases\nanything it understates the GPU arm.\nPage cache.\nThe GGUF was hot for both arms. This says nothing about cold\nstart, and the lazy-embedding claim above is inferred from the source and the\nresident-memory figures rather than from a dropped-cache experiment.\nCPU affinity.\nNothing was pinned. A separate\nllama-bench\nsweep on this\ndie found affinity worth\n1.61x on prefill\n, because llama.cpp splits work\nevenly and every barrier waits on the slowest thread, so adding the eight\nE-cores to the four P-cores makes prefill\nslower\n. The CPU arm here is\ntherefore not at its best, and\n3.63x is an upper bound on the prefill gap\n.\nDecode is unaffected — thread count is a weak lever there — so the 4.27x\nstands.\nConcurrency.\nSingle stream throughout,\n--parallel 1\n.\nSummary\nThe goal of this article was to measure what a 4 GB laptop GPU is worth against a modern 12-core CPU for serving a small quantized model. The key to the solution was a single-variable comparison: one llama.cpp commit, one prompt set, one flag apart, with the serving device read from\n/proc\nat run time and recorded beside every number. The results were:\nGPU decode is\n4.27x\nthe CPU arm, 4.04x to 4.34x across every cell\nGPU prefill is\n3.63x\n, an upper bound, since the CPU arm was left unpinned\nEnd-to-end is\n3.81x\nDecode is flat in context on both arms; TTFT is linear in it on both\nThe model needs\n1598 MiB\nof the card, because 58% of the file is a lazily\nread embedding table that never leaves host memory\nScope: one laptop, one GGUF, llama.cpp\nc6824a9\nbuilt twice from the same commit, eight paired cells, three repeats per cell, concurrency 1, CPU arm first with a 120 second cooldown. CPU affinity was not set on either arm and the page cache was warm for both.\nThe strategy for using MCP for local accelerator comparison was validated with an incremental step by step approach.",
  "metadata": {
    "title": "A 4 GB Laptop GPU Beats a 12-Core CPU by 4.3x on Gemma 4 - DEV Community",
    "description": "Serving Gemma 4 E2B q4_0 through llama.cpp on one laptop, twice: CPU-only and on a 2021-era 4 GB GTX 1650 Ti. Same GGUF, same binary, same prompts, one flag apart. The card takes decode by 4.3x, and needs only 1598 MiB to do it. Tagged with machinelearning, gpu, benchmarking, python.",
    "keywords": "machinelearning, gpu, benchmarking, python, software, coding, development, engineering, inclusive, community",
    "author": null,
    "og:type": "article",
    "og:url": "https://dev.to/gde/a-4-gb-laptop-gpu-beats-a-12-core-cpu-by-43x-on-gemma-4-4150",
    "og:title": "A 4 GB Laptop GPU Beats a 12-Core CPU by 4.3x on Gemma 4",
    "og:description": "Serving Gemma 4 E2B q4_0 through llama.cpp on one laptop, twice: CPU-only and on a 2021-era 4 GB GTX 1650 Ti. Same GGUF, same binary, same prompts, one flag apart. The card takes decode by 4.3x, and needs only 1598 MiB to do it.",
    "og:site_name": "DEV Community",
    "og:image": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fraw.githubusercontent.com%2Fxbill9%2Fgemma4-dev%2Fmain%2Fdocs%2Fcpu-vs-1650ti%2Fdevto-cover.274ec661.jpg",
    "twitter:site": "@thepracticaldev",
    "twitter:creator": "@",
    "twitter:title": "A 4 GB Laptop GPU Beats a 12-Core CPU by 4.3x on Gemma 4",
    "twitter:description": "Serving Gemma 4 E2B q4_0 through llama.cpp on one laptop, twice: CPU-only and on a 2021-era 4 GB GTX 1650 Ti. Same GGUF, same binary, same prompts, one flag apart. The card takes decode by 4.3x, and needs only 1598 MiB to do it.",
    "twitter:card": "summary_large_image",
    "twitter:widgets:new-embed-design": "on",
    "twitter:image:src": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fraw.githubusercontent.com%2Fxbill9%2Fgemma4-dev%2Fmain%2Fdocs%2Fcpu-vs-1650ti%2Fdevto-cover.274ec661.jpg"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [
    {
      "file": "attachments/10799e11da81/code/body-000.txt",
      "chars": 139,
      "language": "text"
    }
  ],
  "images": [],
  "audio": [],
  "videos": [],
  "links": [
    {
      "href": "https://github.com/xbill9/gemma4-dev",
      "label": "https://github.com/xbill9/gemma4-dev",
      "type": "a"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:48:30Z",
  "source_url": "https://dev.to/gde/a-4-gb-laptop-gpu-beats-a-12-core-cpu-by-43x-on-gemma-4-4150",
  "papers": [],
  "actual_video_files": [],
  "body_chars": 7780,
  "expected_body_images": 0,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/10799e11da81/pages/source.html",
  "local_html": "attachments/10799e11da81/pages/article.html",
  "body_base_path": "attachments/10799e11da81/pages/",
  "body_markdown_path": "attachments/10799e11da81/pages/article.md"
}