{ "source_file": "0930", "source_platform": "Dev.to", "author": "xbill", "title_listed": "Gemma 4 on a Tesla T4, Part 2: The Minimum GCE VM and a Script to Drive It", "keywords": "", "domain": "大模型算法", "content_type": "工程实践", "is_paid_flag": "否", "url": "https://dev.to/gde/gemma-4-on-a-tesla-t4-part-2-the-minimum-gce-vm-and-a-script-to-drive-it-3gk1", "publish_date": "2026-09-22", "corpus_stage": "", "notes": "", "id": "7247794cef54", "collected_at": "2026-10-02T13:51:18", "title": "Gemma 4 on a Tesla T4, Part 2: The Minimum GCE VM and a Script to Drive It - DEV Community", "body_markdown": "This article provides a step by step deployment guide for **Gemma 4 E2B\n* to a\nTesla T4\nhosted GPU enabled system. A suite of Python MCP tools is built to simplify management of the vLLM hosted deployment. Part 1 measured which checkpoint the card runs fastest; this part builds the machine underneath it, installs the stack after first boot, and walks through\nvllm-t4\n, the shell script that owns the host state the MCP tools do not.*\ngithub.com/xbill9/gemma4-dev/tree/main/gpu-vllm-t4-2b\nHost\nCompute Engine\nn1-standard-2\n,\nus-west2-b\n— 2 vCPU, 7.8 GB RAM\nGPU\n1x Tesla T4, Turing, compute capability 7.5, 15360 MiB\nImage\ndebian-13-trixie-v20260609\n, no driver preinstalled\nDisks\n32 GB pd-ssd boot, 250 GB pd-balanced at\n/opt1\nSoftware\nvLLM 0.29.0, torch 2.13.0+cu130, driver 615.71.09\nResult\n362 s\nfrom\nvllm-t4 start\nto a healthy endpoint, 13371 MiB of 15360 claimed\nWhere Do I Start?\nPart 1 began with the GPU already working. This part begins with a Google Cloud project and nothing in it.\nThe work splits in two. Everything before first boot is a single\ngcloud\ncommand whose choices are GPU choices: which zone sells a T4, what a GPU does to the maintenance policy, and how much disk to attach given where the checkpoints land. Everything after first boot is software on a Debian image that ships no NVIDIA driver at all.\nAt This Point You Should Have…\nA Google Cloud project with billing enabled and the Compute Engine API turned on\nGPU quota in the region you intend to use — T4 GPU quota in that region starts at zero on a new project\ngcloud\nauthenticated locally\nA Hugging Face account, for the Gemma checkpoints\nWhat the Machine Is\nRead from the instance metadata server and the guest OS on the running VM:\nmachine-type\n:\nn1-standard-2\nzone\n:\nus-west2-b\nimage\n:\nprojects/debian-cloud/global/images/debian-13-trixie-v20260609\nscheduling\n:\n{\n\"\nautomaticRestart\"\n:\n\"\nTRUE\"\n,\n\"\nonHostMaintenance\"\n:\n\"\nTERMINATE\"\n,\n\"\npreemptible\"\n:\n\"\nFALSE\"\n}\ndisks\n:\n[{\n\"\ndeviceName\"\n:\n\"\ndebian13\"\n,\n\"\ntype\"\n:\n\"\nPERSISTENT-SSD\"\n},\n{\n\"\ndeviceName\"\n:\n\"\npersistent-disk-1\"\n,\n\"\ntype\"\n:\n\"\nPERSISTENT-BALANCED\"\n}]\nEnter fullscreen mode\nExit fullscreen mode\nTesla T4, 7.5, 15360 MiB, 615.71.09\nnproc: 2\nMemTotal: 7436 MiB\nEnter fullscreen mode\nExit fullscreen mode\nOn Compute Engine the T4 attaches to the N1 machine family, so the host shape is an N1 choice. This one is the smallest N1 that has been used for this work. Two vCPU costs startup time, and 7.8 GB of RAM is the number that decides the swapfile section below.\nCreating the VM\nThe command below is reconstructed from the running VM's own metadata — machine type, zone, image, scheduling policy and both disk types are read back from the instance, and the accelerator from\nnvidia-smi\n. Re-running the create call was out of scope for this article.\ngcloud compute instances create gemma4-t4\n\\\n--zone\n=\nus-west2-b\n\\\n--machine-type\n=\nn1-standard-2\n\\\n--accelerator\n=\ntype\n=\nnvidia-tesla-t4,count\n=\n1\n\\\n--maintenance-policy\n=\nTERMINATE\n\\\n--restart-on-failure\n\\\n--image-project\n=\ndebian-cloud\n\\\n--image-family\n=\ndebian-13\n\\\n--boot-disk-size\n=\n32GB\n\\\n--boot-disk-type\n=\npd-ssd\n\\\n--create-disk\n=\nname\n=\ngemma4-t4-data,size\n=\n250GB,type\n=\npd-balanced,auto-delete\n=\nno\nEnter fullscreen mode\nExit fullscreen mode\nThree flags in there are about the GPU.\nThe Maintenance Policy Is Forced\n--maintenance-policy=TERMINATE\nis required on any instance with an attached GPU. Compute Engine live-migrates ordinary VMs during host maintenance and cannot migrate one with a GPU, so the API rejects the default\nMIGRATE\npolicy. The metadata confirms what the instance ended up with:\n{\n\"automaticRestart\"\n:\n\"TRUE\"\n,\n\"onHostMaintenance\"\n:\n\"TERMINATE\"\n,\n\"preemptible\"\n:\n\"FALSE\"\n}\nEnter fullscreen mode\nExit fullscreen mode\nPairing it with\n--restart-on-failure\nmeans maintenance stops the VM and brings it back. The server process does not come back with it, and neither does the swapfile — there is a reboot checklist at the end of this article for that reason.\nTwo Disks, Because the Checkpoints Do Not Fit on One\nThe boot disk is 32 GB and the model cache goes on a second 250 GB disk mounted at\n/opt1\n:\nNAME SIZE TYPE MOUNTPOINT\nsda 32G disk\n├─sda1 31.9G part /\n├─sda14 3M part\n└─sda15 124M part /boot/efi\nsdb 250G disk /opt1\nEnter fullscreen mode\nExit fullscreen mode\nA bf16 E2B checkpoint is 10.2 GB and the QAT build is 8.3 GB, so both together are most of a 32 GB root disk before pip has unpacked a CUDA torch wheel. Splitting them keeps the root disk for the OS and puts every multi-gigabyte write on the larger, cheaper volume.\n~/.cache\nis a symlink to\n/opt1/cache\n, so Hugging Face downloads land there without any environment variable.\nThe data disk is mounted from\n/etc/fstab\nand survives a reboot:\nUUID\n=\n8bd96fe8-e301-41d9-8ccc-8123ce89c4a8 /opt1 ext4 discard,defaults,nofail 0 2\nEnter fullscreen mode\nExit fullscreen mode\nMeasure Disk Per Path, Never Once\nMounted on 1B-blocks Avail\n/ 33570021376 8450138112\n/tmp 3898789888 3897544704\n/opt1 263086084096 200857059328\nEnter fullscreen mode\nExit fullscreen mode\nThree filesystems, and the two multi-gigabyte writes an install makes land on different ones.\n/tmp\nis its own 3.9 GB filesystem, which is where pip unpacks wheels, and a CUDA torch plus its NVIDIA runtime dependencies do not fit in it. A single\ndf /\nreports the wrong answer for all three writes.\nAfter Boot: the Driver\nThe Debian 13 image carries no NVIDIA driver.\nnvidia-smi\ndoes not exist on a fresh boot. The driver comes from NVIDIA's own CUDA repository for Debian 13:\n/etc/apt/sources.list.d/cuda-debian13-x86_64.list:\ndeb\n[\nsigned-by\n=\n/usr/share/keyrings/cuda-archive-keyring.gpg] https://developer.download.nvidia.com/compute/cuda/repos/debian13/x86_64/ /\nEnter fullscreen mode\nExit fullscreen mode\nsudo\napt-get\ninstall\n-y\nlinux-headers-\n$(\nuname\n-r\n)\nwget https://developer.download.nvidia.com/compute/cuda/repos/debian13/x86_64/cuda-keyring_1.1-1_all.deb\nsudo\ndpkg\n-i\ncuda-keyring_1.1-1_all.deb\nsudo\napt-get update\nsudo\napt-get\ninstall\n-y\nnvidia-driver\nEnter fullscreen mode\nExit fullscreen mode\nWhat that pulls in:\nlinux-headers-7.1.8+deb13-cloud-amd64 7.1.8-1~bpo13+1\nnvidia-driver 615.71.09-2\nnvidia-driver-cuda 615.71.09-2\nnvidia-kernel-open-dkms 615.71.09-2\nnvidia-kernel-support 615.71.09-2\nEnter fullscreen mode\nExit fullscreen mode\nThe kernel module is the open variant and DKMS compiles it against the running kernel, so the matching\nlinux-headers\npackage has to be installed first. The running kernel here is\n7.1.8+deb13-cloud-amd64\nand its headers come from backports, so the headers install gets its own line ahead of the driver.\nReboot, then check:\n$\nnvidia-smi\n--query-gpu\n=\nname,compute_cap,memory.total,driver_version\n--format\n=\ncsv\nname, compute_cap, memory.total [MiB], driver_version\nTesla T4, 7.5, 15360 MiB, 615.71.09\nEnter fullscreen mode\nExit fullscreen mode\nCompute capability 7.5 is Turing, and it is the number that governs everything in Part 1: no bfloat16 datapath, no fp8, and a 64 KiB shared-memory ceiling per block that the Triton attention kernel has to be clamped to fit.\nThe Swapfile Is Part of the Deploy\n7.8 GB of RAM, and the VM ships with no swap. vLLM is killed by the kernel while staging E2B weights without some, which reads as a crash with no traceback.\nsudo\nfallocate\n-l\n16G /opt1/swapfile\nsudo chmod\n600 /opt1/swapfile\nsudo\nmkswap /opt1/swapfile\nsudo\nswapon /opt1/swapfile\nEnter fullscreen mode\nExit fullscreen mode\nThe swapfile lives on the data disk, so the file itself persists.\n/etc/fstab\nhas no swap line, so a reboot leaves the file in place and disabled. While serving the QAT build, 3592 MiB of the 16 GB is in use.\nPython, and Where Packages Land\nThere are two interpreters on this host and the default is the one with no room:\npython3 -> pyenv 3.12.13, site-packages on / 8.45 GB free\n/usr/bin/python3.13 -> user site at /opt1/pyuser 200.86 GB free\nEnter fullscreen mode\nExit fullscreen mode\nThe MCP server runs under\npython3\n. vLLM runs under\n/usr/bin/python3.13\nwith\nPYTHONUSERBASE=/opt1/pyuser\n, which puts its packages on the large disk. A relocated user site is the same interpreter's own site directory on a different filesystem, so site-packages stays system-wide and no virtualenv is involved.\nInstalling vLLM\nBoth redirections below are load-bearing on this host — one for where packages land, one for where pip unpacks them:\nPYTHONUSERBASE\n=\n/opt1/pyuser\nPIP_CACHE_DIR\n=\n/opt1/pipcache\nTMPDIR\n=\n/opt1/tmp\n\\\n/usr/bin/python3.13\n-m\npip\ninstall\n--user\n--break-system-packages\n-U\n\\\n--upgrade-strategy\neager\n\\\n--extra-index-url\nhttps://download.pytorch.org/whl/cu130 vllm\nEnter fullscreen mode\nExit fullscreen mode\nvLLM pins torch to an exact version, so this moves torch with it. Installing torch with\n--no-deps\nskips cuDNN and the NVIDIA runtime wheels, and\nimport torch\nthen fails on\nlibcudnn.so.9\n.\ntorch 2.13.0+cu130\ntransformers 5.17.0\ntriton 3.7.1\nvllm 0.29.0\nEnter fullscreen mode\nExit fullscreen mode\nThe published cu130 wheels carry\nsm_75\n, so Turing needs no source build.\nmake arch\nasks the interpreter for the list instead of reading the directory.\nThe Turing Clamp\nGemma 4 mixes two attention widths — 256 in its sliding-window layers, 512 in its global layers — and vLLM forces its Triton backend for that mix. At width 512 the kernel asks for more shared memory than a Turing block allows.\nmake patch\nclamps the tile sizes in the installed vLLM:\n✅\n**Patched**\n, in the site-packages this host's\n`python3`\nimports.\n__FILE__\n/opt1/pyuser/lib/python3.13/site-packages/vllm/v1/attention/ops/triton_unified_attention.py\nCLAMP PRESENT\nOCCURRENCES 1\nEnter fullscreen mode\nExit fullscreen mode\nReinstalling or upgrading vLLM reverts it, because the patch edits a file in site-packages and pip replaces that file. Re-run\nmake verify-patch\nafter any vLLM change.\nEnter\nvllm-t4\nAt this point the host is ready and the remaining job is running the server. The rig's\ntpu.env\nholds every serving value, and the MCP tools cover capacity, architecture and patching. Three things are neither rig config nor MCP concerns: the swapfile, launching a process that outlives the shell, and waiting for the endpoint to answer.\n~/bin/vllm-t4\nowns those three. Every serving value it passes to vLLM is read out of the rig's\ntpu.env\nat call time and none is spelled in the script:\nrigval\n()\n{\nsed\n-n\n\"s/^\n$1\n=//p\"\n\"\n$rig\n/tpu.env\"\n|\ntail\n-1\n}\nEnter fullscreen mode\nExit fullscreen mode\ntpu.env\nis a dotenv file whose comments are prose, so sourcing it would both fail as shell and drag\nTMPDIR\nand\nPYTHONUSERBASE\ninto the calling shell. Reading one key at a time keeps the file authoritative without importing it.\nWhy the Script Launches with\nnohup\nmake serve\nruns\npython3 -c \"asyncio.run(server.start_vllm_server())\"\n. asyncio terminates the child subprocess when that short-lived interpreter exits, so the engine is gone about a second after launch, having written nothing to the log. The Part 1 sweep was started under the long-lived MCP server process, where the parent stays alive.\nvllm-t4 start\nlaunches with\nnohup\nand\ndisown\nfrom bash, and writes the same\nrun/vllm.pid\nand\nrun/vllm.log\nthat the rig's own status and stop tools read, so the script and the MCP tools agree about what is running.\nThe Options\nCommand\nWhat it does\nvllm-t4 start\nEnable swap, confirm the clamp, launch detached, wait for\n/health\n, print status\nvllm-t4 start nowait\nThe same, returning as soon as the process is up\nvllm-t4 stop\nmake stop\n— SIGTERM to the pid, VRAM released on exit\nvllm-t4 status\nmake status\n— serving or not, plus claimed VRAM\nvllm-t4 query\nmake query\n— one chat completion against the endpoint\nvllm-t4 log\ntail -f\non\nrun/vllm.log\nvllm-t4 swap\nCreate and enable the swapfile, without starting anything\nFour environment variables override the defaults:\nVariable\nDefault\nUse\nVLLM_T4_RIG\n$HOME/gemma4-dev/gpu-vllm-t4-2b\nPoint at a different rig directory\nVLLM_T4_SWAPFILE\n/opt1/swapfile\nPut swap on another volume\nVLLM_T4_SWAPSIZE\n16G\nSize it differently\nVLLM_T4_WAIT\n1800\nSeconds to wait for\n/health\nstart\nRuns Three Guards Before It Launches\nEach guard costs less than the failure it prevents.\nSwap\n, because the engine is killed during weight loading without it and the kernel log is the only place that says so.\nensure_swap\ncreates the file if it is absent, enables it if it exists, and reports when it is already on.\nThe clamp\n, because an unpatched engine spends minutes loading and then dies with an out-of-resources error that gets attributed to configuration. The check matches the verifier's positive string and refuses on anything else:\ncase\n$out\nin\n*\n'✅ **Patched**'\n*\n)\necho\n\"\n$prog\n: Turing clamp confirmed\"\n;;\n*\n)\necho\n\"\n$out\n\"\n>\n&2\n die\n\"the Turing clamp is not confirmed -- run 'make -C\n$rig\npatch'\"\n;;\nesac\nEnter fullscreen mode\nExit fullscreen mode\nA verifier that cannot import vLLM at all answers neither way, so a whitelist of known-bad strings would admit every unknown-bad one. Matching the single good answer fails closed.\nThe config\n, because an empty value from\ntpu.env\nbecomes an empty CLI argument. Every key is checked for a value and\nPYTHON_BIN\nfor executability before anything launches.\nA Full Start\n$\nvllm-t4 start\nvllm-t4: swap already on: /opt1/swapfile (16777212 KB)\nvllm-t4: Turing clamp confirmed\nvllm-t4: starting google/gemma-4-E2B-it-qat-w4a16-ct on 127.0.0.1:8000\nvllm-t4: pid 17215, log /home/xbill_glitnir_com/gemma4-dev/gpu-vllm-t4-2b/run/vllm.log\nvllm-t4: waiting up to 1800s for http://127.0.0.1:8000/health\n 0s VRAM 0 MiB, 0 %\n 75s VRAM 0 MiB, 0 %\n 90s VRAM 1093 MiB, 10 %\n 105s VRAM 9005 MiB, 10 %\n 166s VRAM 9301 MiB, 0 %\n 181s VRAM 8403 MiB, 0 %\n 241s VRAM 13371 MiB, 0 %\n 347s VRAM 13371 MiB, 0 %\nvllm-t4: healthy after 362s -- http://127.0.0.1:8000\n✅ Serving at http://127.0.0.1:8000 (pid 17215).\n\nVRAM 13371 MiB, 15360 MiB, 0 %\nEnter fullscreen mode\nExit fullscreen mode\nVRAM is printed beside the clock because for the first minutes a compiling engine and a dead one look identical from outside, and claimed device memory is what tells them apart. The shape of that column is the startup: nothing for 75 seconds while Python imports and the weights are read off disk, 9005 MiB once the weights are resident, a dip to 8403 while the engine profiles, then 13371 when the KV cache is allocated.\nWhat the Engine Allocated\nThe same start, in\nrun/vllm.log\n:\nnon\n-\ndefault\nargs\n:\n{\n'\nhost\n'\n:\n'\n127.0.0.1\n'\n,\n'\nmodel\n'\n:\n'\ngoogle/gemma-4-E2B-it-qat-w4a16-ct\n'\n,\n'\ndtype\n'\n:\n'\nfloat16\n'\n,\n'\nmax_model_len\n'\n:\n16384\n,\n'\ngpu_memory_utilization\n'\n:\n0.9\n,\n'\nmax_num_seqs\n'\n:\n8\n}\nCasting\ntorch\n.\nbfloat16\nto\ntorch\n.\nfloat16\n.\nUsing\nMarlinLinearKernel\nfor\nCompressedTensorsWNA16\nModel\nloading\ntook\n8.02\nGiB\nmemory\nand\n88.981559\nseconds\nAvailable\nKV\ncache\nmemory\n:\n4.66\nGiB\nGPU\nKV\ncache\nsize\n:\n519\n,\n681\ntokens\n,\nMaximum\nconcurrency\nfor\n16\n,\n384\ntokens\nper\nrequest\n:\n31.72\nx\ninit\nengine\n(\nprofile\n,\ncreate\nkv\ncache\n,\nwarmup\nmodel\n)\ntook\n61.77\ns\n(\ncompilation\n:\n2.59\ns\n)\nEnter fullscreen mode\nExit fullscreen mode\nCasting torch.bfloat16 to torch.float16\nis the checkpoint's stored dtype meeting\n--dtype float16\nfrom\ntpu.env\n. Turing has no bfloat16 datapath, so PyTorch would upconvert regardless; setting the flag makes the conversion a decision with a record.\nCompilation took 2.59 s here against 112.77 s on the first start of this stack, because\ntorch.compile\ncaches its artifacts under\n~/.cache\nand that cache is warm. A first start on a fresh VM pays the full compile, and on 2 vCPU it is the largest single item in the wall clock.\nstatus\nand\nquery\n$\nvllm-t4 status\n✅ Serving at http://127.0.0.1:8000 (pid 17215).\n\nVRAM 13371 MiB, 15360 MiB, 0 %\nEnter fullscreen mode\nExit fullscreen mode\n$\nvllm-t\n4\nquery\n{\n\"model\"\n:\n\"google/gemma-4-E2B-it-qat-w4a16-ct\"\n,\n\"choices\"\n:\n[{\n\"message\"\n:\n{\n\"role\"\n:\n\"assistant\"\n,\n\"content\"\n:\n\"A TPU, or Tensor Processing Unit, is a specialized type of integrated circuit designed to significantly accelerate the mathematical operations central to training and running machine learning models, particularly those involving matrix multiplications.\"\n},\n\"finish_reason\"\n:\n\"stop\"\n}],\n\"system_fingerprint\"\n:\n\"vllm-0.29.0-9a66a08c\"\n,\n\"usage\"\n:\n{\n\"prompt_tokens\"\n:\n18\n,\n\"total_tokens\"\n:\n56\n,\n\"completion_tokens\"\n:\n38\n}\n}\nEnter fullscreen mode\nExit fullscreen mode\nThat target posts to\n/v1/chat/completions\n. Raw\n/v1/completions\nreturns an empty string on an instruction-tuned checkpoint, so an empty result there means the wrong endpoint was called.\nQuerying It Directly\nThe server is the standard OpenAI-compatible one, on\n127.0.0.1:8000\n:\n$\ncurl\n-fsS\nhttp://127.0.0.1:8000/v1/models\n{\"object\":\"list\",\"data\":[{\"id\":\"google/gemma-4-E2B-it-qat-w4a16-ct\",\"object\":\"model\",\n \"owned_by\":\"vllm\",\"root\":\"google/gemma-4-E2B-it-qat-w4a16-ct\",\"max_model_len\":16384}]}\nEnter fullscreen mode\nExit fullscreen mode\ncurl\n-fsS\nhttp://127.0.0.1:8000/v1/chat/completions\n\\\n-H\n'Content-Type: application/json'\n\\\n-d\n'{\"model\":\"google/gemma-4-E2B-it-qat-w4a16-ct\",\n \"messages\":[{\"role\":\"user\",\"content\":\"Name the four inner planets, comma separated.\"}],\n \"temperature\":0,\"max_tokens\":64}'\nEnter fullscreen mode\nExit fullscreen mode\nMercury, Venus, Earth, Mars\nusage: {'prompt_tokens': 18, 'total_tokens': 26, 'completion_tokens': 8}\nEnter fullscreen mode\nExit fullscreen mode\nStreaming works the same way, with\n\"stream\": true\n:\nchunks: 16\ndata: {\"object\":\"chat.completion.chunk\",\"choices\":[{\"delta\":{\"role\":\"assistant\",\"content\":\"\"}}]}\ndata: {\"object\":\"chat.completion.chunk\",\"choices\":[{\"delta\":{\"content\":\"1\"}}]}\n...\ndata: [DONE]\n\nassembled: 1, 2, 3, 4, 5\nEnter fullscreen mode\nExit fullscreen mode\nPrometheus metrics are on\n/metrics\n, which is the route to per-request counters without a benchmark tool:\nvllm:num_requests_running\n{\nengine\n=\n\"0\"\n,\nmodel_name\n=\n\"google/gemma-4-E2B-it-qat-w4a16-ct\"\n}\n0.0\nvllm:prompt_tokens_total\n{\n...\n}\n53.0\nvllm:generation_tokens_total\n{\n...\n}\n49.0\nEnter fullscreen mode\nExit fullscreen mode\nThe model id in every request body has to match what\n/v1/models\nreports, which is\nMODEL_NAME\nfrom\ntpu.env\n. Serving the bf16 build for an A/B means overriding\nMODEL_NAME\nand\nMODEL_SAFETENSORS_BYTES\nin the environment, and the id in the request body changes with it.\nWhat Each Subcommand Refuses\n$\nvllm-t4 start\n# one is already running\nvllm-t4: already running (pid 14977) -- 'vllm-t4 status', or 'vllm-t4 stop' first [exit 1]\n$\nvllm-t4 bogus\nusage: vllm-t4 [start|stop|status|query|log|swap] [nowait] [exit 2]\n$\nvllm-t4 swap\n# already enabled\nvllm-t4: swap already on: /opt1/swapfile (16777212 KB) [exit 0]\n$\nVLLM_T4_RIG\n=\n/nonexistent vllm-t4 status\nvllm-t4: rig not found: /nonexistent (set VLLM_T4_RIG) [exit 1]\nEnter fullscreen mode\nExit fullscreen mode\nIf the engine dies during the wait, the script prints the last 20 log lines and points at the kernel log, because a memory kill leaves no traceback in vLLM's own output:\nif\n[\n-z\n\"\n`\nrunning_pid\n`\n\"\n]\n;\nthen\necho\n\"\n$prog\n: the engine died during startup; last 20 log lines:\"\n>\n&2\ntail\n-20\n\"\n$rig\n/run/vllm.log\"\n>\n&2\necho\n\"\n$prog\n: if there is no traceback, check 'dmesg -T | grep -i oom'\"\n>\n&2\nEnter fullscreen mode\nExit fullscreen mode\nStopping\n$ vllm-t4 stop\n✅ Sent SIGTERM to vLLM (pid 14977). VRAM is released on exit.\n\nThe T4 stays attached and the VM stays billed — stopping the server is not releasing capacity here.\nEnter fullscreen mode\nExit fullscreen mode\nVRAM returns to\n0 MiB, 15360 MiB\nwithin seconds. Swap is left enabled, since it costs nothing idle and the next start needs it.\nThe VM and its attached T4 bill by the hour whether vLLM runs or not.\ngcloud compute instances stop gemma4-t4 --zone us-west2-b\nstops the instance charge and keeps both disks, which keep billing at the much lower storage rate. Deleting the instance with\nauto-delete=no\non the data disk leaves the 250 GB of checkpoints for the next VM.\nAfter a Reboot\nThree things come back on their own and one does not.\nSurvives a reboot\n/opt1\ndata disk\n🟢 in\n/etc/fstab\nNVIDIA driver\n🟢 DKMS module, loads at boot\nTuring clamp\n🟢 a file in site-packages\n/opt1/swapfile\n❌ the file persists, the\nswapon\ndoes not\nvllm-t4 start\nre-enables swap every time, so the checklist after a reboot is one command. Adding a swap line to\n/etc/fstab\nmakes it survive on its own.\nSummary\nThe goal of this article was to build the smallest Compute Engine VM that serves Gemma 4 E2B on one Tesla T4, deploy the stack after first boot, and document the script that runs it. The key to the solution was keeping the rig's\ntpu.env\nauthoritative for every serving value and giving the script only the three things that are host state: the swapfile, a detached launch, and waiting for the endpoint. The measured results were:\n🟢\n362 s\nfrom\nvllm-t4 start\nto a healthy endpoint on a warm compile cache,\n13371 MiB\nof 15360 claimed\n🟢\n8.02 GiB\nof weights and\n4.66 GiB\nof KV cache, giving\n519,681\ntokens and 31.72x concurrency at 16,384 tokens per request\n🟢 One\ngcloud\ncommand, one apt repository for the driver, one pip install, one patch\n⚠️\n--maintenance-policy=TERMINATE\nis required for an attached GPU, and stops the VM during host maintenance\n⚠️ 7.8 GB of host RAM needs a\n16 GB\nswapfile, and the\nswapon\ndoes not survive a reboot\n❌ A single\ndf\nreports the wrong free space for all three writes the install makes\nOne Tesla T4 on one\nn1-standard-2\nVM in\nus-west2-b\n, Debian 13, vLLM 0.29.0 on torch 2.13.0+cu130, driver 615.71.09. The start timing is a single run on a warm\ntorch.compile\ncache; a first start on a fresh VM pays the full compilation, which was 112.77 s on this host. The\ngcloud compute instances create\ncommand is reconstructed from the running instance's metadata and was not re-executed. Throughput figures for the two checkpoints are in Part 1.\nThe strategy for using MCP for Tesla T4 deployment and benchmarking was validated with an incremental step by step approach.", "metadata": { "title": "Gemma 4 on a Tesla T4, Part 2: The Minimum GCE VM and a Script to Drive It - DEV Community", "description": "Building the smallest Compute Engine VM that serves Gemma 4 E2B on one Tesla T4, installing the driver and vLLM after boot, and a walkthrough of every option in the shell script that starts, checks and queries the server. Tagged with gemma, vllm, gcp, cuda.", "keywords": "gemma, vllm, gcp, cuda, software, coding, development, engineering, inclusive, community", "author": null, "og:type": "article", "og:url": "https://dev.to/gde/gemma-4-on-a-tesla-t4-part-2-the-minimum-gce-vm-and-a-script-to-drive-it-3gk1", "og:title": "Gemma 4 on a Tesla T4, Part 2: The Minimum GCE VM and a Script to Drive It", "og:description": "Building the smallest Compute Engine VM that serves Gemma 4 E2B on one Tesla T4, installing the driver and vLLM after boot, and a walkthrough of every option in the shell script that starts, checks and queries the server.", "og:site_name": "DEV Community", "og:image": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fraw.githubusercontent.com%2Fxbill9%2Fgemma4-dev%2Fmain%2Fgpu-vllm-t4-2b%2Fdevto-t4-vm-cover.25debff5.jpg", "twitter:site": "@thepracticaldev", "twitter:creator": "@", "twitter:title": "Gemma 4 on a Tesla T4, Part 2: The Minimum GCE VM and a Script to Drive It", "twitter:description": "Building the smallest Compute Engine VM that serves Gemma 4 E2B on one Tesla T4, installing the driver and vLLM after boot, and a walkthrough of every option in the shell script that starts, checks and queries the server.", "twitter:card": "summary_large_image", "twitter:widgets:new-embed-design": "on", "twitter:image:src": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fraw.githubusercontent.com%2Fxbill9%2Fgemma4-dev%2Fmain%2Fgpu-vllm-t4-2b%2Fdevto-t4-vm-cover.25debff5.jpg" }, "paywall": { "listed_paid": false, "detected": false, "marker": "", "public_only": false, "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙" }, "crawl": { "success": true, "status_code": 200, "error": "" }, "code_blocks": [ { "file": "attachments/7247794cef54/code/body-000.txt", "chars": 372, "language": "text" }, { "file": "attachments/7247794cef54/code/body-001.txt", "chars": 64, "language": "text" }, { "file": "attachments/7247794cef54/code/body-002.txt", "chars": 401, "language": "text" }, { "file": "attachments/7247794cef54/code/body-003.txt", "chars": 82, "language": "text" }, { "file": "attachments/7247794cef54/code/body-004.txt", "chars": 143, "language": "text" }, { "file": "attachments/7247794cef54/code/body-005.txt", "chars": 81, "language": "text" }, { "file": "attachments/7247794cef54/code/body-006.txt", "chars": 148, "language": "text" }, { "file": "attachments/7247794cef54/code/body-007.txt", "chars": 188, "language": "text" }, { "file": "attachments/7247794cef54/code/body-008.txt", "chars": 253, "language": "text" }, { "file": "attachments/7247794cef54/code/body-009.txt", "chars": 264, "language": "text" }, { "file": "attachments/7247794cef54/code/body-010.txt", "chars": 173, "language": "text" }, { "file": "attachments/7247794cef54/code/body-011.txt", "chars": 121, "language": "text" }, { "file": "attachments/7247794cef54/code/body-012.txt", "chars": 152, "language": "text" }, { "file": "attachments/7247794cef54/code/body-013.txt", "chars": 242, "language": "text" }, { "file": "attachments/7247794cef54/code/body-014.txt", "chars": 85, "language": "text" }, { "file": "attachments/7247794cef54/code/body-015.txt", "chars": 196, "language": "text" }, { "file": "attachments/7247794cef54/code/body-016.txt", "chars": 59, "language": "text" }, { "file": "attachments/7247794cef54/code/body-017.txt", "chars": 176, "language": "text" }, { "file": "attachments/7247794cef54/code/body-018.txt", "chars": 678, "language": "text" }, { "file": "attachments/7247794cef54/code/body-019.txt", "chars": 548, "language": "text" }, { "file": "attachments/7247794cef54/code/body-020.txt", "chars": 97, "language": "text" }, { "file": "attachments/7247794cef54/code/body-021.txt", "chars": 558, "language": "text" }, { "file": "attachments/7247794cef54/code/body-022.txt", "chars": 218, "language": "text" }, { "file": "attachments/7247794cef54/code/body-023.txt", "chars": 283, "language": "text" }, { "file": "attachments/7247794cef54/code/body-024.txt", "chars": 101, "language": "text" }, { "file": "attachments/7247794cef54/code/body-025.txt", "chars": 230, "language": "text" }, { "file": "attachments/7247794cef54/code/body-026.txt", "chars": 164, "language": "text" }, { "file": "attachments/7247794cef54/code/body-027.txt", "chars": 574, "language": "text" }, { "file": "attachments/7247794cef54/code/body-028.txt", "chars": 216, "language": "text" }, { "file": "attachments/7247794cef54/code/body-029.txt", "chars": 178, "language": "text" } ], "images": [], "audio": [], "videos": [], "links": [ { "href": "https://github.com/xbill9/gemma4-dev/tree/main/gpu-vllm-t4-2b", "label": "github.com/xbill9/gemma4-dev/tree/main/gpu-vllm-t4-2b", "type": "a" } ], "repair_version": 2, "repaired_at": "2026-10-14T10:48:27Z", "source_url": "https://dev.to/gde/gemma-4-on-a-tesla-t4-part-2-the-minimum-gce-vm-and-a-script-to-drive-it-3gk1", "papers": [], "actual_video_files": [], "body_chars": 22093, "expected_body_images": 0, "discovery_status": "ok", "delivery_status": "ok", "missing_reason": "", "raw_html": "attachments/7247794cef54/pages/source.html", "local_html": "attachments/7247794cef54/pages/article.html", "body_base_path": "attachments/7247794cef54/pages/", "body_markdown_path": "attachments/7247794cef54/pages/article.md" }