{
  "source_file": "0930",
  "source_platform": "Dev.to",
  "author": "xbill",
  "title_listed": "Two Rust Clients for Gemma 4: Calling the Endpoint vs. Calling the MCP Server 🦀",
  "keywords": "",
  "domain": "大模型算法",
  "content_type": "工程实践",
  "is_paid_flag": "否",
  "url": "https://dev.to/gde/two-rust-clients-for-gemma-4-calling-the-endpoint-vs-calling-the-mcp-server-3kj4",
  "publish_date": "2026-09-11",
  "corpus_stage": "",
  "notes": "",
  "id": "f0eb6a5ba135",
  "collected_at": "2026-10-02T13:51:33",
  "title": "Two Rust Clients for Gemma 4: Calling the Endpoint vs. Calling the MCP Server 🦀 - DEV Community",
  "body_markdown": "This article provides a step by step guide to two small Rust CLIs that ask a self-hosted Gemma 4 E2B the same question. The first calls the model's OpenAI-compatible HTTP endpoint directly. The second is an MCP client: it launches the rig's own MCP server and asks through its tools.\nhttps://github.com/xbill9/gemma-rust\nhttps://github.com/xbill9/gemma-rust-mcp\nWhat is this project trying to Do?\nBoth CLIs are demos, and their output is read by an audience. So neither hides anything behind a\n--verbose\nflag: every run prints the target, the health check, the request, the answer, the model's reasoning, token counts, latency, and whatever the server says about itself.\nThey run against two very different deployments of the same model with the same code:\nlocal\n: llama.cpp's\nllama-server\non a 2021-era laptop GPU, a GTX 1650 Ti with 4 GiB, no auth\nCloud Run\n: vLLM on an NVIDIA L4, behind Google Cloud IAM\nThe interesting part is what changes when the same question goes through MCP instead of HTTP. It is not the answer.\nWhy Two Clients?\nBecause they answer two different questions.\ngemma-rust shows what the model said.\nOne HTTP call, the raw OpenAI-style response, every field printed.\ngemma-rust-mcp shows what an agent sees.\nAn MCP client like Claude Code never touches the endpoint. It calls tools, and gets back whatever those tools choose to report. Writing a second client in Rust — one that is not Claude Code and not the Python SDK the servers were built with — is the fastest way to find out what those servers actually return.\nNeither one starts, stops or deploys anything. The rigs do that.\nHow Does This All Fit Together?\ngemma-rust ─────── HTTP (reqwest) ───────────────────────┐\n                                                           ├──▶  llama-server   GTX 1650 Ti, local\n                                                           │     vLLM           NVIDIA L4, Cloud Run\n  gemma-rust-mcp ─── MCP over stdio (rmcp) ──▶ server.py ──┘\n                                               (Python, the rig's own)\nEnter fullscreen mode\nExit fullscreen mode\nThe MCP path makes the same HTTP call in the end. It just makes it from inside a Python process that the Rust client launched, and hands back markdown instead of JSON.\nWhere do I start?\nThe strategy for building the two clients is an incremental step by step approach.\nFirst, a model server is brought up locally and checked with\ncurl\n. Then the HTTP client is built and validated against it, including the two ways Gemma 4 returns an empty answer from a healthy server. The same binary is then pointed at Cloud Run.\nThen the rig's Python MCP server is installed, the MCP client is built, and the same question goes through the same two servers again — which is where the comparison comes from.\nAt This Point You Should Have…\nA Linux machine with an NVIDIA GPU, a working driver and the CUDA toolkit (\nnvcc\n) — this one is a GTX 1650 Ti, CUDA 13.3\ngit\n,\ncmake\n, a C++ compiler, and\ncurl\nPython 3 — for the rig's MCP server, not for the Rust clients\nA Hugging Face account that has accepted the Gemma license, and the\nhf\nCLI\nOptional: the Google Cloud SDK and a Gemma 4 Cloud Run service, for the cloud half.\nThis article\ndeploys one.\nStep 1 — Install Rust\nUse\nrustup\n:\ncurl\n--proto\n'=https'\n--tlsv1\n.2\n-sSf\nhttps://sh.rustup.rs | sh\nsource\n~/.cargo/env\nrustc\n--version\nEnter fullscreen mode\nExit fullscreen mode\nrustc 1.98.1 (48a229cea 2026-09-01)\nEnter fullscreen mode\nExit fullscreen mode\nAnything recent works. The floors come from the dependencies' own\nrust-version\n:\nCrate\nNeeds Rust\nNeeded by\nreqwest\n0.13.5\n1.85.0\ngemma-rust\nclap\n4.6.6\n1.85\nboth\nrmcp\n3.3.0\n1.88\ngemma-rust-mcp\nBoth crates are edition 2024.\nStep 2 — Build llama.cpp With CUDA\nllama-server\nis the local model server. Build it from source, at the commit the rig runs:\ngit clone https://github.com/ggml-org/llama.cpp ~/llama.cpp\ncd\n~/llama.cpp\ngit checkout 95ef7fc\ncmake\n-B\nbuild\n-DGGML_CUDA\n=\nON\n-DCMAKE_CUDA_ARCHITECTURES\n=\n75\n-DCMAKE_BUILD_TYPE\n=\nRelease\ncmake\n--build\nbuild\n--config\nRelease\n-j\n--target\nllama-server\nls\nbuild/bin/llama-server\nEnter fullscreen mode\nExit fullscreen mode\nbuild/bin/llama-server\nEnter fullscreen mode\nExit fullscreen mode\n75\nis Turing, which is what a GTX 1650 Ti is. Set your own card's compute capability there, or leave the flag off and let CMake detect it.\nStep 3 — Download the Gemma 4 Checkpoint\nThe rig serves Google's QAT q4_0 GGUF of Gemma 4 E2B:\nhf auth login\nhf download google/gemma-4-E2B-it-qat-q4_0-gguf\n--local-dir\n~/models/gemma-4-E2B-it-qat-q4_0\nls\n-l\n~/models/gemma-4-E2B-it-qat-q4_0/\nEnter fullscreen mode\nExit fullscreen mode\n-rw-rw-r-- 1 xbill xbill 3349516256 Sep  3 13:19 gemma-4-E2B_q4_0-it.gguf\nEnter fullscreen mode\nExit fullscreen mode\nIt fits a 4 GiB card because most of the file never leaves the host — the\nprevious article\nmeasures that.\nStep 4 — Start the Model Server\nRun it in the foreground; Ctrl-C is the whole teardown:\n~/llama.cpp/build/bin/llama-server\n\\\n-m\n~/models/gemma-4-E2B-it-qat-q4_0/gemma-4-E2B_q4_0-it.gguf\n\\\n--host\n127.0.0.1\n--port\n8080\n-ngl\n99\n-c\n8192\nEnter fullscreen mode\nExit fullscreen mode\nFrom a second terminal:\ncurl\n-s\nhttp://127.0.0.1:8080/health\nEnter fullscreen mode\nExit fullscreen mode\n{\"status\":\"ok\"}\nEnter fullscreen mode\nExit fullscreen mode\n🟢 That is the whole server side for the local target. The rig wraps this same command as\nmake serve\n, with its flags in\ntpu.env\n.\nStep 5 — Build the HTTP Client\ncd\n~\ngit clone https://github.com/xbill9/gemma-rust\ncd\ngemma-rust\nmake prod\nEnter fullscreen mode\nExit fullscreen mode\nBuilding release...\n    Finished `release` profile [optimized] target(s) in 0.08s\nBinary: target/release/gemma-rust\nEnter fullscreen mode\nExit fullscreen mode\n(That time is an incremental rebuild; a clean one compiles the dependency tree first.) The dependencies are few:\n[dependencies]\nanyhow\n=\n\"1.0.104\"\nclap\n=\n{\nversion\n=\n\"4.6.6\"\n,\nfeatures\n=\n[\n\"derive\"\n,\n\"env\"\n]\n}\nreqwest\n=\n{\nversion\n=\n\"0.13.5\"\n,\ndefault-features\n=\nfalse\n,\nfeatures\n=\n[\n\"blocking\"\n,\n\"json\"\n,\n\"rustls\"\n]\n}\nrustyline\n=\n\"18.0.1\"\nserde\n=\n{\nversion\n=\n\"1.0.229\"\n,\nfeatures\n=\n[\n\"derive\"\n]\n}\nserde_json\n=\n\"1.0.151\"\nEnter fullscreen mode\nExit fullscreen mode\nblocking\nis deliberate.\nOne question, one answer — there is nothing to run concurrently, so there is no async runtime in the client's own code.\nLint is the gate:\nmake lint\nEnter fullscreen mode\nExit fullscreen mode\nLinting code...\n    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.09s\nEnter fullscreen mode\nExit fullscreen mode\nThat is\ncargo clippy --all-targets -- -D warnings\nand\ncargo fmt --check\n.\nmake test\nruns and finds\n0 tests\n: both crates are demos, validated by running them against live servers, which is what the rest of this article does.\nStep 6 — Ask the Local Model\n./target/release/gemma-rust\n\"In one sentence, what is a TPU?\"\nEnter fullscreen mode\nExit fullscreen mode\n== Target ============================================================\n  endpoint               http://127.0.0.1:8080\n  target                 local (llama.cpp rig)\n  auth                   none\n\n== Health ============================================================\n  GET /health            200 OK in 0 ms\n  body                   {\"status\":\"ok\"}\n\n== Model =============================================================\n  served                 /home/xbill/models/gemma-4-E2B-it-qat-q4_0/gemma-4-E2B_q4_0-it.gguf (context 8192 tokens)\n  using                  /home/xbill/models/gemma-4-E2B-it-qat-q4_0/gemma-4-E2B_q4_0-it.gguf (first model the server lists)\n\n== Request ===========================================================\n  POST                   http://127.0.0.1:8080/v1/chat/completions\n  prompt                 In one sentence, what is a TPU?\n  max_tokens             1024\n\n== Answer ============================================================\n  A TPU (Tensor Processing Unit) is a specialized hardware accelerator designed by Google specifically to speed up the computationally intensive matrix operations required for training and running machine learning models.\n\n== Reasoning =========================================================\n  length                 1327 chars\n  1.  **Identify the core concept:** The user wants a one-sentence definition of a TPU (Tensor Processing Unit).\n  ...\n\n== Stats =============================================================\n  finish_reason          stop\n  prompt_tokens          25\n  cached_tokens          7\n  completion_tokens      330\n  total_tokens           355\n  latency (client)       4786 ms\n  tokens/s (client)      69.0  (completion tokens / latency; includes network and prefill)\n\n== Server timings (llama.cpp) ========================================\n  predicted_ms           4615.20\n  predicted_n            330\n  predicted_per_second   71.29\n  prompt_ms              147.38\n  prompt_n               18\n  ...\n\n== Response ==========================================================\n  id                     chatcmpl-rTO4HJkYHsrE4WisG2mileq6Jmxn0R4f\n  model                  /home/xbill/models/gemma-4-E2B-it-qat-q4_0/gemma-4-E2B_q4_0-it.gguf\n  system_fingerprint     b1-95ef7fc\nEnter fullscreen mode\nExit fullscreen mode\n✅ A one-sentence answer, and 1,327 characters of thinking in front of it. Gemma 4 on llama.cpp reasons by default, and that is where most of the 330 completion tokens went.\nWhat the HTTP Client Is Doing\nThree decisions make one code path work on two servers that disagree about almost everything.\nThe model id comes from the server.\nllama.cpp accepts any\nmodel\nvalue; vLLM returns 404 unless it is exactly the served id. The only default that works on both is the first id from\n/v1/models\n:\nlet\nmodel\n=\nserved\n.first\n()\n.and_then\n(|\nm\n|\nm\n[\n\"id\"\n]\n.as_str\n())\n.context\n(\n\"the server listed no models at /v1/models; pass --model\"\n)\n?\n.to_string\n();\nEnter fullscreen mode\nExit fullscreen mode\nBoth reasoning fields are read.\nThe two servers put Gemma's thinking in different places:\n#[derive(Deserialize)]\nstruct\nMessage\n{\ncontent\n:\nOption\n<\nString\n>\n,\n/// Where llama.cpp puts Gemma 4's thinking\nreasoning_content\n:\nOption\n<\nString\n>\n,\n/// Where vLLM puts it\nreasoning\n:\nOption\n<\nString\n>\n,\n}\nEnter fullscreen mode\nExit fullscreen mode\nServer stats are optional and printed generically.\nllama.cpp returns a\ntimings\nobject; vLLM returns none. Both are\nOption<Value>\n, and whichever is present gets its own section.\nAuth follows the host:\n--auth auto\nruns\ngcloud auth print-identity-token\nfor\n*.run.app\nendpoints only, and prints the token's length, never the token.\n🔎 Tip: Two Ways to Get an Empty Reply From a Healthy Server\nUse\n/v1/chat/completions\n, never\n/v1/completions\n.\nOn these instruction-tuned checkpoints the raw completions endpoint returns empty text, which looks exactly like a broken server.\nGive Gemma room to think.\nOn llama.cpp,\ncontent\nstays empty until the thinking closes. Starve it and see:\n./target/release/gemma-rust\n--max-tokens\n32\n\"In one sentence, what is a TPU?\"\n;\necho\n\"exit=\n$?\n\"\nEnter fullscreen mode\nExit fullscreen mode\n== Request ===========================================================\n  POST                   http://127.0.0.1:8080/v1/chat/completions\n  prompt                 In one sentence, what is a TPU?\n  max_tokens             32\n  warning: below 512, Gemma 4 may still be reasoning when it hits the limit and return an empty answer\n\n== Answer ============================================================\n  (empty) The model was still reasoning when it stopped. This is Gemma 4 thinking, not a broken server: raise --max-tokens.\n\n== Reasoning =========================================================\n  length                 116 chars\n  1.  **Analyze the Request:** The user wants a definition of a TPU (Tensor Processing Unit) in a *single sentence*.\n  2\n\n== Stats =============================================================\n  finish_reason          length\n  The reply hit max_tokens and is cut off.\nexit=2\nEnter fullscreen mode\nExit fullscreen mode\nfinish_reason: length\n, empty\ncontent\n, non-empty reasoning. The CLI says what happened and\nexits 2\n, so a script cannot mistake an empty answer for success. That is why\n--max-tokens\ndefaults to 1024.\nWhat Does the Server Say About Itself?\n--status\nskips the question and probes every endpoint either server might offer:\n./target/release/gemma-rust\n--status\nEnter fullscreen mode\nExit fullscreen mode\n== Server ============================================================\n  GET /version           404 Not Found (not served by this server)\n  GET /props             200 OK in 0 ms\n  llama.cpp build        b1-95ef7fc\n  model_ftype            Q4_0\n  n_ctx                  8192\n  total_slots            1\n  modalities             text only\n\n== Model details =====================================================\n  GET /v1/models         200 OK in 0 ms\n  n_ctx_train            131072\n  n_embd                 1536\n  n_params               4628569635 (4.63 B parameters)\n  size                   3333699724 (3.33 GB on disk)\n\n== Slots =============================================================\n  GET /slots             200 OK in 0 ms\n  slots                  1 (0 busy)\n\n== Metrics ===========================================================\n  GET /metrics           200 OK in 0 ms\n  predicted_tokens_seconds 67.523\n  requests_processing    0\n  ...\nEnter fullscreen mode\nExit fullscreen mode\nA 404 is reported, not an error.\n/version\nis vLLM's;\n/props\nand\n/slots\nare llama.cpp's. Each server answers half the probes, and the list of which ones is itself useful.\nStep 7 — Point It at Cloud Run\nSame binary, different endpoint. If you deployed the Cloud Run rig, its Makefile prints the URL:\nGEMMA_ENDPOINT\n=\n$(\nmake\n-s\n-C\n~/gemma4-dev/gpu-2B-cloudrun-devops-agent endpoint\n)\n\\\n./target/release/gemma-rust\n\"In one sentence, what is a TPU?\"\nEnter fullscreen mode\nExit fullscreen mode\n== Target ============================================================\n  endpoint               https://<your-service>.a.run.app\n  target                 Cloud Run\n  auth                   bearer token from gcloud auth print-identity-token (845 chars, not shown)\n\n== Health ============================================================\n  Cloud Run scales to zero: the first request can take minutes while a GPU instance starts.\n  GET /health            200 OK in 185 ms\n\n== Model =============================================================\n  served                 /mnt/models/gemma-4-E2B-it (context 16384 tokens)\n  using                  /mnt/models/gemma-4-E2B-it (first model the server lists)\n\n== Answer ============================================================\n  A TPU (Tensor Processing Unit) is a specialized integrated circuit designed to accelerate machine learning workloads, particularly those involving large matrix multiplications common in deep learning.\n\n== Reasoning =========================================================\n  (none returned)\n\n== Stats =============================================================\n  finish_reason          stop\n  prompt_tokens          18\n  cached_tokens          -\n  completion_tokens      31\n  total_tokens           49\n  latency (client)       662 ms\n  tokens/s (client)      46.8  (completion tokens / latency; includes network and prefill)\n\n== Response ==========================================================\n  id                     chatcmpl-a870c80a585e2371\n  model                  /mnt/models/gemma-4-E2B-it\n  system_fingerprint     vllm-0.26.0-a3e182ca\nEnter fullscreen mode\nExit fullscreen mode\n🟢 31 tokens and no reasoning — vLLM does not think by default here — so the round trip is 662 ms. The service is\n--no-allow-unauthenticated\n; without a token the CLI explains itself instead of dumping Google's error page:\nGET /health            403 Forbidden in 304 ms\n  body                   (299 bytes, not shown)\n  The server rejected the request's credentials. Cloud Run needs an identity token from an account with roles/run.invoker (`gcloud auth print-identity-token`).\nEnter fullscreen mode\nExit fullscreen mode\nWhat the one code path had to absorb:\nlocal llama.cpp\nCloud Run vLLM 0.26\nauth\nnone\nidentity token (403 without)\nmodel\nfield\nany value\nexact served id (404 otherwise)\nreasoning field\nreasoning_content\nreasoning\nthinks by default\nyes\nno\nserver stats\ntimings\nobject\nnone\ncontext\n8192\n16384\nStep 8 — Install the Rig's MCP Server\nThe MCP client does not talk to the model. It launches a rig's\nserver.py\n, so the rigs come next:\ngit clone https://github.com/xbill9/gemma4-dev ~/gemma4-dev\nmake\n-C\n~/gemma4-dev/local-llamacpp-1650ti-2b-q4_0\ninstall\npython3\n-c\n\"import importlib.metadata as m;print('mcp', m.version('mcp'))\"\nEnter fullscreen mode\nExit fullscreen mode\nmcp 2.2.0\nEnter fullscreen mode\nExit fullscreen mode\nThe rig servers need\nmcp>=2\n— the Python SDK line where\nFastMCP\nbecame\nMCPServer\n. They install into the system\npython3\n; if yours refuses system-wide installs, use a virtualenv and point the client at it with\n--python\nor\nGEMMA_PYTHON\n.\nThe rig reads its model path and\nllama-server\nlocation from\ntpu.env\n. A real environment variable wins over that file, so set\nMODEL_PATH\nand\nLLAMA_SERVER_BIN\nif yours live somewhere else.\nStep 9 — Build the MCP Client\ncd\n~\ngit clone https://github.com/xbill9/gemma-rust-mcp\ncd\ngemma-rust-mcp\nmake prod\nEnter fullscreen mode\nExit fullscreen mode\nBuilding release...\n    Finished `release` profile [optimized] target(s) in 0.04s\nBinary: target/release/gemma-rust-mcp\nEnter fullscreen mode\nExit fullscreen mode\nThe feature flags are the part to get right.\nrmcp\n's defaults are\nbase64\n,\nmacros\nand\nserver\n— a server's feature set. A client has to ask for\nclient\nand a transport by name:\n[dependencies]\nanyhow\n=\n\"1.0.104\"\nclap\n=\n{\nversion\n=\n\"4.6.6\"\n,\nfeatures\n=\n[\n\"derive\"\n,\n\"env\"\n]\n}\nrmcp\n=\n{\nversion\n=\n\"3.3.0\"\n,\nfeatures\n=\n[\n\"client\"\n,\n\"transport-child-process\"\n]\n}\nrustyline\n=\n\"18.0.1\"\nserde\n=\n\"1.0.229\"\nserde_json\n=\n\"1.0.151\"\ntokio\n=\n{\nversion\n=\n\"1.53.1\"\n,\nfeatures\n=\n[\n\"macros\"\n,\n\"rt-multi-thread\"\n,\n\"process\"\n,\n\"time\"\n]\n}\nEnter fullscreen mode\nExit fullscreen mode\nFeature\nWhat it brings\nclient\nServiceExt::serve\non the client side,\ncall_tool\n,\nlist_all_tools\ntransport-child-process\nTokioChildProcess\n: spawn a server, speak MCP over its stdin/stdout\nmake lint\nis clean here too, and\nmake test\nagain finds\n0 tests\n.\nThe MCP Client in Four Moves\nLaunch the server as a child process.\nThe rig opens its files by relative path, so the working directory matters. Stderr is piped so the server's own log can be printed at the end:\nlet\nmut\ncmd\n=\nCommand\n::\nnew\n(\n&\nargs\n.python\n);\ncmd\n.arg\n(\n\"server.py\"\n)\n.current_dir\n(\n&\ndir\n);\nlet\n(\ntransport\n,\nstderr\n)\n=\nTokioChildProcess\n::\nbuilder\n(\ncmd\n)\n.stderr\n(\nStdio\n::\npiped\n())\n.spawn\n()\n?\n;\nEnter fullscreen mode\nExit fullscreen mode\nHandshake.\nThe client handler is\n()\n— this client has no callbacks to offer the server:\nlet\nclient\n=\ntokio\n::\ntime\n::\ntimeout\n(\ntimeout\n,\n()\n.serve\n(\ntransport\n))\n.await\n??\n;\nEnter fullscreen mode\nExit fullscreen mode\nList the tools\n, then\ncall one\n:\nlet\ntools\n=\nclient\n.list_all_tools\n()\n.await\n?\n;\nlet\nparams\n=\nCallToolRequestParams\n::\nnew\n(\nname\n)\n.with_arguments\n(\narguments\n);\nlet\nresult\n=\nclient\n.call_tool\n(\nparams\n)\n.await\n?\n;\nEnter fullscreen mode\nExit fullscreen mode\n💡\nCallToolRequestParams\nis\n#[non_exhaustive]\n, so a struct literal will not compile. Use the constructor and the builder method.\nStep 10 — Ask Through MCP\n./target/release/gemma-rust-mcp\n\"In one sentence, what is a TPU?\"\nEnter fullscreen mode\nExit fullscreen mode\n== MCP server ========================================================\n  rig                    local-llamacpp-1650ti-2b-q4_0\n  command                python3 server.py\n  working dir            /home/xbill/gemma4-dev/local-llamacpp-1650ti-2b-q4_0\n  transport              stdio (child process)\n  pid                    334541\n  initialize             ok in 793 ms\n\n== Server info =======================================================\n  name                   local-llamacpp-1650ti-2b-q4_0\n  version                (empty)\n  protocol               2025-11-25\n  capabilities           tools, resources, prompts\n\n== Tools =============================================================\n  tools/list             7 tools in 2 ms\n  * gpu_status           Report the local GPU: name, compute capability, VRAM total/…\n  * model_info           Report the configured checkpoint, where it is, and the resi…\n    start_model_server   Start llama-server on the local GPU. No-op if it is already…\n    stop_model_server    Stop the running llama-server. Teardown is complete — nothi…\n  * model_server_status  Check whether llama-server is up and serving at the known l…\n  * query_model          Send a chat completion to the local endpoint and return the…\n    get_help             List the tools this rig exposes.\n  (* = called by this demo, which only calls read-only tools)\n\n== tools/call gpu_status =============================================\n  arguments              {}\n  latency                14 ms\n  isError                false\n  result:\n    📡 **GPU** — `local-llamacpp-1650ti-2b-q4_0`\n    NVIDIA GeForce GTX 1650 Ti with Max-Q Design, 7.5, 4096 MiB, 1632 MiB, 2101 MiB, 615.71.09\n\n== tools/call model_server_status ====================================\n  latency                27 ms\n  result:\n    ✅ Serving at http://127.0.0.1:8080 (pid 83619). `/health` → 200.\n\n== tools/call query_model ============================================\n  arguments              {\"max_tokens\":1024,\"prompt\":\"In one sentence, what is a TPU?\"}\n  latency                4608 ms\n  isError                false\n  result:\n    ✅ **Reply**\n\n    A TPU (Tensor Processing Unit) is a specialized integrated circuit developed by Google designed to accelerate machine learning workloads, specifically the complex matrix multiplications required by neural networks, significantly speeding up training and inference.\n\n    ---\n    _(plus 1173 chars of reasoning, suppressed)_\n    prompt 25 tok · completion 326 tok · 71.3 tok/s\n\n== Server log (stderr) ===============================================\n  2026-09-11 16:27:09,682 INFO HTTP Request: GET http://127.0.0.1:8080/health \"HTTP/1.1 200 OK\"\n  2026-09-11 16:27:14,292 INFO HTTP Request: POST http://127.0.0.1:8080/v1/chat/completions \"HTTP/1.1 200 OK\"\nEnter fullscreen mode\nExit fullscreen mode\n✅ Same model, same kind of answer. What came back around it is completely different: the GPU and its memory, the server's health, and a reply formatted for an agent to read — with the reasoning reduced to its length.\nThe server log at the bottom shows the HTTP call the tool made on the client's behalf. That is the \"same servers\" arrow in the diagram, visible.\nOnly Read-Only Tools — a Hard Rule\nThe rig servers also offer tools that start and stop the model server, and on Cloud Run, deploy, destroy and rescale a billed GPU service. The client calls a fixed list and nothing else:\n/// Read-only tools called before the query. Never add a tool that deploys, destroys,\n/// scales, starts, or stops anything: those are billed or destructive.\nfn\nstatus_tools\n(\nself\n)\n->\n&\n'static\n[\n&\n'static\nstr\n]\n{\nmatch\nself\n{\nRig\n::\nLocal\n=>\n&\n[\n\"gpu_status\"\n,\n\"model_server_status\"\n,\n\"model_info\"\n],\nRig\n::\nCloudrun\n=>\n&\n[\n\"cloudrun_status\"\n,\n\"cloudrun_get_system_status\"\n,\n\"cloudrun_get_model_details\"\n,\n],\n}\n}\nEnter fullscreen mode\nExit fullscreen mode\nThere is no \"call any tool\" option, on the command line or in interactive mode. A demo that can be talked into\ncloudrun_destroy\nis not a demo anyone should run in front of an audience.\n🔎 Tip: Failure Is in the Text, Not the Protocol\nMCP has an\nisError\nflag on every tool result. These rig tools never set it. They report failure as markdown that starts with\n❌\n, and a reasoning-only reply as markdown that starts with\n📡\n:\n./target/release/gemma-rust-mcp\n--no-status\n--max-tokens\n32\n\"In one sentence, what is a TPU?\"\n;\necho\n\"exit=\n$?\n\"\nEnter fullscreen mode\nExit fullscreen mode\n== tools/call query_model ============================================\n  arguments              {\"max_tokens\":32,\"prompt\":\"In one sentence, what is a TPU?\"}\n  latency                499 ms\n  isError                false\n  result:\n    📡 **Reasoning only — no answer yet.** `finish_reason: length` after 32 tokens, all of them thinking.\n\n    This is Gemma 4 reasoning, not a broken server. Re-run with a larger `max_tokens` (currently 32).\n    ...\n  No answer: the model was still reasoning when it stopped. Raise --max-tokens.\nexit=2\nEnter fullscreen mode\nExit fullscreen mode\nisError false\n, and still not an answer. A client that trusts the protocol flag reports success here. This one reads the first character of the text, and exits 2 exactly like the HTTP client does.\nMCP Against Cloud Run\n./target/release/gemma-rust-mcp\n--rig\ncloudrun\n\"In one sentence, what is a TPU?\"\nEnter fullscreen mode\nExit fullscreen mode\n== MCP server ========================================================\n  rig                    gpu-2B-cloudrun-devops-agent\n  initialize             ok in 3219 ms\n\n== Server info =======================================================\n  name                   Self-Hosted vLLM DevOps Agent\n  protocol               2025-11-25\n\n== Tools =============================================================\n  tools/list             27 tools in 2 ms\n  ...\n\n== tools/call cloudrun_query_gemma4_with_stats =======================\n  arguments              {\"prompt\":\"In one sentence, what is a TPU?\"}\n  latency                1333 ms\n  isError                false\n  result:\n    ### 📊 Performance Stats\n    - **Model:** `/mnt/models/gemma-4-E2B-it`\n    - **Time to First Token (TTFT):** `0.086s`\n    - **Total Generation Time:** `0.688s`\n    - **Tokens per Second:** `53.11 tokens/s`\n    - **Total Tokens (approx.):** `32`\n\n    ### 💬 Model Response\n    A TPU (Tensor Processing Unit) is a specialized type of integrated circuit designed to accelerate machine learning workloads, particularly those involving tensor operations common in deep learning.<turn|>\n\n== Server log (stderr) ===============================================\n  ... INFO - 📡 Automatically discovered vLLM at: https://<your-service>.a.run.app\n  2026-09-11 11:41:36,754 - httpx - INFO - HTTP Request: GET https://<your-service>.a.run.app/health \"HTTP/1.1 200 OK\"\n  2026-09-11 11:41:37,399 - httpx2 - INFO - HTTP Request: GET https://<your-service>.a.run.app/v1/models \"HTTP/1.1 200 OK\"\n  2026-09-11 11:41:37,479 - httpx2 - INFO - HTTP Request: POST https://<your-service>.a.run.app/v1/chat/completions \"HTTP/1.1 200 OK\"\nEnter fullscreen mode\nExit fullscreen mode\nThree things the HTTP client could not have shown:\nTTFT.\nThe tool streams, so it measures time to first token — 0.086 s. The HTTP client makes one non-streaming call and sees only the total.\nThe cost of the extra hop.\nThe server log shows the tool doing a\nGET /health\nand a\nGET /v1/models\nbefore its\nPOST\n. The HTTP client does those once per session; this tool does them on every call.\nA bug, in the server.\nThe answer ends in\n<turn|>\n, Gemma's end-of-turn marker, leaked by the rig's streaming tool. The client prints what the tool returned — which is how the bug was found.\nCompare and Contrast\nThe same question, through the same two servers:\n🦀 gemma-rust (HTTP)\n🦀 gemma-rust-mcp (MCP)\nWhat you get\n🥇 the raw OpenAI-style response\nwhat the tool chooses to report, as markdown\nReasoning\n🥇 full text\nlocal: its length only; Cloud Run: none\nToken counts\n🥇 exact, from\nusage\n, incl. cached\nlocal: exact; Cloud Run: approximate\nRig status\nHTTP probes of the server\n🥇 GPU, model and deployment, from the rig's tools\n-i\nkeeps the conversation\n🥇 yes\nno — the tool takes one prompt\nRuns on its own\n🥇 yes\nneeds Python 3,\nmcp>=2\nand the rig's\nserver.py\nAnd what it cost, measured:\n🦀 gemma-rust (HTTP)\n🦀 gemma-rust-mcp (MCP)\nLocal query\n4786 ms, 330 tokens\n4608 ms, 326 tokens\nCloud Run query\n🥇 662 ms\n1333 ms\nBefore the first query, Cloud Run\n🥇 185 ms\nGET /health\n, plus the token fetch\n3219 ms\ninitialize\nCrates in\nCargo.lock\n186\n🥇 121\nRelease binary\n8.5M\n🥇 6.5M\nsrc/main.rs\n777 lines\n🥇 542 lines\nLocally, it is a tie.\n4786 ms for 330 tokens against 4608 ms for 326: the model's thinking dominates both, and one sample each cannot separate the MCP layer from the difference between one Gemma reply and the next.\nOn Cloud Run, the hop shows.\n1333 ms against 662 ms is 671 ms (arithmetic), roughly the two extra round trips the tool makes before it asks. Session start is where MCP really pays: 3219 ms to launch Python, import the SDK and discover the service URL through\ngcloud\n, before the first tool call.\nThe smaller crate is the MCP one\n, because it speaks stdio and never opens a TLS connection.\nreqwest\nwith\nrustls\nbrings a TLS stack; the MCP client leaves the HTTPS to the Python server.\nSo, Which One?\nUse the HTTP client to see the model.\nReasoning, exact token counts, cached tokens, server timings, the raw JSON with\n--raw\n. When the question is \"what did Gemma do\", this is the one.\nUse the MCP client to see the agent's view.\nIt is the fastest way to test an MCP server from outside the SDK it was written with, and it found two things no unit test did: a failure flag that is never set, and an end-of-turn marker leaking into answers.\nNeither half of that is about Rust being fast. The work is a model thinking on a GPU. Rust earns its place here with one static binary per client, a type for every response field, and a compiler that notices when vLLM and llama.cpp disagree about where the reasoning goes.\nSummary\nThe goal of this article was to ask Gemma 4 E2B the same question from Rust in two ways — directly over its HTTP endpoint, and through the rig's MCP server — on a local llama.cpp GPU and on Cloud Run. The key to the solution was printing everything each path returns, which turns two small clients into a side-by-side view of what a model says and what an agent sees. The results were:\n🟢 One HTTP code path served both targets by taking the model id from\n/v1/models\nand reading both reasoning fields\n🟢 Both clients report an empty answer from a thinking-starved Gemma 4 as exit 2, not as success\n🟢 The MCP client added GPU, model and deployment status, plus TTFT on Cloud Run, from the rig's own tools\n❌ The MCP path loses the reasoning text and, on Cloud Run, exact token counts\n⚠️ On Cloud Run the MCP query took 1333 ms against 662 ms over HTTP, and 3219 ms to initialize\n⚠️ The rig tools report failure in text with\nisError\nfalse, and the Cloud Run tool leaks\n<turn|>\nScope: one laptop with a GTX 1650 Ti Max-Q (4 GiB, driver 615.71.09) running llama.cpp\nb1-95ef7fc\nwith the Gemma 4 E2B QAT q4_0 GGUF, and one Cloud Run service on an NVIDIA L4 running vLLM 0.26.0. One run per client per target, so each figure is a single sample and model replies vary from run to run; the local runs were captured at 16:27 on 2026-09-11 and the Cloud Run runs earlier the same day, and the HTTP and MCP runs are separate requests, not the same one observed twice. Rust 1.98.1,\nrmcp\n3.3.0,\nreqwest\n0.13.5, Python 3.14.7 with\nmcp\n2.2.0.\nThe strategy for using Rust to call Gemma 4 over HTTP and over MCP was validated with an incremental step by step approach.\nReferences\ngemma-rust | GitHub\ngemma-rust-mcp | GitHub\ngemma4-dev rigs | GitHub\nrmcp — the official Rust MCP SDK\nllama.cpp\nGemma 4 on an 2021 4 GB Laptop GPU: QAT Takes It From 9.5 GiB to 1.6\n2B Gemma 4 Deployment with Cloud Run, NVIDIA L4, MCP SDK 2.x, and Claude Code",
  "metadata": {
    "title": "Two Rust Clients for Gemma 4: Calling the Endpoint vs. Calling the MCP Server 🦀 - DEV Community",
    "description": "Step by step: a Rust HTTP client (reqwest) and a Rust MCP client (rmcp) ask Gemma 4 E2B the same question on a local llama.cpp GPU and on Cloud Run — what each one can see, and what each one costs. Tagged with rust, mcp, gemma, llamacpp.",
    "keywords": "rust, mcp, gemma, llamacpp, software, coding, development, engineering, inclusive, community",
    "author": null,
    "og:type": "article",
    "og:url": "https://dev.to/gde/two-rust-clients-for-gemma-4-calling-the-endpoint-vs-calling-the-mcp-server-3kj4",
    "og:title": "Two Rust Clients for Gemma 4: Calling the Endpoint vs. Calling the MCP Server 🦀",
    "og:description": "Step by step: a Rust HTTP client (reqwest) and a Rust MCP client (rmcp) ask Gemma 4 E2B the same question on a local llama.cpp GPU and on Cloud Run — what each one can see, and what each one costs.",
    "og:site_name": "DEV Community",
    "og:image": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fraw.githubusercontent.com%2Fxbill9%2Fgemma-rust%2Fmain%2Fdocs%2Fdevto-cover.8693fe91.jpg",
    "twitter:site": "@thepracticaldev",
    "twitter:creator": "@",
    "twitter:title": "Two Rust Clients for Gemma 4: Calling the Endpoint vs. Calling the MCP Server 🦀",
    "twitter:description": "Step by step: a Rust HTTP client (reqwest) and a Rust MCP client (rmcp) ask Gemma 4 E2B the same question on a local llama.cpp GPU and on Cloud Run — what each one can see, and what each one costs.",
    "twitter:card": "summary_large_image",
    "twitter:widgets:new-embed-design": "on",
    "twitter:image:src": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fraw.githubusercontent.com%2Fxbill9%2Fgemma-rust%2Fmain%2Fdocs%2Fdevto-cover.8693fe91.jpg"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [
    {
      "file": "attachments/f0eb6a5ba135/code/body-000.txt",
      "chars": 393,
      "language": "text"
    },
    {
      "file": "attachments/f0eb6a5ba135/code/body-001.txt",
      "chars": 99,
      "language": "text"
    },
    {
      "file": "attachments/f0eb6a5ba135/code/body-002.txt",
      "chars": 36,
      "language": "text"
    },
    {
      "file": "attachments/f0eb6a5ba135/code/body-003.txt",
      "chars": 271,
      "language": "text"
    },
    {
      "file": "attachments/f0eb6a5ba135/code/body-004.txt",
      "chars": 23,
      "language": "text"
    },
    {
      "file": "attachments/f0eb6a5ba135/code/body-005.txt",
      "chars": 147,
      "language": "text"
    },
    {
      "file": "attachments/f0eb6a5ba135/code/body-006.txt",
      "chars": 74,
      "language": "text"
    },
    {
      "file": "attachments/f0eb6a5ba135/code/body-007.txt",
      "chars": 149,
      "language": "text"
    },
    {
      "file": "attachments/f0eb6a5ba135/code/body-008.txt",
      "chars": 37,
      "language": "text"
    },
    {
      "file": "attachments/f0eb6a5ba135/code/body-009.txt",
      "chars": 16,
      "language": "text"
    },
    {
      "file": "attachments/f0eb6a5ba135/code/body-010.txt",
      "chars": 76,
      "language": "text"
    },
    {
      "file": "attachments/f0eb6a5ba135/code/body-011.txt",
      "chars": 116,
      "language": "text"
    },
    {
      "file": "attachments/f0eb6a5ba135/code/body-012.txt",
      "chars": 294,
      "language": "text"
    },
    {
      "file": "attachments/f0eb6a5ba135/code/body-013.txt",
      "chars": 10,
      "language": "text"
    },
    {
      "file": "attachments/f0eb6a5ba135/code/body-014.txt",
      "chars": 88,
      "language": "text"
    },
    {
      "file": "attachments/f0eb6a5ba135/code/body-015.txt",
      "chars": 62,
      "language": "text"
    },
    {
      "file": "attachments/f0eb6a5ba135/code/body-016.txt",
      "chars": 2244,
      "language": "text"
    },
    {
      "file": "attachments/f0eb6a5ba135/code/body-017.txt",
      "chars": 159,
      "language": "text"
    },
    {
      "file": "attachments/f0eb6a5ba135/code/body-018.txt",
      "chars": 216,
      "language": "text"
    },
    {
      "file": "attachments/f0eb6a5ba135/code/body-019.txt",
      "chars": 94,
      "language": "text"
    },
    {
      "file": "attachments/f0eb6a5ba135/code/body-020.txt",
      "chars": 904,
      "language": "text"
    },
    {
      "file": "attachments/f0eb6a5ba135/code/body-021.txt",
      "chars": 37,
      "language": "text"
    },
    {
      "file": "attachments/f0eb6a5ba135/code/body-022.txt",
      "chars": 947,
      "language": "text"
    },
    {
      "file": "attachments/f0eb6a5ba135/code/body-023.txt",
      "chars": 146,
      "language": "text"
    },
    {
      "file": "attachments/f0eb6a5ba135/code/body-024.txt",
      "chars": 1621,
      "language": "text"
    },
    {
      "file": "attachments/f0eb6a5ba135/code/body-025.txt",
      "chars": 257,
      "language": "text"
    },
    {
      "file": "attachments/f0eb6a5ba135/code/body-026.txt",
      "chars": 194,
      "language": "text"
    },
    {
      "file": "attachments/f0eb6a5ba135/code/body-027.txt",
      "chars": 10,
      "language": "text"
    },
    {
      "file": "attachments/f0eb6a5ba135/code/body-028.txt",
      "chars": 84,
      "language": "text"
    },
    {
      "file": "attachments/f0eb6a5ba135/code/body-029.txt",
      "chars": 120,
      "language": "text"
    },
    {
      "file": "attachments/f0eb6a5ba135/code/body-030.txt",
      "chars": 326,
      "language": "text"
    },
    {
      "file": "attachments/f0eb6a5ba135/code/body-031.txt",
      "chars": 183,
      "language": "text"
    },
    {
      "file": "attachments/f0eb6a5ba135/code/body-032.txt",
      "chars": 73,
      "language": "text"
    },
    {
      "file": "attachments/f0eb6a5ba135/code/body-033.txt",
      "chars": 164,
      "language": "text"
    },
    {
      "file": "attachments/f0eb6a5ba135/code/body-034.txt",
      "chars": 66,
      "language": "text"
    },
    {
      "file": "attachments/f0eb6a5ba135/code/body-035.txt",
      "chars": 2765,
      "language": "text"
    },
    {
      "file": "attachments/f0eb6a5ba135/code/body-036.txt",
      "chars": 464,
      "language": "text"
    },
    {
      "file": "attachments/f0eb6a5ba135/code/body-037.txt",
      "chars": 110,
      "language": "text"
    },
    {
      "file": "attachments/f0eb6a5ba135/code/body-038.txt",
      "chars": 533,
      "language": "text"
    },
    {
      "file": "attachments/f0eb6a5ba135/code/body-039.txt",
      "chars": 81,
      "language": "text"
    },
    {
      "file": "attachments/f0eb6a5ba135/code/body-040.txt",
      "chars": 1676,
      "language": "text"
    }
  ],
  "images": [],
  "audio": [],
  "videos": [],
  "links": [
    {
      "href": "https://github.com/xbill9/gemma-rust",
      "label": "https://github.com/xbill9/gemma-rust",
      "type": "a"
    },
    {
      "href": "https://github.com/xbill9/gemma-rust-mcp",
      "label": "https://github.com/xbill9/gemma-rust-mcp",
      "type": "a"
    },
    {
      "href": "https://dev.to/gde/2b-gemma-4-deployment-with-cloud-run-nvidia-l4-mcp-sdk-2x-and-claude-code-4ml3",
      "label": "This article",
      "type": "a"
    },
    {
      "href": "https://dev.to/gde/gemma-4-on-an-old-4-gb-laptop-gpu-qat-takes-it-from-95-gib-to-16-b5l",
      "label": "previous article",
      "type": "a"
    },
    {
      "href": "https://github.com/xbill9/gemma4-dev",
      "label": "gemma4-dev rigs | GitHub",
      "type": "a"
    },
    {
      "href": "https://github.com/modelcontextprotocol/rust-sdk",
      "label": "rmcp — the official Rust MCP SDK",
      "type": "a"
    },
    {
      "href": "https://github.com/ggml-org/llama.cpp",
      "label": "llama.cpp",
      "type": "a"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:48:32Z",
  "source_url": "https://dev.to/gde/two-rust-clients-for-gemma-4-calling-the-endpoint-vs-calling-the-mcp-server-3kj4",
  "papers": [],
  "actual_video_files": [],
  "body_chars": 31419,
  "expected_body_images": 0,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/f0eb6a5ba135/pages/source.html",
  "local_html": "attachments/f0eb6a5ba135/pages/article.html",
  "body_base_path": "attachments/f0eb6a5ba135/pages/",
  "body_markdown_path": "attachments/f0eb6a5ba135/pages/article.md"
}