{
  "source_file": "0930",
  "source_platform": "Dev.to",
  "author": "xbill",
  "title_listed": "Gemma 4 on an Amazon SageMaker Endpoint: AWS CLI, NVIDIA L4, and an MCP Server",
  "keywords": "",
  "domain": "大模型算法",
  "content_type": "工程实践",
  "is_paid_flag": "否",
  "url": "https://dev.to/gde/gemma-4-on-an-amazon-sagemaker-endpoint-aws-cli-nvidia-l4-and-an-mcp-server-5cdd",
  "publish_date": "2026-09-26",
  "corpus_stage": "",
  "notes": "",
  "id": "b0042d8b19d7",
  "collected_at": "2026-10-02T13:51:09",
  "title": "Gemma 4 on an Amazon SageMaker Endpoint: AWS CLI, NVIDIA L4, and an MCP Server - DEV Community",
  "body_markdown": "This article provides a step by step deployment guide for Gemma 4 E2B to an Amazon SageMaker hosted GPU enabled system. A suite of Python MCP tools is built to simplify management of the vLLM hosted deployment with Claude Code.\nhttps://github.com/xbill9/sagemaker-gemma\nWhat is this project trying to Do?\nThis project serves Gemma 4 E2B from a SageMaker real-time endpoint on one NVIDIA L4 GPU, using the vLLM container AWS publishes for SageMaker. Every AWS call is a plain\naws\nCLI command, so each step can be run by hand or by the MCP server.\nA SageMaker real-time endpoint is a managed HTTPS inference server. SageMaker places the container on a GPU instance, health-checks it, routes requests to it and writes its logs to CloudWatch. There is no instance to patch, no security group to open and no load balancer to build.\nWhere do I start?\nThe strategy for starting MCP development for model management is a incremental step by step approach.\nFirst, the basic development environment is setup with the required system variables and a working Claude Code configuration.\nThen, the Python MCP server is brought up over stdio and validated with Claude Code in the local environment. The deployment follows as eight steps, each shown as the raw\naws\ncommand and the MCP tool that runs it.\nAt This Point You Should Have…\nAn AWS account and the AWS CLI v2, signed in with\naws login\nA SageMaker endpoint quota of at least 1 for a single-L4 instance type (\nml.g6.xlarge\n,\nml.g6.2xlarge\nor\nml.g6.4xlarge\n)\nPython 3.11 or newer with\nmcp\n2.x\nClaude Code or Gemini CLI installed and working\njq\nfor reading JSON replies\nSetup the Basic Environment\nClone the repository and install the one requirement into the system Python:\ngit clone https://github.com/xbill9/sagemaker-gemma\ncd\nsagemaker-gemma\npython3\n-m\npip\ninstall\n-r\nrequirements.txt\ncp\n.env.example .env\nEnter fullscreen mode\nExit fullscreen mode\n.env\nis gitignored. It holds the settings every tool reads:\ncat\n.env\nEnter fullscreen mode\nExit fullscreen mode\n# Copy to .env (gitignored) and edit. sm.py reads it, so the MCP server, CLI and Makefile all see it.\nAWS_REGION=us-east-2\nMODEL_ID=google/gemma-4-E2B-it\nINSTANCE_TYPE=ml.g6.xlarge\nENDPOINT_NAME=gemma-4-e2b\nROLE_NAME=sagemaker-gemma-execution-role\nMAX_MODEL_LEN=8192\n# Leave empty to use the newest SageMaker vLLM image in the region.\nIMAGE_URI=\n# Fallback instance types, highest priority first (same GPU keeps runs comparable).\nINSTANCE_POOLS=ml.g6.xlarge,ml.g6.2xlarge,ml.g6.4xlarge\nEnter fullscreen mode\nExit fullscreen mode\nGemma 4 is Apache-2.0 and ungated on Hugging Face, so no Hugging Face token is needed.\nModel Management Tool with MCP Stdio Transport\nThe simplest MCP transport is stdio: the client launches the server as a local process and talks to it over stdin and stdout. In this project Claude Code is the MCP client. The server is one file,\nserver.py\n, on the MCP Python SDK 2.x:\nmcp\n=\nMCPServer\n(\nRIG_NAME\n)\nREAD_ONLY\n=\nToolAnnotations\n(\nreadOnlyHint\n=\nTrue\n,\nidempotentHint\n=\nTrue\n)\nWRITE\n=\nToolAnnotations\n(\ndestructiveHint\n=\nFalse\n)\nDESTRUCTIVE\n=\nToolAnnotations\n(\ndestructiveHint\n=\nTrue\n)\nEnter fullscreen mode\nExit fullscreen mode\nEvery tool carries one of the three annotations, so a client can tell a status check from a deploy from a delete.\nThe tools do no AWS work themselves. They call\nsm.py\n, which runs each request as an\naws\nCLI subprocess:\ndef\naws\n(\n*\nargs\n:\nstr\n,\nregion\n:\nstr\n|\nNone\n=\nREGION\n,\nparse\n:\nbool\n=\nTrue\n)\n->\nAny\n:\n\"\"\"\nRun `aws <args> --output json` and return the parsed result.\n\"\"\"\ncmd\n=\n[\n\"\naws\n\"\n,\n*\nargs\n,\n\"\n--output\n\"\n,\n\"\njson\n\"\n]\nEnter fullscreen mode\nExit fullscreen mode\nThe CLI renews an\naws login\nsession on its own, so a server that stays up for hours keeps working credentials.\nsm.py\nalso drops any\nAWS_SESSION_TOKEN\nthe server inherits from its parent process, because a static token expires inside a long-running server and outranks the login session.\nRunning the Python Code\nThe project can be linted:\nmake lint\nEnter fullscreen mode\nExit fullscreen mode\nAll checks passed!\n16 files already formatted\nEnter fullscreen mode\nExit fullscreen mode\nand tested:\nmake\ntest\nEnter fullscreen mode\nExit fullscreen mode\n----------------------------------------------------------------------\nRan 16 tests in 0.010s\n\nOK\nEnter fullscreen mode\nExit fullscreen mode\nThe tests replace the\naws\nsubprocess with a fake, so they run offline with no credentials. One of them compares the registered tool set and annotations against a fixed list: a tool that failed to register, or a delete that lost its destructive flag, fails the suite. ✅\nTest the Protocol by Hand\nA client speaks JSON-RPC over stdio. Hold stdin open with\nsleep\n, or the server sees end-of-input and exits before it answers:\n{\nprintf\n'%s\\n'\n\\\n'{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"initialize\",\"params\":{\"protocolVersion\":\"2025-06-18\",\"capabilities\":{},\"clientInfo\":{\"name\":\"probe\",\"version\":\"0\"}}}'\n\\\n'{\"jsonrpc\":\"2.0\",\"method\":\"notifications/initialized\"}'\n\\\n'{\"jsonrpc\":\"2.0\",\"id\":2,\"method\":\"tools/list\",\"params\":{}}'\n;\nsleep\n3\n;\n}\n\\\n| python3 server.py 2>/dev/null\nEnter fullscreen mode\nExit fullscreen mode\nSummarised:\n1 sagemaker-gemma\n2 ['check_quotas', 'delete_endpoint', 'deploy_endpoint', 'find_vllm_image', 'get_deployment_config', 'get_endpoint_logs', 'get_endpoint_status', 'get_help', 'list_endpoints', 'query_model', 'verify_model_health']\nEnter fullscreen mode\nExit fullscreen mode\n🟢 The server answers the handshake and lists 11 tools.\nClaude Code .mcp.json\nClaude Code reads\n.mcp.json\nin the project directory and launches the server with the system\npython3\n:\n{\n\"mcpServers\"\n:\n{\n\"sagemaker-gemma\"\n:\n{\n\"command\"\n:\n\"python3\"\n,\n\"args\"\n:\n[\n\"/home/xbill/sagemaker-gemma/server.py\"\n]\n}\n}\n}\nEnter fullscreen mode\nExit fullscreen mode\nGemini CLI reads the same entry from\n.gemini/settings.json\n.\nValidation with Claude Code\nclaude mcp get sagemaker-gemma\nEnter fullscreen mode\nExit fullscreen mode\nsagemaker-gemma:\n  Scope: Project config (shared via .mcp.json)\n  Status: ✔ Connected\nEnter fullscreen mode\nExit fullscreen mode\nFrom inside Claude Code,\nget_help\nreturns the resolved settings and the order of work:\n### sagemaker-gemma\nServing\n`google/gemma-4-E2B-it`\nwith vLLM on a\n**SageMaker real-time endpoint**\n, managed\nentirely through the aws CLI.\n\n| Setting | Value |\n| --- | --- |\n| Region |\n`us-east-2`\n|\n| Endpoint |\n`gemma-4-e2b`\n|\n| Instance types (priority order) |\n`ml.g6.xlarge`\n,\n`ml.g6.2xlarge`\n,\n`ml.g6.4xlarge`\n|\n| Max model length |\n`8192`\n|\n| Image |\n`newest SageMaker vLLM image (find_vllm_image)`\n|\n| Execution role |\n`sagemaker-gemma-execution-role`\n|\n**Order of work:**\ncheck_quotas → deploy_endpoint → get_endpoint_status until\n`InService`\n(about 10 minutes once an instance is placed) → verify_model_health\n→ query_model → delete_endpoint.\nEnter fullscreen mode\nExit fullscreen mode\nThe steps below follow that order. Each one shows the raw CLI command;\ndeploy_endpoint\nruns Steps 2 to 6 in one call.\nexport\nAWS_REGION\n=\nus-east-2\nAWS_PAGER\n=\nexport\nNAME\n=\ngemma-4-e2b\nexport\nMODEL_ID\n=\ngoogle/gemma-4-E2B-it\nEnter fullscreen mode\nExit fullscreen mode\nStep 1 — Check the Endpoint Quota\nSageMaker endpoint quotas are per instance type, per region, and count instances.\naws service-quotas list-service-quotas\n--service-code\nsagemaker\n\\\n--query\n\"Quotas[?QuotaName=='ml.g6.xlarge for endpoint usage'].Value\"\nEnter fullscreen mode\nExit fullscreen mode\n[\n    1.0\n]\nEnter fullscreen mode\nExit fullscreen mode\nThe\ncheck_quotas\ntool reads the same quota for each fallback type across the US regions:\n| Instance | us-east-1 | us-east-2 | us-west-1 | us-west-2 |\n| --- | ---: | ---: | ---: | ---: |\n| `ml.g6.xlarge` | 1 | 1 | - | 1 |\n| `ml.g6.2xlarge` | 1 | 1 | - | 1 |\n| `ml.g6.4xlarge` | 1 | 1 | - | 1 |\n\nRegions with quota for at least one of these types: us-east-1, us-east-2, us-west-2.\nEnter fullscreen mode\nExit fullscreen mode\nA\n-\nmeans the type is not offered in that region. A\n0\nmeans a quota increase request first.\n🔎 Tip: One Quota Call per Region\nlist-service-quotas\npages through every SageMaker quota in the region, and the Service Quotas API is rate limited per account. Twelve calls at once, one per type per region, came back as:\nAn error occurred (TooManyRequestsException) when calling the ListServiceQuotas operation (reached max retries: 2)\nEnter fullscreen mode\nExit fullscreen mode\ncheck_quotas\nmakes one call per region, one region at a time, and filters the types from that one result.\nsm.py\nalso sets\nAWS_RETRY_MODE=adaptive\nso the CLI backs off and retries.\nStep 2 — Find the vLLM Container\nThe AWS vLLM repository holds pinned release tags such as\n0.30.0-gpu-py312-cu130-ubuntu24.04-sagemaker-v1.1\n, floating aliases such as\n0.30-gpu-py312\n, and\n-soci\nindex tags. Older release lines receive patch rebuilds, so the most recent push can carry an older vLLM. Sort the pinned tags by version:\nTAG\n=\n$(\naws ecr describe-images\n--registry-id\n763104351884\n--repository-name\nvllm\n\\\n--query\n\"imageDetails[].imageTags[]\"\n--output\ntext |\ntr\n'\\t'\n'\\n'\n\\\n|\ngrep\n-E\n--\n'^[0-9]+\\.[0-9]+\\.[0-9]+-.*-sagemaker-v[0-9]+\\.[0-9]+$'\n|\nsort\n-V\n|\ntail\n-1\n)\nexport\nIMAGE\n=\n763104351884.dkr.ecr.\n$AWS_REGION\n.amazonaws.com/vllm:\n$TAG\necho\n$IMAGE\nEnter fullscreen mode\nExit fullscreen mode\n763104351884.dkr.ecr.us-east-2.amazonaws.com/vllm:0.30.0-gpu-py312-cu130-ubuntu24.04-sagemaker-v1.1\nEnter fullscreen mode\nExit fullscreen mode\nThe\nfind_vllm_image\ntool applies the same rule. The image runs vLLM 0.30.0.\nStep 3 — Create the Execution Role\nSageMaker assumes this role to pull the image and write logs. It is created once per account.\naws iam create-role\n--role-name\nsagemaker-gemma-execution-role\n\\\n--assume-role-policy-document\n'{\"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Allow\",\"Principal\":{\"Service\":\"sagemaker.amazonaws.com\"},\"Action\":\"sts:AssumeRole\"}]}'\naws iam attach-role-policy\n--role-name\nsagemaker-gemma-execution-role\n\\\n--policy-arn\narn:aws:iam::aws:policy/AmazonSageMakerFullAccess\nexport\nROLE\n=\n$(\naws iam get-role\n--role-name\nsagemaker-gemma-execution-role\n--query\nRole.Arn\n--output\ntext\n)\nEnter fullscreen mode\nExit fullscreen mode\nOn a second run\ncreate-role\nreports\nEntityAlreadyExists\n;\nget-role\nstill sets\nROLE\n.\ndeploy_endpoint\ncreates the role only when it is missing.\nStep 4 — Create the Model\nA SageMaker model pairs an image with its settings. The container turns each\nSM_VLLM_\nvariable into the matching vLLM flag:\nSM_VLLM_MAX_MODEL_LEN\nbecomes\n--max-model-len\n.\naws sagemaker create-model\n--model-name\n$NAME\n--execution-role-arn\n$ROLE\n\\\n--primary-container\n\"{\n\\\"\nImage\n\\\"\n:\n\\\"\n$IMAGE\n\\\"\n,\n\\\"\nEnvironment\n\\\"\n:{\n\\\"\nSM_VLLM_MODEL\n\\\"\n:\n\\\"\n$MODEL_ID\n\\\"\n,\n\\\"\nSM_VLLM_MAX_MODEL_LEN\n\\\"\n:\n\\\"\n8192\n\\\"\n,\n\\\"\nSM_VLLM_GPU_MEMORY_UTILIZATION\n\\\"\n:\n\\\"\n0.9\n\\\"\n}}\"\nEnter fullscreen mode\nExit fullscreen mode\n{\n\"ModelArn\"\n:\n\"arn:aws:sagemaker:us-east-2:<account-id>:model/gemma-4-e2b\"\n}\nEnter fullscreen mode\nExit fullscreen mode\nStep 5 — Create the Endpoint Config With Fallback Instance Types\nThe endpoint config says where the model runs.\nInstancePools\nlists up to five instance types in priority order, and SageMaker places the first one with a free instance. All three types below carry one L4, so the model sees the same GPU whichever is placed.\naws sagemaker create-endpoint-config\n--endpoint-config-name\n$NAME\n\\\n--production-variants\n\"[{\n\\\"\nVariantName\n\\\"\n:\n\\\"\nAllTraffic\n\\\"\n,\n\\\"\nModelName\n\\\"\n:\n\\\"\n$NAME\n\\\"\n,\n\\\"\nInitialInstanceCount\n\\\"\n:1,\n\\\"\nInstancePools\n\\\"\n:[{\n\\\"\nInstanceType\n\\\"\n:\n\\\"\nml.g6.xlarge\n\\\"\n,\n\\\"\nPriority\n\\\"\n:1},\n                       {\n\\\"\nInstanceType\n\\\"\n:\n\\\"\nml.g6.2xlarge\n\\\"\n,\n\\\"\nPriority\n\\\"\n:2},\n                       {\n\\\"\nInstanceType\n\\\"\n:\n\\\"\nml.g6.4xlarge\n\\\"\n,\n\\\"\nPriority\n\\\"\n:3}],\n\\\"\nContainerStartupHealthCheckTimeoutInSeconds\n\\\"\n:1800,\n\\\"\nModelDataDownloadTimeoutInSeconds\n\\\"\n:1800}]\"\nEnter fullscreen mode\nExit fullscreen mode\n{\n\"EndpointConfigArn\"\n:\n\"arn:aws:sagemaker:us-east-2:<account-id>:endpoint-config/gemma-4-e2b\"\n}\nEnter fullscreen mode\nExit fullscreen mode\nThe two 1800-second timeouts give the container time to download the weights and compile before SageMaker's health check gives up.\n🔎 Tip: A Fallback List Holds Quota for Every Type in It\nWith one Gemma endpoint running on\nml.g6.xlarge\nfrom the three-type list above, a second endpoint asking for\nml.g6.2xlarge\nin the same region was refused:\nResourceLimitExceeded: The account-level service limit 'ml.g6.2xlarge for endpoint usage' is 1 Instances, with current utilization of 1 Instances and a request delta of 1 Instances.\nEnter fullscreen mode\nExit fullscreen mode\nWith a quota of 1 per type, a second endpoint goes in another region, or uses types outside the first endpoint's list.\nStep 6 — Create the Endpoint\naws sagemaker create-endpoint\n--endpoint-name\n$NAME\n--endpoint-config-name\n$NAME\nEnter fullscreen mode\nExit fullscreen mode\n{\n\"EndpointArn\"\n:\n\"arn:aws:sagemaker:us-east-2:<account-id>:endpoint/gemma-4-e2b\"\n}\nEnter fullscreen mode\nExit fullscreen mode\nBilling starts when an instance is placed.\naws sagemaker wait endpoint-in-service --endpoint-name $NAME\nblocks until it is ready;\nget_endpoint_status\nreports the same state and the instance type that was placed:\n✅ `gemma-4-e2b` in `us-east-2`: **InService**\n- Instance: `ml.g6.xlarge`\nEnter fullscreen mode\nExit fullscreen mode\nThe container log is in CloudWatch, and\nget_endpoint_logs\ntails it:\naws logs\ntail\n/aws/sagemaker/Endpoints/\n$NAME\n--follow\nEnter fullscreen mode\nExit fullscreen mode\nStart-up on the L4, in minutes after\ncreate-endpoint\n:\nPhase\nMinutes\nWeights loaded, 9.75 GiB in 82.75 s\n7.4\nKV cache sized, 723,484 tokens\n9.2\nInService\n9.9\n🔎 Tip: A Creating Endpoint Cannot Be Deleted\ncreate-endpoint\ncannot be undone until the endpoint settles. A\ndelete-endpoint\nsent while it is\nCreating\nis refused:\naws: [ERROR]: An error occurred (ValidationException) when calling the DeleteEndpoint operation: Cannot update in-progress endpoint \"arn:aws:sagemaker:us-east-2:<account-id>:endpoint/gemma-4-e2b\".\nEnter fullscreen mode\nExit fullscreen mode\nThe endpoint finishes starting, bills from the moment its instance is placed, and can be deleted once it reaches\nInService\nor\nFailed\n. Check the model ID and instance types before Step 6.\n🔎 Tip: Capacity Is Separate From Quota\nA quota of 1 lets you request one instance; the region still has to have one free. In\nus-east-1\n, two requests for L4 instances each stayed in\nCreating\nfor about 30 minutes and then failed:\nUnable to provision requested ML compute capacity due to InsufficientInstanceCapacity error. Please retry using a different ML instance type or after some time.\nEnter fullscreen mode\nExit fullscreen mode\nThe same request in\nus-east-2\nplaced an instance at once. While SageMaker waits for capacity, no container starts and the CloudWatch log group never appears, which is how\nget_endpoint_logs\ntells a capacity wait from a slow model load. No charge accrues during the wait. When it fails, repeat Steps 4 to 6 in another region where Step 1 shows the same quota.\nStep 7 — Cross Check the Deployed Model\nThe vLLM container accepts an OpenAI chat body on\ninvoke-endpoint\n:\necho\n'{\"messages\":[{\"role\":\"user\",\"content\":\"Why is the sky blue?\"}],\"max_tokens\":256}'\n>\nreq.json\naws sagemaker-runtime invoke-endpoint\n--endpoint-name\n$NAME\n\\\n--content-type\napplication/json\n--body\nfileb://req.json out.json\njq\n-r\n'.choices[0].message.content'\nout.json |\nhead\n-1\njq\n-c\n'{model,usage}'\nout.json\nEnter fullscreen mode\nExit fullscreen mode\n{\n    \"ContentType\": \"application/json\",\n    \"InvokedProductionVariant\": \"AllTraffic\"\n}\nThe sky is blue due to a phenomenon called **Rayleigh scattering**. This process is caused by how sunlight interacts with the Earth's atmosphere.\n{\"model\":\"google/gemma-4-E2B-it\",\"usage\":{\"prompt_tokens\":15,\"total_tokens\":271,\"completion_tokens\":256,\"prompt_tokens_details\":null,\"completion_tokens_details\":null}}\nEnter fullscreen mode\nExit fullscreen mode\nverify_model_health\nsends one short request and checks for a reply:\n✅ model=`google/gemma-4-E2B-it` tokens=2 wall=0.749s reply='ok'\nEnter fullscreen mode\nExit fullscreen mode\nStep 8 — Teardown\nThe endpoint bills by the hour until it is deleted. Deleting the config and the model as well leaves nothing behind.\ndelete_endpoint\nruns all three:\naws sagemaker delete-endpoint\n--endpoint-name\n$NAME\naws sagemaker delete-endpoint-config\n--endpoint-config-name\n$NAME\naws sagemaker delete-model\n--model-name\n$NAME\nEnter fullscreen mode\nExit fullscreen mode\nEach command prints nothing and exits 0.\nlist_endpoints\nconfirms the account is clear:\n### 0 endpoint(s) matching `gemma`\nEnter fullscreen mode\nExit fullscreen mode\nSummary\nThe goal of this article was to deploy Gemma 4 E2B to an Amazon SageMaker real-time endpoint with the AWS CLI and manage it from an MCP server. The key to the solution was the AWS vLLM SageMaker container, which turns the deployment into three\ncreate-\ncalls and makes the endpoint answer OpenAI-style chat requests. The deployment results were:\n🟢 Eight CLI steps take an account from quota check to a serving endpoint and back to nothing\n🟢 The endpoint reached\nInService\n9.9 minutes after\ncreate-endpoint\n, with the weights using 9.75 GiB of the L4's 24 GB\n🟢 The MCP server runs every step from Claude Code, marks each tool read-only, write or destructive, and passes its tests offline\n🟢\nInstancePools\ngives one endpoint config several fallback instance types\n⚠️ L4 capacity varied by region:\nus-east-1\nrefused twice, about 30 minutes each, while\nus-east-2\nplaced an instance at once\n⚠️ A fallback list holds quota for every type in it\nScope: one account,\nml.g6.xlarge\nwith one NVIDIA L4, vLLM 0.30.0 from the AWS container,\ngoogle/gemma-4-E2B-it\nat full precision, deployed in\nus-east-2\non 2026-09-25. Start-up times are from a single deployment.\nThe strategy for using MCP for SageMaker deployment was validated with an incremental step by step approach.\nReferences\nRepository:\nhttps://github.com/xbill9/sagemaker-gemma\nGemma 4 E2B on Hugging Face:\nhttps://huggingface.co/google/gemma-4-E2B-it\nAWS Deep Learning Containers:\nhttps://github.com/aws/deep-learning-containers\nSageMaker real-time inference:\nhttps://docs.aws.amazon.com/sagemaker/latest/dg/realtime-endpoints.html\nCreateEndpointConfig API:\nhttps://docs.aws.amazon.com/sagemaker/latest/APIReference/API_CreateEndpointConfig.html\nMCP Python SDK:\nhttps://github.com/modelcontextprotocol/python-sdk\nvLLM:\nhttps://docs.vllm.ai/",
  "metadata": {
    "title": "Gemma 4 on an Amazon SageMaker Endpoint: AWS CLI, NVIDIA L4, and an MCP Server - DEV Community",
    "description": "Step by step deployment of Gemma 4 E2B to a SageMaker real-time endpoint on one NVIDIA L4 with the AWS vLLM container, driven by the aws CLI and managed by a Python MCP server from Claude Code or Gemini CLI. Tagged with aws, sagemaker, gemma, mcp.",
    "keywords": "aws, sagemaker, gemma, mcp, software, coding, development, engineering, inclusive, community",
    "author": null,
    "og:type": "article",
    "og:url": "https://dev.to/gde/gemma-4-on-an-amazon-sagemaker-endpoint-aws-cli-nvidia-l4-and-an-mcp-server-5cdd",
    "og:title": "Gemma 4 on an Amazon SageMaker Endpoint: AWS CLI, NVIDIA L4, and an MCP Server",
    "og:description": "Step by step deployment of Gemma 4 E2B to a SageMaker real-time endpoint on one NVIDIA L4 with the AWS vLLM container, driven by the aws CLI and managed by a Python MCP server from Claude Code or Gemini CLI.",
    "og:site_name": "DEV Community",
    "og:image": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fraw.githubusercontent.com%2Fxbill9%2Fsagemaker-gemma%2Fmain%2Fdocs%2Farticles%2Fsagemaker-gemma-deploy%2Fdevto-cover.e07a8baa.jpg",
    "twitter:site": "@thepracticaldev",
    "twitter:creator": "@",
    "twitter:title": "Gemma 4 on an Amazon SageMaker Endpoint: AWS CLI, NVIDIA L4, and an MCP Server",
    "twitter:description": "Step by step deployment of Gemma 4 E2B to a SageMaker real-time endpoint on one NVIDIA L4 with the AWS vLLM container, driven by the aws CLI and managed by a Python MCP server from Claude Code or Gemini CLI.",
    "twitter:card": "summary_large_image",
    "twitter:widgets:new-embed-design": "on",
    "twitter:image:src": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fraw.githubusercontent.com%2Fxbill9%2Fsagemaker-gemma%2Fmain%2Fdocs%2Farticles%2Fsagemaker-gemma-deploy%2Fdevto-cover.e07a8baa.jpg"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [
    {
      "file": "attachments/b0042d8b19d7/code/body-000.txt",
      "chars": 135,
      "language": "text"
    },
    {
      "file": "attachments/b0042d8b19d7/code/body-001.txt",
      "chars": 9,
      "language": "text"
    },
    {
      "file": "attachments/b0042d8b19d7/code/body-002.txt",
      "chars": 486,
      "language": "text"
    },
    {
      "file": "attachments/b0042d8b19d7/code/body-003.txt",
      "chars": 193,
      "language": "text"
    },
    {
      "file": "attachments/b0042d8b19d7/code/body-004.txt",
      "chars": 193,
      "language": "text"
    },
    {
      "file": "attachments/b0042d8b19d7/code/body-005.txt",
      "chars": 10,
      "language": "text"
    },
    {
      "file": "attachments/b0042d8b19d7/code/body-006.txt",
      "chars": 46,
      "language": "text"
    },
    {
      "file": "attachments/b0042d8b19d7/code/body-007.txt",
      "chars": 10,
      "language": "text"
    },
    {
      "file": "attachments/b0042d8b19d7/code/body-008.txt",
      "chars": 98,
      "language": "text"
    },
    {
      "file": "attachments/b0042d8b19d7/code/body-009.txt",
      "chars": 347,
      "language": "text"
    },
    {
      "file": "attachments/b0042d8b19d7/code/body-010.txt",
      "chars": 231,
      "language": "text"
    },
    {
      "file": "attachments/b0042d8b19d7/code/body-011.txt",
      "chars": 141,
      "language": "text"
    },
    {
      "file": "attachments/b0042d8b19d7/code/body-012.txt",
      "chars": 31,
      "language": "text"
    },
    {
      "file": "attachments/b0042d8b19d7/code/body-013.txt",
      "chars": 86,
      "language": "text"
    },
    {
      "file": "attachments/b0042d8b19d7/code/body-014.txt",
      "chars": 652,
      "language": "text"
    },
    {
      "file": "attachments/b0042d8b19d7/code/body-015.txt",
      "chars": 101,
      "language": "text"
    },
    {
      "file": "attachments/b0042d8b19d7/code/body-016.txt",
      "chars": 138,
      "language": "text"
    },
    {
      "file": "attachments/b0042d8b19d7/code/body-017.txt",
      "chars": 12,
      "language": "text"
    },
    {
      "file": "attachments/b0042d8b19d7/code/body-018.txt",
      "chars": 290,
      "language": "text"
    },
    {
      "file": "attachments/b0042d8b19d7/code/body-019.txt",
      "chars": 115,
      "language": "text"
    },
    {
      "file": "attachments/b0042d8b19d7/code/body-020.txt",
      "chars": 326,
      "language": "text"
    },
    {
      "file": "attachments/b0042d8b19d7/code/body-021.txt",
      "chars": 100,
      "language": "text"
    },
    {
      "file": "attachments/b0042d8b19d7/code/body-022.txt",
      "chars": 476,
      "language": "text"
    },
    {
      "file": "attachments/b0042d8b19d7/code/body-023.txt",
      "chars": 265,
      "language": "text"
    },
    {
      "file": "attachments/b0042d8b19d7/code/body-024.txt",
      "chars": 81,
      "language": "text"
    },
    {
      "file": "attachments/b0042d8b19d7/code/body-025.txt",
      "chars": 519,
      "language": "text"
    },
    {
      "file": "attachments/b0042d8b19d7/code/body-026.txt",
      "chars": 100,
      "language": "text"
    },
    {
      "file": "attachments/b0042d8b19d7/code/body-027.txt",
      "chars": 182,
      "language": "text"
    },
    {
      "file": "attachments/b0042d8b19d7/code/body-028.txt",
      "chars": 81,
      "language": "text"
    },
    {
      "file": "attachments/b0042d8b19d7/code/body-029.txt",
      "chars": 87,
      "language": "text"
    },
    {
      "file": "attachments/b0042d8b19d7/code/body-030.txt",
      "chars": 73,
      "language": "text"
    },
    {
      "file": "attachments/b0042d8b19d7/code/body-031.txt",
      "chars": 54,
      "language": "text"
    },
    {
      "file": "attachments/b0042d8b19d7/code/body-032.txt",
      "chars": 197,
      "language": "text"
    },
    {
      "file": "attachments/b0042d8b19d7/code/body-033.txt",
      "chars": 161,
      "language": "text"
    },
    {
      "file": "attachments/b0042d8b19d7/code/body-034.txt",
      "chars": 314,
      "language": "text"
    },
    {
      "file": "attachments/b0042d8b19d7/code/body-035.txt",
      "chars": 402,
      "language": "text"
    },
    {
      "file": "attachments/b0042d8b19d7/code/body-036.txt",
      "chars": 64,
      "language": "text"
    },
    {
      "file": "attachments/b0042d8b19d7/code/body-037.txt",
      "chars": 164,
      "language": "text"
    },
    {
      "file": "attachments/b0042d8b19d7/code/body-038.txt",
      "chars": 35,
      "language": "text"
    }
  ],
  "images": [],
  "audio": [],
  "videos": [],
  "links": [
    {
      "href": "https://github.com/xbill9/sagemaker-gemma",
      "label": "https://github.com/xbill9/sagemaker-gemma",
      "type": "a"
    },
    {
      "href": "https://huggingface.co/google/gemma-4-E2B-it",
      "label": "https://huggingface.co/google/gemma-4-E2B-it",
      "type": "a"
    },
    {
      "href": "https://github.com/aws/deep-learning-containers",
      "label": "https://github.com/aws/deep-learning-containers",
      "type": "a"
    },
    {
      "href": "https://docs.aws.amazon.com/sagemaker/latest/dg/realtime-endpoints.html",
      "label": "https://docs.aws.amazon.com/sagemaker/latest/dg/realtime-endpoints.html",
      "type": "a"
    },
    {
      "href": "https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_CreateEndpointConfig.html",
      "label": "https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_CreateEndpointConfig.html",
      "type": "a"
    },
    {
      "href": "https://github.com/modelcontextprotocol/python-sdk",
      "label": "https://github.com/modelcontextprotocol/python-sdk",
      "type": "a"
    },
    {
      "href": "https://docs.vllm.ai/",
      "label": "https://docs.vllm.ai/",
      "type": "a"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:48:23Z",
  "source_url": "https://dev.to/gde/gemma-4-on-an-amazon-sagemaker-endpoint-aws-cli-nvidia-l4-and-an-mcp-server-5cdd",
  "papers": [],
  "actual_video_files": [],
  "body_chars": 18371,
  "expected_body_images": 0,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/b0042d8b19d7/pages/source.html",
  "local_html": "attachments/b0042d8b19d7/pages/article.html",
  "body_base_path": "attachments/b0042d8b19d7/pages/",
  "body_markdown_path": "attachments/b0042d8b19d7/pages/article.md"
}