{
  "source_file": "0930",
  "source_platform": "Hashnode",
  "author": "Juan Carlos Olamendy",
  "title_listed": "Deploying vLLM with Docker: The Complete Guide to Production-Ready LLM Inference",
  "keywords": "",
  "domain": "大模型算法",
  "content_type": "深度技术解析",
  "is_paid_flag": "否",
  "url": "https://juancolamendy.hashnode.dev/deploying-vllm-with-docker-the-complete-guide-to-production-ready-llm-inference",
  "publish_date": "2026-02-03",
  "corpus_stage": "",
  "notes": "",
  "id": "0e323c4189e5",
  "collected_at": "2026-10-02T14:10:47",
  "title": "Deploying vLLM with Docker: The Complete Guide to Production-Ready LLM Inference",
  "body_markdown": "Deploying vLLM with Docker: The Complete Guide to Production-Ready LLM Inference\nPublished\nFebruary 3, 2026\n•\n10\nmin read\n•\nView as Markdown\nJ\nJuan Carlos Olamendy\nSenior AI Engineer | Building & Telling Stories about AI/ML Systems | Software Engineer\nYour GPU is sitting idle while your LLM inference requests queue up, one by one, painfully slow.\nYou know there's a better way. You've heard about continuous batching, PagedAttention, and throughput numbers that seem too good to be true.\nWelcome to vLLM—the inference engine that's quietly becoming the backbone of production LLM deployments everywhere.\nRunning large language models in production isn't just about downloading weights and calling a predict function.\nIt's about squeezing every drop of performance from expensive GPU hardware. It's about handling hundreds of concurrent requests without melting your infrastructure. It's about exposing a clean API that your applications can actually use.\nvLLM solves all of these problems, and Docker makes deploying it almost embarrassingly simple. In this guide, you'll learn exactly how to deploy vLLM using Docker, configure it for your specific needs, and scale it to production with Kubernetes. Let's dive in.\nWhat is vLLM and Why Should You Care?\nvLLM is a high-throughput inference engine designed specifically for large language models. Developed by researchers at UC Berkeley, it introduces PagedAttention—a memory management technique that treats the KV cache like virtual memory pages. This seemingly simple innovation unlocks massive efficiency gains.\nTraditional LLM serving wastes enormous amounts of GPU memory on the key-value cache. Each request reserves a contiguous block of memory, even when it doesn't need it all. vLLM's PagedAttention allocates memory in small blocks on demand, dramatically reducing waste. The result? You can serve 2-4x more concurrent requests on the same hardware.\ndocs.vllm\nBut memory efficiency is only part of the story. vLLM implements continuous batching, which means new requests can join an existing batch mid-inference. Traditional batching waits for an entire batch to complete before starting the next one. Continuous batching keeps your GPU busy almost constantly, maximizing throughput.\nThe cherry on top? vLLM exposes an OpenAI-compatible API out of the box. Your existing applications that call OpenAI's\n/v1/chat/completions\nendpoint can switch to vLLM with a single URL change. No code rewrites. No SDK changes. Just point to your new endpoint and go.\nThe vLLM Architecture: How It Actually Works\nUnderstanding how vLLM works under the hood helps you make better deployment decisions. Let's walk through the request lifecycle.\nStep 1: Model Loading\nWhen vLLM starts, it loads model weights from Hugging Face Hub (or a local path) into GPU memory. It initializes the tokenizer, configures the KV cache based on available memory, and prepares the inference engine.\ndocs.vllm\nThe KV cache configuration is crucial. vLLM calculates how many tokens it can cache based on your\n--gpu-memory-utilization\nsetting. Higher utilization means more concurrent requests but less headroom for spikes.\nStep 2: Request Reception\nYour application sends an HTTP POST to\n/v1/chat/completions\nor\n/v1/completions\n. The request includes your prompt, generation parameters (temperature, max_tokens, etc.), and optional streaming flags.\ndocs.premai\nvLLM's server validates the request, tokenizes the input, and adds it to the scheduling queue. If you've configured an API key, authentication happens at this stage.\nStep 3: Scheduling and Batching\nHere's where vLLM's magic happens. The scheduler doesn't wait for a full batch before starting inference. Instead, it uses continuous batching to add new requests to running batches whenever there's capacity.\nideas2it\nPagedAttention manages memory allocation. Each request gets blocks of KV cache as needed, not a pre-allocated contiguous chunk. When a request finishes, its blocks return to the pool immediately.\nStep 4: Token Generation\nThe inference loop runs the transformer forward pass on the current batch. For each request, vLLM updates the KV cache, samples the next token based on your temperature/top_p settings, and checks stopping conditions.\nx\nThis loop repeats until all requests in the batch hit their max_tokens limit or generate a stop token. New requests can join mid-loop thanks to continuous batching.\nStep 5: Response Streaming\nIf streaming is enabled, vLLM sends tokens back as they're generated using server-sent events (SSE). Otherwise, it accumulates the full response and returns it as a JSON object—matching OpenAI's response format exactly.\ndocs.vllm\nDeploying vLLM with Docker: The Minimal Setup\nNow let's get practical.\nThe fastest way to run vLLM is with the official\nvllm/vllm-openai\nDocker image. This image packages everything you need: vLLM, its dependencies, and the OpenAI-compatible server.\nPrerequisites\nBefore you start, ensure you have:\nDocker with NVIDIA GPU support\n: Install the NVIDIA Container Toolkit (\nnvidia-docker2\nor the newer\nnvidia-container-toolkit\n).\nA capable GPU\n: vLLM works best with NVIDIA GPUs that have sufficient VRAM for your target model.\nHugging Face token\n(optional): Required only for gated models like Llama 3.\nVerify your GPU is visible to Docker:\ndocker run --rm --gpus all nvidia/cuda:12.0-base nvidia-smi\nYou should see your GPU listed. If not, troubleshoot your NVIDIA Container Toolkit installation before proceeding.\nThe Minimal Docker Command\nHere's the simplest command to run vLLM:\ndocker run --rm \\\n  --gpus all \\\n  -p 8000:8000 \\\n  -v ~/.cache/huggingface:/root/.cache/huggingface \\\n  -e HUGGING_FACE_HUB_TOKEN=<your_hf_token> \\\n  vllm/vllm-openai:latest \\\n  --model mistralai/Mistral-7B-v0.1\nLet's break down each flag:\n--rm\n: Remove the container when it stops. Clean and simple for testing.\n--gpus all\n: Expose all host NVIDIA GPUs to the container. You can also specify\n--gpus '\"device=0\"'\nfor a single GPU.\n-p 8000:8000\n: Map container port 8000 to host port 8000. vLLM's server listens here by default.\n-v ~/.cache/huggingface:/root/.cache/huggingface\n: Mount your local HF cache to avoid re-downloading models on every run.\n-e HUGGING_FACE_HUB_TOKEN\n: Your HF token for accessing gated models.\n--model\n: The model to load. Can be a HF model ID or a local path.\nThe server starts, downloads the model (if not cached), and begins listening for requests.\nTesting Your Deployment\nOnce the server is running, test it with a simple curl command:\ncurl http://localhost:8000/v1/models\nYou should see a JSON response listing your loaded model. Now try a chat completion:\ncurl http://localhost:8000/v1/chat/completions \\\n  -H\n\"Content-Type: application/json\"\n\\\n  -d\n'{\n    \"model\": \"mistralai/Mistral-7B-v0.1\",\n    \"messages\": [\n      {\"role\": \"user\", \"content\": \"Explain Docker in one sentence.\"}\n    ],\n    \"max_tokens\": 100\n  }'\nThe response follows OpenAI's format exactly. Your existing OpenAI client libraries will work without modification—just change the base URL.\nCommon Customizations and Configuration Options\nThe minimal setup gets you running, but production deployments need tuning. vLLM offers extensive configuration options through command-line arguments.\nChanging Models\nSwitch models by changing the\n--model\nargument:\ndocker run --rm --gpus all -p 8000:8000 \\\n  vllm/vllm-openai:latest \\\n  --model meta-llama/Meta-Llama-3-8B-Instruct\nFor local models, mount the model directory and point to it:\ndocker run --rm --gpus all -p 8000:8000 \\\n  -v /path/to/models:/models \\\n  vllm/vllm-openai:latest \\\n  --model /models/my-fine-tuned-llama\nSecuring Your API\nIn production, you don't want an open endpoint. vLLM supports API key authentication:\ndocker run --rm --gpus all -p 8000:8000 \\\n  -e VLLM_API_KEY=your-secret-key \\\n  vllm/vllm-openai:latest \\\n  --model mistralai/Mistral-7B-v0.1\nClients must include the key in the\nAuthorization\nheader:\ncurl http://localhost:8000/v1/models \\\n  -H\n\"Authorization: Bearer your-secret-key\"\nTuning GPU Memory Utilization\nBy default, vLLM uses 90% of available GPU memory. You can adjust this with\n--gpu-memory-utilization\n:\ndocker run --rm --gpus all -p 8000:8000 \\\n  vllm/vllm-openai:latest \\\n  --model mistralai/Mistral-7B-v0.1 \\\n  --gpu-memory-utilization 0.85\nLower values leave more headroom for memory spikes but reduce the maximum number of concurrent requests. Higher values maximize throughput but risk out-of-memory errors under heavy load.\nnm-vllm.readthedocs\nConfiguring Tensor Parallelism\nFor models too large for a single GPU, use tensor parallelism:\ndocker run --rm --gpus all -p 8000:8000 \\\n  vllm/vllm-openai:latest \\\n  --model meta-llama/Meta-Llama-3-70B-Instruct \\\n  --tensor-parallel-size 4\nThis splits the model across 4 GPUs. Ensure your container has access to all required GPUs.\nSetting Maximum Model Length\nSome models support longer contexts than their default configuration. Override with\n--max-model-len\n:\ndocker run --rm --gpus all -p 8000:8000 \\\n  vllm/vllm-openai:latest \\\n  --model mistralai/Mistral-7B-v0.1 \\\n  --max-model-len 16384\nBe careful: longer contexts require more KV cache memory, reducing concurrent capacity.\nEnabling Quantization\nFor memory-constrained deployments, enable quantization:\ndocker run --rm --gpus all -p 8000:8000 \\\n  vllm/vllm-openai:latest \\\n  --model TheBloke/Mistral-7B-v0.1-AWQ \\\n  --quantization awq\nvLLM supports AWQ, GPTQ, and other quantization formats. Quantized models use less memory at the cost of some quality.\nBuilding Your Own vLLM Docker Image\nThe official image works for most cases, but sometimes you need customizations. Maybe you want to pin a specific vLLM version, add custom dependencies, or bake in your model weights.\nBuilding from Source\nClone the vLLM repository and build:\ngit\nclone\nhttps://github.com/vllm-project/vllm.git\ncd\nvllm\n\nDOCKER_BUILDKIT=1 docker build . \\\n  --target vllm-openai \\\n  --tag my-vllm:latest \\\n  --file docker/Dockerfile\nThe build takes a while—it compiles CUDA kernels for your target architecture. Once complete, run your custom image:\ndocker run --rm --gpus all -p 8000:8000 \\\n  my-vllm:latest \\\n  --model mistralai/Mistral-7B-v0.1\nBaking in Model Weights\nFor faster startup in production, bake model weights into the image:\nFROM\nvllm/vllm-openai:latest\n# Download model during build\nRUN\npython -c\n\"from huggingface_hub import snapshot_download; \\\n    snapshot_download('mistralai/Mistral-7B-v0.1', local_dir='/models/mistral-7b')\"\n# Set default model\nCMD\n[\n\"--model\"\n,\n\"/models/mistral-7b\"\n]\nBuild and push to your registry. Now containers start instantly without downloading models.\nPS:\nIf you like this article, share it with others ♻️\nWould help a lot ❤️\nAnd feel free to follow me for more content like this.\nDeploying vLLM on Kubernetes\nDocker is great for single-node deployments. For production clusters, Kubernetes provides orchestration, scaling, and resilience. vLLM deploys cleanly on K8s with GPU node pools.\ndocs.vllm\nPrerequisites\nYour Kubernetes cluster needs:\nNVIDIA device plugin\n: Makes\nnvidia.com/gpu\nresources visible to the scheduler.\nGPU node pool\n: At least one node with NVIDIA GPUs.\nkubectl\naccess configured.\nCreate a namespace for your vLLM deployment:\nkubectl create namespace vllm\nIf you're using gated models, create a secret for your HF token:\nkubectl create secret generic hf-token \\\n  --from-literal=token=<your_hf_token> \\\n  --namespace vllm\nThe Deployment Manifest\nHere's a production-ready Deployment:\napiVersion:\napps/v1\nkind:\nDeployment\nmetadata:\nname:\nvllm-server\nnamespace:\nvllm\nspec:\nreplicas:\n1\nselector:\nmatchLabels:\napp:\nvllm-server\ntemplate:\nmetadata:\nlabels:\napp:\nvllm-server\nspec:\ncontainers:\n-\nname:\nvllm\nimage:\nvllm/vllm-openai:latest\nargs:\n-\n\"--model\"\n-\n\"mistralai/Mistral-7B-v0.1\"\n-\n\"--host\"\n-\n\"0.0.0.0\"\n-\n\"--port\"\n-\n\"8000\"\n-\n\"--gpu-memory-utilization\"\n-\n\"0.9\"\nenv:\n-\nname:\nHUGGING_FACE_HUB_TOKEN\nvalueFrom:\nsecretKeyRef:\nname:\nhf-token\nkey:\ntoken\nports:\n-\ncontainerPort:\n8000\nname:\nhttp\nresources:\nrequests:\ncpu:\n\"2\"\nmemory:\n\"6Gi\"\nnvidia.com/gpu:\n\"1\"\nlimits:\ncpu:\n\"4\"\nmemory:\n\"16Gi\"\nnvidia.com/gpu:\n\"1\"\nvolumeMounts:\n-\nmountPath:\n/root/.cache/huggingface\nname:\ncache-volume\n-\nname:\nshm\nmountPath:\n/dev/shm\nreadinessProbe:\nhttpGet:\npath:\n/health\nport:\n8000\ninitialDelaySeconds:\n60\nperiodSeconds:\n5\nlivenessProbe:\nhttpGet:\npath:\n/health\nport:\n8000\ninitialDelaySeconds:\n60\nperiodSeconds:\n10\nvolumes:\n-\nname:\ncache-volume\nemptyDir:\n{}\n-\nname:\nshm\nemptyDir:\nmedium:\nMemory\nKey configuration points:\nGPU resources\n: Request and limit\nnvidia.com/gpu: \"1\"\nto ensure GPU scheduling.\nShared memory\n: Mount\n/dev/shm\nas a memory-backed emptyDir for PyTorch's multiprocessing.\nHealth probes\n: vLLM exposes\n/health\nfor readiness and liveness checks.\nInitial delay\n: 60 seconds gives the model time to load before health checks start.\nHF cache volume\n: Persists downloaded models across pod restarts within the same node.\nApply the deployment:\nkubectl apply -f vllm-deployment.yaml\nExposing the Service\nCreate a Service to make vLLM accessible within your cluster:\ndocs.vllm\napiVersion:\nv1\nkind:\nService\nmetadata:\nname:\nvllm-service\nnamespace:\nvllm\nspec:\ntype:\nClusterIP\nselector:\napp:\nvllm-server\nports:\n-\nname:\nhttp\nport:\n8000\ntargetPort:\n8000\nApply it:\nkubectl apply -f vllm-service.yaml\nOther pods in your cluster can now reach vLLM at:\nhttp://vllm-service.vllm.svc.cluster.local:\n8000\nAdding an Ingress\nFor external access, add an Ingress:\napiVersion:\nnetworking.k8s.io/v1\nkind:\nIngress\nmetadata:\nname:\nvllm-ingress\nnamespace:\nvllm\nannotations:\nnginx.ingress.kubernetes.io/proxy-read-timeout:\n\"300\"\nnginx.ingress.kubernetes.io/proxy-send-timeout:\n\"300\"\nspec:\ningressClassName:\nnginx\nrules:\n-\nhost:\nvllm.your-domain.com\nhttp:\npaths:\n-\npath:\n/\npathType:\nPrefix\nbackend:\nservice:\nname:\nvllm-service\nport:\nnumber:\n8000\nThe timeout annotations are important—LLM requests can take 30+ seconds for long generations.\nUsing the Helm Chart\nFor more complex deployments, vLLM provides an official Helm chart:\ndocs.vllm\nhelm repo add vllm https://vllm-project.github.io/helm-charts\nhelm repo update\n\nhelm upgrade --install vllm vllm/vllm \\\n  --namespace vllm --create-namespace \\\n  --\nset\nresources.requests.\n\"nvidia\\.com/gpu\"\n=1 \\\n  --\nset\nresources.limits.\n\"nvidia\\.com/gpu\"\n=1 \\\n  --\nset\nextraArgs=\n\"{--model,mistralai/Mistral-7B-v0.1}\"\nThe Helm chart supports autoscaling, multiple replicas, and advanced configuration through values files.\nvllm.hyper\nConclusion\nvLLM transforms LLM deployment from a research project into a production-ready operation.\nPagedAttention and continuous batching deliver throughput that was previously impossible without custom infrastructure. The OpenAI-compatible API means you can adopt vLLM without rewriting your applications.\nDocker makes deployment straightforward: pull the official image, expose your GPUs, and start serving. For production scale, Kubernetes provides the orchestration layer you need for resilience and scaling.\nThe key decisions you'll make are:\nGPU memory utilization\n: Balance concurrent capacity against OOM risk.\nModel selection\n: Match model size to your GPU capabilities.\nScaling strategy\n: Decide between horizontal replicas or tensor parallelism.\nCaching and persistence\n: Avoid cold start delays with persistent model storage.\nStart with the minimal Docker command, validate your setup with a few test requests, then iterate toward production. vLLM's extensive configuration options let you tune performance as you learn your workload patterns.\nYour GPU no longer needs to sit idle. Deploy vLLM, and put that hardware to work.\nPS:\nIf you like this article, share it with others ♻️\nWould help a lot ❤️\nAnd feel free to follow me for more content like this.\n#\nartificial-intelligence\n#\nllm\n#\nlarge-language-models\n#\nvllm",
  "metadata": {
    "title": "Deploying vLLM with Docker: The Complete Guide to Production-Ready LLM Inference",
    "description": "Your GPU is sitting idle while your LLM inference requests queue up, one by one, painfully slow.\nYou know there's a better way. You've heard about continuous batching, PagedAttention, and throughput numbers that seem too good to be true.\nWelcome to v...",
    "keywords": null,
    "author": null,
    "og:title": "Deploying vLLM with Docker: The Complete Guide to Production-Ready LLM Inference",
    "og:description": "Your GPU is sitting idle while your LLM inference requests queue up, one by one, painfully slow.\nYou know there's a better way. You've heard about continuous batching, PagedAttention, and throughput numbers that seem too good to be true.\nWelcome to v...",
    "og:url": "https://juancolamendy.hashnode.dev/deploying-vllm-with-docker-the-complete-guide-to-production-ready-llm-inference",
    "og:site_name": "Juan C Olamendy",
    "og:image": "https://cdn.hashnode.com/res/hashnode/image/upload/v1770143557706/9d9bffc9-2234-469a-9680-79b9c4035fe8.png",
    "og:image:width": "1200",
    "og:image:height": "630",
    "og:type": "article",
    "twitter:card": "summary_large_image",
    "twitter:creator": "https://twitter.com/juancolamendy",
    "twitter:title": "Deploying vLLM with Docker: The Complete Guide to Production-Ready LLM Inference",
    "twitter:description": "Your GPU is sitting idle while your LLM inference requests queue up, one by one, painfully slow.\nYou know there's a better way. You've heard about continuous batching, PagedAttention, and throughput numbers that seem too good to be true.\nWelcome to v...",
    "twitter:image": "https://cdn.hashnode.com/res/hashnode/image/upload/v1770143557706/9d9bffc9-2234-469a-9680-79b9c4035fe8.png",
    "article:published_time": "2026-02-03T18:37:51.893Z",
    "article:author": "https://juancolamendy.hashnode.dev/@juancolamendy"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [
    {
      "file": "attachments/0e323c4189e5/code/body-000.txt",
      "chars": 60,
      "language": "text"
    },
    {
      "file": "attachments/0e323c4189e5/code/body-001.txt",
      "chars": 213,
      "language": "text"
    },
    {
      "file": "attachments/0e323c4189e5/code/body-002.txt",
      "chars": 37,
      "language": "text"
    },
    {
      "file": "attachments/0e323c4189e5/code/body-003.txt",
      "chars": 260,
      "language": "text"
    },
    {
      "file": "attachments/0e323c4189e5/code/body-004.txt",
      "chars": 116,
      "language": "text"
    },
    {
      "file": "attachments/0e323c4189e5/code/body-005.txt",
      "chars": 139,
      "language": "text"
    },
    {
      "file": "attachments/0e323c4189e5/code/body-006.txt",
      "chars": 142,
      "language": "text"
    },
    {
      "file": "attachments/0e323c4189e5/code/body-007.txt",
      "chars": 84,
      "language": "text"
    },
    {
      "file": "attachments/0e323c4189e5/code/body-008.txt",
      "chars": 140,
      "language": "text"
    },
    {
      "file": "attachments/0e323c4189e5/code/body-009.txt",
      "chars": 146,
      "language": "text"
    },
    {
      "file": "attachments/0e323c4189e5/code/body-010.txt",
      "chars": 132,
      "language": "text"
    },
    {
      "file": "attachments/0e323c4189e5/code/body-011.txt",
      "chars": 132,
      "language": "text"
    },
    {
      "file": "attachments/0e323c4189e5/code/body-012.txt",
      "chars": 172,
      "language": "text"
    },
    {
      "file": "attachments/0e323c4189e5/code/body-013.txt",
      "chars": 97,
      "language": "text"
    },
    {
      "file": "attachments/0e323c4189e5/code/body-014.txt",
      "chars": 267,
      "language": "text"
    },
    {
      "file": "attachments/0e323c4189e5/code/body-015.txt",
      "chars": 30,
      "language": "text"
    },
    {
      "file": "attachments/0e323c4189e5/code/body-016.txt",
      "chars": 101,
      "language": "text"
    },
    {
      "file": "attachments/0e323c4189e5/code/body-017.txt",
      "chars": 1660,
      "language": "text"
    },
    {
      "file": "attachments/0e323c4189e5/code/body-018.txt",
      "chars": 38,
      "language": "text"
    },
    {
      "file": "attachments/0e323c4189e5/code/body-019.txt",
      "chars": 201,
      "language": "text"
    },
    {
      "file": "attachments/0e323c4189e5/code/body-020.txt",
      "chars": 35,
      "language": "text"
    },
    {
      "file": "attachments/0e323c4189e5/code/body-021.txt",
      "chars": 48,
      "language": "text"
    },
    {
      "file": "attachments/0e323c4189e5/code/body-022.txt",
      "chars": 509,
      "language": "text"
    },
    {
      "file": "attachments/0e323c4189e5/code/body-023.txt",
      "chars": 312,
      "language": "text"
    }
  ],
  "images": [
    {
      "status": "ok",
      "url": "https://cdn.hashnode.com/res/hashnode/image/upload/v1770143557706/9d9bffc9-2234-469a-9680-79b9c4035fe8.png",
      "bytes": 1740459,
      "reused": true,
      "local_path": "attachments/0e323c4189e5/images/000.png",
      "src": "https://cdn.hashnode.com/res/hashnode/image/upload/v1770143557706/9d9bffc9-2234-469a-9680-79b9c4035fe8.png",
      "alt": "Deploying vLLM with Docker: The Complete Guide to Production-Ready LLM Inference",
      "downloaded": true
    }
  ],
  "audio": [],
  "videos": [],
  "links": [
    {
      "href": "https://juancolamendy.hashnode.dev/deploying-vllm-with-docker-the-complete-guide-to-production-ready-llm-inference.md",
      "label": "View as Markdown",
      "type": "a"
    },
    {
      "href": "https://hashnode.com/@juancolamendy",
      "label": "J",
      "type": "a"
    },
    {
      "href": "https://twitter.com/juancolamendy",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://www.linkedin.com/in/juancolamendy/",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://docs.vllm.ai/en/v0.8.3/serving/openai_compatible_server.html",
      "label": "docs.vllm",
      "type": "a"
    },
    {
      "href": "https://docs.premai.io/guides/general/serve-models-to-vllm",
      "label": "docs.premai",
      "type": "a"
    },
    {
      "href": "https://www.ideas2it.com/blogs/deploying-llm-powered-applications-in-production-using-tgi",
      "label": "ideas2it",
      "type": "a"
    },
    {
      "href": "https://x.com/hyperbolic_labs/status/1917283791485428205?lang=en",
      "label": "x",
      "type": "a"
    },
    {
      "href": "https://nm-vllm.readthedocs.io/en/0.4.0/serving/openai_compatible_server.html",
      "label": "nm-vllm.readthedocs",
      "type": "a"
    },
    {
      "href": "https://docs.vllm.ai/en/v0.8.4/deployment/k8s.html",
      "label": "docs.vllm",
      "type": "a"
    },
    {
      "href": "https://docs.vllm.ai/en/v0.6.3/serving/deploying_with_k8s.html",
      "label": "docs.vllm",
      "type": "a"
    },
    {
      "href": "https://docs.vllm.ai/en/latest/deployment/frameworks/helm.html",
      "label": "docs.vllm",
      "type": "a"
    },
    {
      "href": "https://vllm.hyper.ai/docs/deployment/framworks/helm/",
      "label": "vllm.hyper",
      "type": "a"
    },
    {
      "href": "https://juancolamendy.hashnode.dev/tag/artificial-intelligence",
      "label": "# artificial-intelligence",
      "type": "a"
    },
    {
      "href": "https://juancolamendy.hashnode.dev/tag/llm",
      "label": "# llm",
      "type": "a"
    },
    {
      "href": "https://juancolamendy.hashnode.dev/tag/large-language-models",
      "label": "# large-language-models",
      "type": "a"
    },
    {
      "href": "https://juancolamendy.hashnode.dev/tag/vllm",
      "label": "# vllm",
      "type": "a"
    }
  ],
  "retry_at": "2026-10-03T08:05:27",
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:50:33Z",
  "source_url": "https://juancolamendy.hashnode.dev/deploying-vllm-with-docker-the-complete-guide-to-production-ready-llm-inference",
  "papers": [],
  "actual_video_files": [],
  "body_chars": 15745,
  "expected_body_images": 1,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/0e323c4189e5/pages/source.html",
  "local_html": "attachments/0e323c4189e5/pages/article.html",
  "body_base_path": "attachments/0e323c4189e5/pages/",
  "body_markdown_path": "attachments/0e323c4189e5/pages/article.md"
}