{
  "source_file": "0930",
  "source_platform": "Substack",
  "author": "swyx & Alessio Fanelli / Latent Space",
  "title_listed": "$2 H100s: How the GPU Rental Bubble Burst",
  "keywords": "GPU算力,成本分析,数据中心,推理成本",
  "domain": "数据处理",
  "content_type": "行业分析",
  "is_paid_flag": "否",
  "url": "https://www.latent.space/p/gpu-bubble",
  "publish_date": "2024-11",
  "corpus_stage": "预训练",
  "notes": "GPU租赁市场分析，含训练/推理成本数据",
  "id": "a18b3cdd6d13",
  "collected_at": "2026-10-01T19:50:15",
  "title": "$2 H100s: How the GPU Bubble Burst - by Eugene Cheah",
  "body_markdown": "Don’t miss Eugene’s responses on the\nHN\nand\nReddit\nand\nTwitter\ndiscussions!\nSwyx’s note:\nwe’re on a roll catching up with former guests! Apart from our recent guest spot on\nRaza Habib’s chat with Hamel Husain\n(see\nour Raza pod here\n), we’re delighted for the return of Eugene Cheah (see\nour pod on RWKV last year\n) as a rare guest\nwriter\nfor our newsletter\n.\nEugene has now cofounded\nFeatherless.AI\n, an inference platform with the world’s largest collection of open source models (~2,000) instantly accessible via a single API for a\nflat rate\n(starting at $10 a month).\nRecently there has been a lot of excitement with NVIDIA’s new Blackwell series\nrolling out to OpenAI\n, with the company saying it is\nsold out for the next year\nand Jensen noting that it could be the “\nmost successful product in the history of the industry\n”. With cousin Lisa hot on his heels\nannouncing the MI3 25 X\n(yet\na large software gap\n) and\nCerebras filing for IPO\n, it is time to dive deep on the GPU market again (see also\nformer guest\nDylan Patel’s pod\nfor his trademark candid take of course - especially his take on what he calls the “\nAI Neoclouds\n”, which is what Eugene discusses here).\nDo we yet have an answer to\nthe $600bn question\n? It is now consensus that the capex on foundation model training is the “\nfastest depreciating asset in history\n”, but the jury on GPU infra spend is still out and\nthe GPU Rich Wars are raging\n. Meanwhile, we know now that frontier labs are spending more on training+inference than they make in revenue,\nraising $6.6b in the largest venture round of all time\nwhile also\nprojecting losses of $14b in 2026\n. The financial logic requires AGI to parse.\nFortunately, we’ve got someone who spends all his time thinking about this. What follows is Eugene’s take on GPU economics as he is now an inference provider, diving deep on the H100 market, as a possible read for what is to come for the Blackwell generation. Not financial advice! We also recommend\nYangqing Jia’s guide\n.\nTLDR: Don’t buy H100s. The market has flipped from shortage ($8/hr) to oversupplied ($2/hr), because of reserved compute resales, open model finetuning, and decline in new foundation model co’s. Rent instead\n1\n.\nFor the general market, it makes little sense to be investing in new H100s today, when\nyou can rent it at near cost, when you need it\n, with the current oversupply.\nA short history of the AI race\nChatGPT was launched in November 2022, built on the A100 series. The H100s arrived in March 2023.\nThe pitch to investors and founders was simple:\nCompared to A100s,\nthe new H100s were 3x more powerful, but only 2x the sticker price\n.\nIf you were faster to ramp up on H100s, you too, can build a bigger, better model, and maybe even leapfrog OpenAI to Artificial General Intelligence - If you have the capital to match their wallet!\nWith this desire, $10-100’s billions of dollars were invested into GPU-rich AI startups to build this next revolution. Which lead to ….\nThe sudden surge in H100 demand\nMarket prices shot through the roof, the original rental rates of H100 started at approximately\n$4.70 an hour\nbut were going for\nover $8\n. For all the desperate founders rushing to train their models to convince their investors for their next $100 million round.\nNvidia, literally pitched to their investors & datacenter customers, in their 2023 investor presentation - the “market opportunity” on\nrenting H100s at $4/hr\nFor GPU farms, it felt like free money - if you can get these founders to rent your H100 SXMGPUs at $4.70 an hour or more, or even get them to pay it upfront,\nthe payback period was <1.5 years\n. From then on, it was free-flowing cash of over $100k per GPU, per year.\nWith no end to the GPU demand in sight, their investors agreed, with even larger investments\n2\n…\nThe Tulip Folly\n- Painted after the first speculative bubble in recorded history -\non tulip prices which kept climbing in 1634 and collapsed in February 1637\n$600 Billion in investment later …\nPhysical goods, unlike digital goods, suffer from lag time. Especially when there are\nmultiple shipment delays\n.\nFor most of 2023, the H100 prices felt like they would forever be above $4.70 (unless you were willing to do a huge upfront downpayment)\nAt the start of 2024, the H100 prices reached approximately $2.85 across multiple providers.\nAs more providers come online, however… I started to get emails like this:\nWhile I have not been successful with acquiring H100 nodes (8xH100) at $4/hour, I have confirmed multiple times, that you can do so at $8 - $16/hour\nIn Aug 2024, if you're willing to auction for a small slice of H100 time (days to weeks), you can start finding H100 GPUs for $1 to $2 an hour.\nWe are looking at a >= 40% price drop per year\n, especially for small clusters. NVIDIA’s marketing projection of $4 per GPU hour across 4 years, has evaporated away in under 1.5 years.\nAnd that is horrifying because it means someone out there is potentially\nleft holding the bag\n- especially so if they just bought it as a new GPUs. So what is going on?\nWhat’s the ROI on a USD $50k H100 SXM GPU?\nThis will be focusing on the economical cost, and the ROI on leasing, against various market rates. Not the opportunity cost, or buisness value.\nThe average H100 SXM GPU in a data center costs $50k or more to set up, maintain, and operate (aka most of the CAPEX). Excluding electricity and cooling OPEX cost. More details on the calculation are provided later in this article.\nBut what does that mean for unit economics today, as an investment?\nEspecially if we assume a 5-year lifespan on the GPUs itself today.\nGenerally, there are two business models for leasing H100, which we would cover.\nShort on-demand leases (by the hour - by the week - or the month)\nLongterm reservation (3-5 years)\nOn-demand leasing ROI\nSpreadsheet for:\nnew H100 ROI (Aug 2024)\nIn summary, for an on-demand workload\n>$2.85\n: Beat stock market IRR\n<$2.85\n: Loses to stock market IRR\n<$1.65\n: Expect loss in investment\nFor the above ROI and revenue forecast projection, we introduced “blended price”, where we assume a gradual drop to 50% in the rental price across 5 years.\nThis is arguably a conservative/optimistic estimate given the >= 40% price drop per year we see now. But it’s a means of projecting an ROI while taking into account a certain % of price drop.\nAt $4.50/hour, even when blended, we get to see the original pitch for data center providers from NVIDIA, where they practically print money after 2 years. Giving an IRR (Internal rate of return) of 20+%.\nHowever, at $2.85/hour, this is where it starts to be barely above 10% IRR.\nMeaning, if you are buying a new H100 server today, and if the market price is less than $2.85/hour, you can barely beat the market, assuming 100% allocation (which is an unreasonable assumption). Anything, below that price, and you're better off with the stock market, instead of a H100 infrastructure company, as an investment.\nAnd if the price falls below $1.65/hour, you are doomed to make losses on the H100 over the 5 years, as an infra provider\n. Especially, if you just bought the nodes and cluster this year.\nLongterm reservation leases (3 years+)\nMany infrastructure providers, especially the older ones - were not naive about this - Because they had been burnt firsthand by GPU massive rental price drops, after a major price pump, from the crypto days - they had seen this cycle before.\nSo for this cycle, last year, they pushed heavily for a 3-5 year upfront commitment and/or payment at the $4+ price range.\n(typically with 50% to 100% upfront)\n.\nToday, they push the $2.85+ price range - locking in their profits.\nThis happened aggressively during the 2023 AI peak with various foundation model companies, especially in the image generation space, indirectly forced into high-priced 3-5 year contracts, just so to get to the front-of-the-line of a new cluster, and be first to make their target model, to help close the next round.\nIt may not be the most economical move, but it lets them move faster than the competition.\nThis, however, has led to some interesting market dynamics - if you are paying $3 or $4 per hour for your H100, for the next 3 years, locked into a contract.\nWhen a model creator is done training a model, you have no more use for the cluster.\nWhat would they do? - they resell and start recouping some of the costs.\nThe current H100 value chain\nFrom hardware to AI inference / finetune, it can be broadly viewed as the following\nHardware vendors partnered with Nvidia (one-time purchase cost)\nDatacenter Infrastructure providers & partners (selling long-term reservations, on facility space and/or H100 nodes)\nVC Funds, Large Companies, and Startups: that plann\ned\nto build foundation models (or have already finished building their models)\nResellers of capacity: Runpod, SFCompute, Together.ai, Vast.ai, GPUlist.ai\nManaged AI Inference / Finetune providers: who use a combination of the above\nWhile any layer down the stack may be vertically integrated (skipping the infra players for example), the key drivers here are the\n“Resellers of unused capacity”\nand the rise of “good enough” open weights models like\nLlama 3\n, as they are all major influencing factors in the current H100 economical pressures.\nThe rise of open weights models, on-par with closed-source models.\nIs resulting in a fundamental shift in the market\nMarket Trends: The rise of open-weights models\n↑↑ Increased demand for AI inference & fine-tuning\nBecause many “open” models, lack proper “open source” licenses, but are being distributed freely, and used widely, even commercially. We will refer to them collectively as “open-weights” or “open” models instead here.\nIn general, with multiple open-weights models of various sizes being built, so has the growth in demand for inference and fine-tuning them. This is largely driven by two major events\nThe arrival of GPT4 class open models (eg. 405B LLaMA3, DeepSeek-v2)\nThe maturity and adoption of small (~8B) and medium (~70B) fine-tuned models\nToday, for the vast majority of use cases, enterprises may need, there are already off-the-shelf open-weights models. Which might be a small step behind proprietary models in certain benchmarks.\nProvides an advantage with the following\nFlexibility\n: Domain / Task specific finetunes\nReliability\n: No more minor model updates, breaking use case (there is currently low community trust that model weights are not quietly changed without notification in public API endpoints, causing inexplicable regressions)\nSecurity & Privacy\n: Assurance that their prompts and customer data are safe.\nAll of this leads to the current continuous growth and adoption of open models, with the growth in demand for inference and finetunes.\nBut it does cause another problem…\nCompounded collapse of small & medium model creators\n↓↓ Shrinking foundation model creator market (Small & Medium)\nWe used “model creators” to collectively refer to organization that create models from scratch. For fine-tuners, we refer to them as “model finetuners”\nMany enterprises, and multiple small & medium foundation model creator startups - especially those who raised on the pitch of “smaller, specialized domain-specific models”, are groups who had no long-term plans / goals for training large foundation models from scratch ( >= 70B ).\nFor both groups, they both came to the realization that it is more economical and effective to fine-tune existing Open Weights models, instead of “training on their own”.\nThis ended up creating a triple whammy in reducing the demand for H100s!\nFinetuning is significantly cheaper than training from scratch.\nBecause the demands for fine-tuning are significantly less in compute requirements (typically 4 nodes or less, usually a single node), compared to training from scratch (from 16 nodes, usually more, for 7B and up models).\nThis industry-wide switch essentially killed a large part of smaller cluster demands.\nScaling back on foundation model investment (at small/mid-tier)\nIn 2023, there was a huge wave of small and medium foundation models, within the text and image space.\nToday, however, unless you are absolutely confident you can surpass llama3, or you are bringing something new to the table (eg. new architecture, 100x lower inference, 100+ languages, etc), there are ~no more foundation model cos being founded from scratch.\nIn general, the small & medium, open models created by the bigger players (Facebook, etc), make it hard for smaller players to justify training foundation models - unless they have a strong differentiator to do so (tech or data) - or have plans to scale to larger models.\nAnd this has been reflected lately with investors as well, as there has been a sharp decline in new foundation model creators’ funding. With the vast majority of smaller groups having switched over to finetuning. (this sentiment is combined with the recent less than desired exits for multiple companies).\nPresently today, there is approximately worldwide by my estimate:\n<20 Large model creator teams (aka 70B++, may create small models as well)\n<30 Small / Medium model creator teams (7B - 70B)\nCollectively there are less than <50 teams worldwide who would be in the market for 16 nodes of H100s (or much more), at any point in time, to do foundation model training.\nThere are more than 50 clusters of H100 worldwide with more than 16 nodes.\nExcess capacity from reserved nodes is coming online\nFor the cluster owners, especially the various foundation model startups and VCs, who made long reservations, in the initial “land grab” of the year 2023.\nWith the switch to finetuning, and the very long wait times of the H100’s\n(it peaked at >= 6 months), it is very well possible that many of these groups had already made the upfront payment before they made the change, essentially making their prepaid hardware “obsolete on arrival”.\nAlternatively, those who had the hardware arrive on time, to train their first few models, had come to the same realization it would be better to fine-tune their next iteration of models. Instead of building on their own.\nIn both cases, they would have unused capacity, which comes online via\n“Compute Resellers”\njoining the market supply….\nOther factors causing an increase in supply & reduced training demand\n1)\nLarge model creators goes off public cloud platform\nAnother major factor, is how all the major Model Creators, such as Facebook, X.AI, and arguably OpenAI (if you count them as part of Microsoft) are moving away from an existing public provider, and building their own billion-dollar clusters, removing the demand that the existing clusters depend on.\nThe move is happening mostly for the following reasons:\nExisting ~1k node clusters (which costs >$50M to build), is no longer big enough for them, to train bigger models\nAt a billion-dollar scale, it is better for accounting to purchase assets (of servers, land, etc), which has booked value (part of company valuation and assets), instead of pure expenses leasing.\nIf you do not have the people (they do), you could straight up buy small datacenters companies, who have the expertise to build this for you.\nWith the demand gradually weaning away in stages. These clusters are coming online to the public cloud market instead.\nVast.ai essentially does a free market system, where providers from all over the world, are forced to compete with each other\n2) Unused / Delayed supply coming online\nRecall all the H100 large shipment delays in 2023, or 6 months or more? They are coming online, now - along with the H200, B200, etc.\nThis is alongside, the various unused compute, coming online (from existing startups, enterprises or VCs as covered earlier).\nThe bulk of this is done via\nCompute Resellers\n, such as : together.ai, sfcompute, runpod, vast.ai, etc\nIn most cases, cluster owners have a small or medium cluster, (typically 8-64 nodes), that is underutilized. With the money already “spent” for the cluster.\nWith the primary goal is to recoup as much of the cost as possible, they rather undercut the market and guarantee an allocation, instead of competing with the main providers, and possibly have no allocation.\nThis is typically done either via a fixed rate, an auction system, or just a free market listing, etc. With the later 2 driving the market price downwards.\n3) Cheaper GPU alternatives (esp. for inference)\nAnother major factor, is once your outside of the training / fine-tune space. The inference space is filled with alternatives, especially if your running smaller models.\nOne do not need to pay for the premium invoked by H100’s Infiniband and/or nvidia.\na) Nvidia market segmentation\nH100 premium for training is priced into the hardware. For example nvidia themselves recommend the L40S, which is the more price competitive alternative for inference.\nSpreadsheet for:\nH100 Infiniband Cluster math (Aug 2024)\nWhich Is 1/3rd the performance, at 1/5th the price. But does not work well with multi-node training. Undercutting their very own H100 for this segment.\nb) AMD and Intel alternative providers\nBoth AMD and Intel may be late into the game with their MX300, and Gaudi 3 respectively.\nThis has been tested and verified by us, having used these systems. They are generally:\nCheaper than a H100 in purchase cost\nHave more memory and compute than a H100, and outperforms on a single node.\nOverall, they are great hardware!\nThe catch? They have minor driver issues in training and are entirely unproven in large multi-node cluster training.\nWhich as we covered is largely irrelevant to the current landscape. To anyone but <50 teams. The market for H100 has been moving towards inference and single or small cluster fine-tuning.\nAll of which these GPUs have been proven to work at. For the use cases, the vast majority of the market is asking for.\nThese 2 competitors are full drop-in replacements. With working off-the-shelf inference code (eg. VLLM) or finetuning code for most common model architectures (primarily LLaMA3, followed by others).\nSo, if you have compatibility sorted out. Its highly recommended to have a look.\nc) Decline of GPU usage in crypto/web3 space\nWith Ethereum moving towards proof of stake, ASIC dominating the bitcoin mining race, and the general crypto market condition.\nGPU usage in mining for crypto has been a downward trend, and in several cases unprofitable. And has since been flooding the GPU public cloud market.\nAnd while the vast majority of these GPUs are unusable for training, or even for inference, due to hardware constraints (low PCIe bandwidth, network, etc). The hardware has been flooding the market and has been repurposed for AI inference workloads.\nIn most cases if you are under <10B, you can get decent performance with these GPUs, out of the box, for really low prices.\nIf you optimize it further (though various tricks), you can even get large 405B models to run on a small cluster of this hardware, cheaper then an H100 node (which is what is typically used)\nH100 Prices are becoming commodity-prices cheap.\nOr even being rented at a loss - if so, what now?\nWhat are the possible implications?\nNeutral: Segmentation in H100 cluster prices\nOn a high level, it is expected that big clusters still get to charge a premium (>=$2.90 / hour) because there is no other option. For those who truly need it.\nWe are starting to see this trend for example with Voltage Park:\nWhere clusters with Infiniband are charged at a premium.\nWhile the Ethernet-based instances, which are perfectly fine for inference are priced at a lower rate. Adjusting the prices for the respective use case/availability.\nWhile there’s been a general decline in foundation model creator teams, it is hard to predict if there will be a resurgence, with the growth in open weights, and/or alternative architectures.\nIt is also, expected that in the future, we will see further segmentation by cluster sizes. Where a large 512-node cluster with Infiniband may be billed higher per GPU than a 16-node cluster.\nBad: New public cloud H100 clusters, late to the game, might be unprofitable - some investors may get burnt.\nThere is a lot against you, if you price it below $2.25, depending on your OPEX, you risk potentially being unprofitable.\nIf you price it too high >= $3, you might not be able to get sufficient buyers to fill capacity.\nIf you're late, you could not recoup the cost in the early $4/hour days.\nOverall, these cluster investments will be rough for the key stakeholders and investors.\nWhile I doubt it’s the case, if new clusters, make a large segment of the AI portfolio investments. We may see additional rippling effects in the funding ecosystem from burnt investors.\nNeutral: Medium, to large Model creators, who purchased, long-term leases - already extracted value at the premium\nInstead of a negative outlook, a neutral outlook would be some of the unused compute foundation model creators, coming online, are already paid for.\nThe funding market has already priced in and paid for this cluster and its model training. And “extracted its value” which they used for their current and next funding round.\nMost of these purchases were made before the popularity of\nCompute Resellers\n, the cost was already priced in.\nIf anything, the current revenue they get from their excess H100 compute, and the lowered prices we get, are beneficial to both parties\nIf so the negative market impact is minimal, while overall it’s a net positive win for the ecosystem.\nGood: Cheap H100s, could accelerate the open-weights AI adoption wave\nGiven that the open-weights model has entered the GPT-4 class arena. Falling H100 prices will be the multiplier unlock for open-weights AI adoption.\nIt will be more affordable, for hobbyists, AI developers, and engineers, to run, fine-tune, and tinker with these open models.\nEspecially if there is no major leap for GPT5++,\nbecause it will mean that the gap between open-weights and closed-source models will blur.\nThis is strongly needed, as the market is currently not sustainable. As there lacks the value capture on the application layer for paying users (which trickles down the platform, models, and infra layers)\nIn a way, if everyone is building shovels (including us), and applications with paying users are not being built (and collecting revenue and value).\nBut when AI inference and fine-tuning becomes cheaper than ever.\nIt can potentially kick off the AI application wave. If it has not already slowly started so.\nConclusion: Don’t buy brand new H100’s\nSpending on new H100’s hardware is likely a loss-maker\nUnless you have some combination of discounted H100s, discounted electricity, or a Sovereign AI angle where the location of your GPU is critical to your customers. Or you have billions and need a super large cluster.\nIf you're investing, consider investing elsewhere.\nOr the stock market index itself for a better rate of returns. IMO\nFeatherless.AI plug …\nWhat we do …\nAt Featherless.AI - We currently host the world’s largest collection of OpenSource AI models, instantly accessible, serverlessly, with unlimited requests from $10 a month, at a fixed price.\nWe have indexed and made over 2,000 models ready for inference today. This is 10x the catalog of openrouter.ai, the largest model provider\naggregator\n,\nand\nis the world’s largest collection of Open Weights models available serverlessly for instant inference. Without the need for any expensive dedicated GPUs\nAnd our platform makes this possible, as it’s able to dynamically hot-swap between models in seconds.\nIt’s designed to be easy to use, with full OpenAI API compatibility, so you can just plug our platform in as a replacement to your existing AI API for your AI agents. Running in the background\nAnd we do all of this; As we believe that AI should be easily accessible to everyone, regardless of language or social status.\nwhy we decided to be different, from other inference providers…\nOn the technical side of things, related to this article.\nIt is a challenge having PetaBytes’s worth of AI models, and growing, running 24/7 - while being hardware profitable (we are), because we needed to optimize every layer of our platform, down to how we choose the GPU hardware.\nIn an industry, where the typical inference provider pitch is typically along the lines of winning with their, special data center advantages, and CUDA optimization that they perform on their own hardware. Hardware is CAPEX intensive. (Which is being pitched and funded even today)\nWe were saying the opposite, which defied most investors’ sensibilities - we were saying we would be avoiding buying new hardware like the plague.\nWe came to a realization, that most investors, their analysts, and founders failed to realize, thanks to the billions in hardware investments to date. GPUs are commodity hardware. Faster than all of us expected.\nFew investors have even realized we have reached commodity-level prices at $2.85 in certain places, let alone loss-making prices of a dollar. Because most providers (ignoring certain exceptions), only show their full prices after quotation or after login.\nAnd that was the trigger, which got me to write this article.\nWhile we do optimize our inference CUDA and kernels as well. On the hardware side; We’ve bet on hardware commoditizing and have focussed instead on the orchestration layer above.\nSo for us, this is a mix of sources from, AWS spot (preferred), to various data center grade providers (eg. Tensordock, Runpod) with security and networking compliances that meet our standards.\nLeveraging them with our own proprietary model hot swapping, which boots new models up in under a second. Keeping our fleet of GPUs right-sized to our workload, while using a custom version of our RWKV foundation model as a low-cost speculative decoder. All of which allows us to take full advantage of this market trend, and future GPU price drops, as newer (and older) GPUs come online to replace the H100s. And scale aggressively.\nPS: If you are looking at building the world's largest inference platform, and are aligned with our goals - to make AI accessible to everyone, regardless of language or status. Reach out to us at:\n[email protected]\nHead over to Eugene’s Blog\nTech Talk CTO\nfor\nmore footnotes on xAI’s H100 cluster\nwe cut from this piece.\nAdditional Sources:\nGPU data:\nTech Power Up Database\n. The A100 SXM had 624 bf16 TFlops, the H100 SXM was 1,979 bf16 TFlops\nMicrosoft & AWS allocated over $40 billion in AI infra alone:\nWall Street Journal\nNvidia investor slides for Oct 2014:\npage 14 has the pitch for “data centers”\nSemi Analysis:\ndeepdive for H100 clusters, w/ 5 year lifespan approx for components\nSpreadsheet for :\nnew H100 ROI (Aug 2024)\nSpreadsheet for:\nH100 Infiniband Cluster math (Aug 2024)\n1\nUnless you have some combination of discounted H100s, discounted electricity, or a Sovereign AI angle where the location of your GPU is critical to your customers, or you have billions and need a super large cluster for frontier model training\n2\nThe cited “600 Billion Dollars “ is about\nSequoia David Cahn article",
  "metadata": {
    "title": "$2 H100s: How the GPU Bubble Burst - by Eugene Cheah",
    "description": "H100s used to be $8/hr if you could get them. Now there's 7 different resale markets selling them under $2. What happened?",
    "keywords": null,
    "author": "Eugene Cheah",
    "og:type": "article",
    "og:title": "$2 H100s: How the GPU Bubble Burst",
    "og:description": "H100s used to be $8/hr if you could get them. Now there's 7 different resale markets selling them under $2. What happened?",
    "og:image": "https://substackcdn.com/image/fetch/$s_!upry!,w_1200,h_675,c_fill,f_jpg,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F333cc1d8-8e8b-47dc-b9f5-1a02e0eefd7b_2140x1280.png",
    "og:url": "https://www.latent.space/p/gpu-bubble",
    "twitter:title": "$2 H100s: How the GPU Bubble Burst",
    "twitter:description": "H100s used to be $8/hr if you could get them. Now there's 7 different resale markets selling them under $2. What happened?",
    "twitter:image": "https://substackcdn.com/image/fetch/$s_!Rkkw!,f_auto,q_auto:best,fl_progressive:steep/https%3A%2F%2Fswyx.substack.com%2Fapi%2Fv1%2Fpost_preview%2F150077899%2Ftwitter.jpg%3Fversion%3D4",
    "twitter:card": "summary_large_image",
    "article:modified_time": "2024-12-30T08:19:42.673Z"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [],
  "images": [
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!-6EA!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc92c0392-bdd9-4730-be45-2a408142239b_794x696.png",
      "bytes": 96536,
      "reused": true,
      "local_path": "attachments/a18b3cdd6d13/images/027.png",
      "src": "https://substackcdn.com/image/fetch/$s_!-6EA!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc92c0392-bdd9-4730-be45-2a408142239b_794x696.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!NTXQ!,w_2400,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F10d1b8c6-8c88-4579-a65d-3ce677c98d88_2798x1576.png",
      "bytes": 175576,
      "reused": true,
      "local_path": "attachments/a18b3cdd6d13/images/038.png",
      "src": "https://substackcdn.com/image/fetch/$s_!NTXQ!,w_2400,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F10d1b8c6-8c88-4579-a65d-3ce677c98d88_2798x1576.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!bIkE!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2e176ae6-a6d9-4d43-a6bd-827e436ce514_1659x1080.jpeg",
      "bytes": 273848,
      "reused": false,
      "local_path": "attachments/a18b3cdd6d13/images/body-e882caa8505abbab1a65.jpeg",
      "src": "https://substackcdn.com/image/fetch/$s_!bIkE!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2e176ae6-a6d9-4d43-a6bd-827e436ce514_1659x1080.jpeg",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!ESQO!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F14ef9576-25d7-4b63-b70b-57c442789fe3_2202x1248.png",
      "bytes": 104402,
      "reused": false,
      "local_path": "attachments/a18b3cdd6d13/images/body-5e8f8a639e97d4c21e58.png",
      "src": "https://substackcdn.com/image/fetch/$s_!ESQO!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F14ef9576-25d7-4b63-b70b-57c442789fe3_2202x1248.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!N364!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb11bd9db-9b98-4391-bd19-d92edb9174f9_1872x984.png",
      "bytes": 210719,
      "reused": false,
      "local_path": "attachments/a18b3cdd6d13/images/body-048570ef63a55c7774a2.png",
      "src": "https://substackcdn.com/image/fetch/$s_!N364!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb11bd9db-9b98-4391-bd19-d92edb9174f9_1872x984.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!UknB!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6d17ad06-f360-4482-8b16-7960bfbc1a9c_1878x978.png",
      "bytes": 220475,
      "reused": false,
      "local_path": "attachments/a18b3cdd6d13/images/body-dba878165c1a4236c018.png",
      "src": "https://substackcdn.com/image/fetch/$s_!UknB!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6d17ad06-f360-4482-8b16-7960bfbc1a9c_1878x978.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!upry!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F333cc1d8-8e8b-47dc-b9f5-1a02e0eefd7b_2140x1280.png",
      "bytes": 103809,
      "reused": false,
      "local_path": "attachments/a18b3cdd6d13/images/body-7c5d24362ee9848358d2.png",
      "src": "https://substackcdn.com/image/fetch/$s_!upry!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F333cc1d8-8e8b-47dc-b9f5-1a02e0eefd7b_2140x1280.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!izp8!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F23dbc794-e567-4b34-a6e7-761fafef2ccc_2710x1812.png",
      "bytes": 348643,
      "reused": false,
      "local_path": "attachments/a18b3cdd6d13/images/body-7ec73831cadede51f0f8.png",
      "src": "https://substackcdn.com/image/fetch/$s_!izp8!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F23dbc794-e567-4b34-a6e7-761fafef2ccc_2710x1812.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!QLeT!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdf705375-9942-4ea5-86b9-b41d3661096b_1842x556.png",
      "bytes": 140292,
      "reused": false,
      "local_path": "attachments/a18b3cdd6d13/images/body-c7c2c98aad9b5dbd0c4e.png",
      "src": "https://substackcdn.com/image/fetch/$s_!QLeT!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdf705375-9942-4ea5-86b9-b41d3661096b_1842x556.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!rJs2!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F55aee33d-e501-4923-9fd2-39740f781ed5_2610x826.png",
      "bytes": 88686,
      "reused": false,
      "local_path": "attachments/a18b3cdd6d13/images/body-07a5b42d979bf57a8df4.png",
      "src": "https://substackcdn.com/image/fetch/$s_!rJs2!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F55aee33d-e501-4923-9fd2-39740f781ed5_2610x826.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!o1_l!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc1dd60f1-1e51-4ec7-ac64-d5a2b506f2f0_2118x1078.png",
      "bytes": 75395,
      "reused": false,
      "local_path": "attachments/a18b3cdd6d13/images/body-09e10c5f0966d6ba5f41.png",
      "src": "https://substackcdn.com/image/fetch/$s_!o1_l!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc1dd60f1-1e51-4ec7-ac64-d5a2b506f2f0_2118x1078.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!BovM!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3e858927-de4c-499b-9ba0-261dd470a88b_1948x1088.png",
      "bytes": 109389,
      "reused": false,
      "local_path": "attachments/a18b3cdd6d13/images/body-8d749d7985f3f50be185.png",
      "src": "https://substackcdn.com/image/fetch/$s_!BovM!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3e858927-de4c-499b-9ba0-261dd470a88b_1948x1088.png",
      "alt": "",
      "downloaded": true
    }
  ],
  "audio": [],
  "videos": [],
  "links": [
    {
      "href": "https://news.ycombinator.com/item?id=41805446",
      "label": "HN",
      "type": "a"
    },
    {
      "href": "https://www.reddit.com/r/LocalLLaMA/comments/1g12ist/2_h100s_how_the_gpu_rental_bubble_burst/",
      "label": "Reddit",
      "type": "a"
    },
    {
      "href": "https://x.com/latentspacepod/status/1844563363877224889",
      "label": "Twitter",
      "type": "a"
    },
    {
      "href": "https://www.listennotes.com/podcasts/high-agency-the/why-your-ai-product-needs-ALy02ewNtDC/",
      "label": "Raza Habib’s chat with Hamel Husain",
      "type": "a"
    },
    {
      "href": "https://www.latent.space/p/humanloop",
      "label": "our Raza pod here",
      "type": "a"
    },
    {
      "href": "https://www.latent.space/p/rwkv",
      "label": "our pod on RWKV last year",
      "type": "a"
    },
    {
      "href": "https://featherless.ai/",
      "label": "Featherless.AI",
      "type": "a"
    },
    {
      "href": "https://x.com/OpenAI/status/1843704848753283356",
      "label": "rolling out to OpenAI",
      "type": "a"
    },
    {
      "href": "https://x.com/firstadopter/status/1844417947277852925",
      "label": "sold out for the next year",
      "type": "a"
    },
    {
      "href": "https://x.com/The_AI_Investor/status/1844080690046058843",
      "label": "most successful product in the history of the industry",
      "type": "a"
    },
    {
      "href": "https://www.cnbc.com/2024/10/10/amd-launches-mi325x-ai-chip-to-rival-nvidias-blackwell-.html",
      "label": "announcing the MI3 25 X",
      "type": "a"
    },
    {
      "href": "https://x.com/tunguz/status/1871192648830357637",
      "label": "a large software gap",
      "type": "a"
    },
    {
      "href": "https://news.ycombinator.com/item?id=41702789",
      "label": "Cerebras filing for IPO",
      "type": "a"
    },
    {
      "href": "https://www.latent.space/p/semianalysis",
      "label": "former guest",
      "type": "a"
    },
    {
      "href": "https://www.dwarkeshpatel.com/p/dylan-jon",
      "label": "Dylan Patel’s pod",
      "type": "a"
    },
    {
      "href": "https://www.semianalysis.com/p/ai-neocloud-playbook-and-anatomy",
      "label": "AI Neoclouds",
      "type": "a"
    },
    {
      "href": "https://www.latent.space/p/mar-jun-2024",
      "label": "the $600bn question",
      "type": "a"
    },
    {
      "href": "https://x.com/GavinSBaker/status/1720819375517716610",
      "label": "fastest depreciating asset in history",
      "type": "a"
    },
    {
      "href": "https://www.latent.space/i/140396949/mixtral-sparks-a-gpuinference-race-to-the-bottom",
      "label": "the GPU Rich Wars are raging",
      "type": "a"
    },
    {
      "href": "https://www.bloomberg.com/news/articles/2024-10-02/openai-has-closed-new-funding-round-raising-over-6-5-billion",
      "label": "raising $6.6b in the largest venture round of all time",
      "type": "a"
    },
    {
      "href": "https://www.theinformation.com/articles/openai-projections-imply-losses-tripling-to-14-billion-in-2026?rc=ytp67n",
      "label": "projecting losses of $14b in 2026",
      "type": "a"
    },
    {
      "href": "https://substackcdn.com/image/fetch/$s_!-6EA!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc92c0392-bdd9-4730-be45-2a408142239b_794x696.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://blog.lepton.ai/the-missing-guide-to-the-h100-gpu-market-91ebfed34516",
      "label": "Yangqing Jia’s guide",
      "type": "a"
    },
    {
      "href": "https://substackcdn.com/image/fetch/$s_!NTXQ!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F10d1b8c6-8c88-4579-a65d-3ce677c98d88_2798x1576.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://en.wikipedia.org/wiki/The_Tulip_Folly",
      "label": "The Tulip Folly",
      "type": "a"
    },
    {
      "href": "https://en.wikipedia.org/wiki/Tulip_mania",
      "label": "on tulip prices which kept climbing in 1634 and collapsed in February 1637",
      "type": "a"
    },
    {
      "href": "https://www.ft.com/content/c7e9cfa9-3f68-47d3-92fc-7cf85bcb73b3",
      "label": "multiple shipment delays",
      "type": "a"
    },
    {
      "href": "https://substackcdn.com/image/fetch/$s_!ESQO!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F14ef9576-25d7-4b63-b70b-57c442789fe3_2202x1248.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://en.wikipedia.org/wiki/Bagholder",
      "label": "left holding the bag",
      "type": "a"
    },
    {
      "href": "https://substackcdn.com/image/fetch/$s_!N364!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb11bd9db-9b98-4391-bd19-d92edb9174f9_1872x984.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://docs.google.com/spreadsheets/d/1kZosZmvaecG6P4-yCPzMN7Ha3ubMcTmF9AeJNDKeo98/edit?usp=sharing",
      "label": "new H100 ROI (Aug 2024)",
      "type": "a"
    },
    {
      "href": "https://substackcdn.com/image/fetch/$s_!UknB!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6d17ad06-f360-4482-8b16-7960bfbc1a9c_1878x978.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://www.latent.space/p/llama-3",
      "label": "Llama 3",
      "type": "a"
    },
    {
      "href": "https://substackcdn.com/image/fetch/$s_!upry!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F333cc1d8-8e8b-47dc-b9f5-1a02e0eefd7b_2140x1280.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://substackcdn.com/image/fetch/$s_!izp8!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F23dbc794-e567-4b34-a6e7-761fafef2ccc_2710x1812.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://substackcdn.com/image/fetch/$s_!QLeT!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdf705375-9942-4ea5-86b9-b41d3661096b_1842x556.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://docs.google.com/spreadsheets/d/1Ft3RbeZ-w43kYSiLfYc1vxO41mK5lmJpcPC9GOYHAWc/edit?usp=sharing",
      "label": "H100 Infiniband Cluster math (Aug 2024)",
      "type": "a"
    },
    {
      "href": "https://substackcdn.com/image/fetch/$s_!rJs2!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F55aee33d-e501-4923-9fd2-39740f781ed5_2610x826.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://substackcdn.com/image/fetch/$s_!o1_l!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc1dd60f1-1e51-4ec7-ac64-d5a2b506f2f0_2118x1078.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://substackcdn.com/image/fetch/$s_!BovM!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3e858927-de4c-499b-9ba0-261dd470a88b_1948x1088.png",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://www.latent.space/cdn-cgi/l/email-protection",
      "label": "[email protected]",
      "type": "a"
    },
    {
      "href": "https://open.substack.com/pub/techtalkcto",
      "label": "Tech Talk CTO",
      "type": "a"
    },
    {
      "href": "https://substack.tech-talk-cto.com/p/d4ffab7a-3f0d-4e6e-ade0-e74409770196?postPreview=paid&updated=2024-08-25T03%3A36%3A59.886Z&audience=everyone&free_preview=false&freemail=true",
      "label": "more footnotes on xAI’s H100 cluster",
      "type": "a"
    },
    {
      "href": "https://www.techpowerup.com/gpu-specs/h100-sxm5-80-gb.c3900",
      "label": "Tech Power Up Database",
      "type": "a"
    },
    {
      "href": "https://www.wsj.com/tech/ai/big-tech-moves-more-ai-spending-abroad-088988de",
      "label": "Wall Street Journal",
      "type": "a"
    },
    {
      "href": "https://s201.q4cdn.com/141608511/files/doc_presentations/2023/Oct/01/ndr_presentation_oct_2023_final.pdf",
      "label": "page 14 has the pitch for “data centers”",
      "type": "a"
    },
    {
      "href": "https://www.semianalysis.com/p/100000-h100-clusters-power-network",
      "label": "deepdive for H100 clusters, w/ 5 year lifespan approx for components",
      "type": "a"
    },
    {
      "href": "https://www.sequoiacap.com/article/ais-600b-question/",
      "label": "Sequoia David Cahn article",
      "type": "a"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:32:37Z",
  "source_url": "https://www.latent.space/p/gpu-bubble",
  "papers": [
    {
      "status": "ok",
      "url": "https://s201.q4cdn.com/141608511/files/doc_presentations/2023/Oct/01/ndr_presentation_oct_2023_final.pdf",
      "bytes": 8609256,
      "reused": false,
      "local_path": "papers/repair/1cf4750dea5c64e3ea92.pdf"
    }
  ],
  "actual_video_files": [],
  "body_chars": 27057,
  "expected_body_images": 12,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/a18b3cdd6d13/pages/source.html",
  "local_html": "attachments/a18b3cdd6d13/pages/article.html"
}