A GitHub repo called Data Receipts landed on Hacker News this week with an uncomfortable finding: the same NVIDIA H100 SXM 80GB GPU costs $2.69 per hour on RunPod Community Cloud and $12.29 per hour on Azure — a 4.57x spread for identical silicon. The thread hit 124 points and 72 comments in hours, and the top comment set the tone: "You are completely mad if you are running any kind of valuable workload on Runpod community or Vast. Hosts are absolutely peeking inside the VMs."
📖 Read the full version with charts and embedded sources on ComputeLeap →
That single comment reveals the real story. The spread is not a pricing bug. It is a tax — a cost-opacity tax — and you pay it on every GPU hour and every API token you consume. You pay it because nobody in this industry publishes comparable units, and the cheapest option on every comparison chart is always the one charging you in something you cannot see.
View original discussion on Hacker News →
The 4.57x Spread: What You Are Actually Buying
The Data Receipts analysis does something unusually disciplined: it compares only on-demand list prices for the same GPU variant (SXM 80GB), in the same region class (US), using each provider's cheapest available SKU. No spot pricing, no reserved instances, no volume discounts.
View the full dataset on GitHub →
The result:
| Provider | $/GPU-hr | Monthly (730 hrs) |
|---|---|---|
| RunPod Community | $2.69 | $1,964 |
| RunPod Secure | $3.99 | $2,913 |
| AWS p5.48xlarge | $6.88 | $5,022 |
| Azure ND96isr H100 v5 | $12.29 | $8,972 |
The author is careful to note what the 4.57x is not: it is not InfiniBand interconnect (Azure's $12.29 includes 400 Gbps/GPU Quantum-2 InfiniBand with GPU Direct RDMA), it is not a datacenter SLA, and it is not enterprise compliance or committed capacity. Those things cost money. The question is whether they cost 4.57x more money, and the answer — if you are doing single-node fine-tuning — is obviously no.
The practical math gets worse when you add context. SemiAnalysis data shows H100 one-year contract pricing bottomed at $1.70/hr in October 2025 and rebounded to roughly $2.35–$2.80 by mid-2026. Dylan Patel, presenting at Modal's Runtime 2026 conference, put the current midpoint at $2.80 and said he would not be surprised if H100s pass $3/hr. Meanwhile, a separate comparison found that published rates span $1.80 to $12.29 per GPU-hour — a 6.8x spread — and that availability and egress fees narrow the gap less than you would hope.
ℹ️ The real spread is wider than 4.57x. If you include spot and marketplace pricing, the cheapest captured H100 rate drops to $0.31/hr (spot), putting the spread against Azure's $12.29 at nearly 40x. But spot comes with no availability guarantee and zero SLA — you are renting headroom that can vanish mid-job.
The Privacy Price Tag Nobody Lists
The HN thread's sharpest insight was not about price but about what the cheap price costs you. The top commenter reported that Vast.ai and RunPod Community hosts "are absolutely peeking inside the VMs" — and that this is "openly discussed in their Discords."
This is the part of the price comparison that never appears in any comparison chart. RunPod splits its inventory into two tiers: Secure Cloud (RunPod's own datacenters, SOC 2 Type II, ISO 27001) and Community Cloud (third-party hosts renting their own hardware, no SLA). The $2.69 figure is Community. Secure is $3.99 — a 48% premium just for the assurance that your host is not inspecting your workload.
Vast.ai is even more explicit. It is a peer-to-peer marketplace. You choose a GPU and a landlord. Privacy depends entirely on who that landlord is. As one analysis put it, effective costs on unverified Vast hosts run 20–40% higher after accounting for downtime and restarts, and the security tradeoffs of peer-to-peer compute are left entirely to the buyer.
We wrote about a parallel dynamic in Cursor Router: 60% Savings. It Also Sees Every Prompt. The pattern is the same: the cheapest compute is always the compute that can see what you are doing with it. Model routers see your prompts. Community GPU hosts see your weights. Budget token providers see your data. The discount is not free — it is priced in privacy.
⚠️ Contrarian Corner: the opacity is a feature. Hyperscalers deliberately bundle InfiniBand, compliance, egress, and SLA into a single per-GPU-hour price because unbundling would reveal that raw compute is a commodity with razor margins. Transparency would commoditize them — so they fight it. The price spread is not an accident of a young market. It is a moat.
Token-Level Opacity: Same Disease, Different Organ
The GPU price spread is the hardware layer of a problem that runs all the way up the stack. At the software layer, the same disease shows up as tokenomics chaos.
This week at AI Engineer NYC 2026, Pratyusha Duvvuri from Cisco presented a talk titled "Why No One Agrees on What Their AI Tokens Cost." The backstory: after Cisco acquired Galileo, an AI observability platform, they surveyed roughly 100 enterprise companies about their token-cost accounting. The result: roughly 100 different definitions.
This is not a trivial definitional disagreement. It is a measurement crisis. Consider what "cost per token" could mean:
- The API rate card: Claude 4 Sonnet at $3.30/$16.50 per million input/output tokens (DeepInfra pricing, April 2026). This is what most people compare.
- The effective rate with caching: DeepSeek V3.2 drops from $0.26 to $0.13/M tokens with cached input — a 50% discount that only applies if your workload hits the cache.
- The orchestration cost: A single agentic task may involve dozens or hundreds of inference calls. OpsLyft found that hidden costs — retrieval, embeddings, context management, retries — add 40–60% on top of tracked inference bills.
- The total cost of inference: As the FinOps Foundation argues, only inference is priced in tokens. Cloud compute, networking, egress, data licensing, and governance are separate line items that never appear in a tokens/dollar comparison.
The result is a market where GPT-4-equivalent performance costs about $0.40 per million tokens, down from $20 in late 2022 — a 98% drop — while enterprise AI budgets have tripled from $1.2 million to $7 million per year. Artefact calls this the "token cost illusion." Per-token prices are falling. Total bills are rising. And nobody can explain the gap because they are measuring different things.
ℹ️ By the numbers: Uber reportedly used its entire 2026 AI coding budget by April. Microsoft spent $500–$2,000 per engineer per month on tokens before revoking Claude Code licenses. Per-developer token consumption rose roughly 18.6x in nine months, per Jellyfish data. Engineers who used the most tokens were about twice as productive — but consumed roughly 10x the tokens.
We have covered the per-token trap before — see our analysis of why AI got cheaper and your bill didn't. But the tokenomics problem has gotten worse because agentic workflows have multiplied consumption per task by 5–30x (Splunk State of Observability 2026) while the unit of measurement has stayed the same.
The Industry's Attempt at a Fix
The measurement crisis got bad enough that the Linux Foundation launched the Tokenomics Foundation in August 2026, with 30 founding members and a mandate to define open standards for AI token usage and billing. Their proposed metrics include cost-per-intelligence and tokens-per-watt — units that attempt to connect consumption to outcomes and physical resources rather than abstract API pricing.
Read the Linux Foundation announcement →
The FinOps Foundation, which already maintains the open FOCUS billing specification for cloud costs, is working in parallel. FOCUS 1.4 now includes cloud billing normalization, and the two foundations are coordinating on how to extend it to AI-specific spend.
Salesforce's Nishant Gupta captured the mood: "Token economics is fundamentally more abstract and opaque than anything we've managed at this scale before." And the FinOps Foundation's J.R. Storment warned that token tracking is "a trillions-of-rows-a-month data problem" — orders of magnitude beyond traditional cloud billing.
The vendor ecosystem is not waiting for standards. Pay-i, Jellyfish, Waydev, Faros AI, Datadog, New Relic, and now Cisco's Splunk (via Galileo) are all building AI-cost observability products. Factory launched a model router that picks the cheapest adequate model per task. But without a common unit of measurement, each tool measures cost differently — which means comparing observability tools is itself an opacity problem.
What This Means for You
If you are a founder, CTO, or platform engineer making compute decisions right now, here is the framework that the price comparisons will not give you:
1. Normalize for what is included, not what is listed. The $2.69 RunPod Community hour does not include isolation guarantees, an SLA, or InfiniBand. The $12.29 Azure hour does. Neither price includes egress ($0.01–$0.09/GB depending on provider), storage, or the engineering time to manage the deployment. Build a total-cost model that includes at minimum: GPU-hour rate, interconnect requirements, egress at your data volume, SLA value (what does an hour of downtime cost you?), and privacy/compliance posture.
2. Separate training from inference economics. Multi-node training requires InfiniBand interconnect — the cheap tier literally cannot do the job. Single-node fine-tuning and inference do not. As the Data Receipts analysis notes, single-node work pays for interconnect it will never use on Azure. Match the workload to the tier.
3. Budget for tokens-per-task, not tokens-per-call. Agentic workflows are the new normal. A single agent task can burn 50,000–500,000 tokens before producing output. Engineer-level coding assistants consume millions of tokens per developer per day. If your budget assumes per-seat pricing, you are understating costs by an order of magnitude. We covered this in detail in AI's $700B Subsidy Clock Is Ticking.
4. Treat privacy as a line item. The cheapest GPU and the cheapest token both come with a privacy cost. Community GPU hosts see your workloads. Budget token providers process your data. If you are working with proprietary models or sensitive data, the $1.30/hr premium for RunPod Secure over Community is not a luxury — it is a security requirement. And even "secure" tiers may not meet regulatory requirements (SOC 2 and ISO 27001 are not HIPAA or FedRAMP).
💡 Your compute-cost normalization checklist:
- GPU-hour rate (on-demand vs. reserved vs. spot)
- Interconnect tier (InfiniBand for multi-node? Or not needed?)
- Egress fees at your actual data volume
- SLA terms and downtime cost
- Privacy/compliance certification level
- Token orchestration overhead (retries, context window, RAG pipeline multiplier)
- Vendor lock-in switching cost
5. Watch the standards bodies. The Tokenomics Foundation is early, but its proposed metrics — cost-per-intelligence, tokens-per-watt — point toward a future where compute costs are comparable across providers. Until that future arrives, you are doing the normalization yourself. The companies that build internal cost observability now will have a structural advantage when the industry finally agrees on how to count.
The Prediction
The 4.57x GPU spread will compress. Raw H100 compute is commoditizing — as we wrote in Compute Is Now a Tradable Commodity, the emergence of GPU price indices and futures markets will erode the information asymmetry that sustains the spread. SemiAnalysis and Silicon Data already publish real-time price data; CME Group is exploring compute futures.
But the opacity will not disappear. It will migrate. Today it lives in GPU-hour pricing and per-token rates. Tomorrow it will live in agentic task pricing, where a single "agent run" bundles compute, orchestration, tool calls, retries, and data processing into a single opaque fee. The companies that survive this transition will be the ones that can decompose that fee and allocate each component to business value.
Goldman Sachs projects global token usage will grow 24x by 2030. If the unit of measurement is still a mess when that happens, the cost-opacity tax will be the largest invisible line item in every AI budget.
The tax is not the price. The tax is not knowing what the price actually means.
Originally published at ComputeLeap
Top comments (0)