SXF / AI · OPEN MODEL INFRASTRUCTURE · INFERENCE COST LAB
Best Open-Weight LLM Hosting Platforms (2026): Together AI vs Fireworks vs Runpod vs Modal
Where should you host an open-weight model for a production API? Compare two managed token APIs and two custom-GPU paths using verified November 2026 prices, the same gpt-oss-120b reference model, an original break-even calculator and a reproducible research protocol.
Which is the best open-source LLM hosting platform in 2026?
Together AI and Fireworks AI are strong starting candidates for hosted per-token APIs. Runpod is suited to custom container and GPU controls; Modal to code-first inference deployments and elastic compute. Modal also has token-priced Shared Endpoints. No honest universal speed winner exists without matching model, workload and region. Compare hosting modes.
PLATFORM SHORTLIST
Together AI vs Fireworks vs Runpod vs Modal: which LLM hosting platform should you choose?
Direct answer: Together AI and Fireworks offer ready-to-call token-billed model APIs; Runpod offers Pods and Serverless GPU workers; Modal offers custom GPU deployment as well as token-billed Shared Endpoints. There is no meaningful universal winner without fixing the model, traffic and service tier.
| Decision factor | Together AI | Fireworks AI | Runpod | Modal |
|---|---|---|---|---|
| Fast starting point | Serverless model API | Serverless model API | Managed public endpoints or custom Serverless workers | Shared Endpoint or custom Python GPU service |
| Billing path in this study | Input/output tokens | Input/output tokens | Serverless GPU-worker hours | Custom GPU-second resources |
| Control over serving | Dedicated endpoint options | On-demand deployment options | Container and GPU configuration | Python code, images and autoscaling controls |
| Best first fit to evaluate | Fast managed API integration | Managed model APIs and dedicated serving | Custom engine, GPU and container requirements | Code-first inference pipeline |
| Cost question | Cached and uncached token mix | Input, cached-input and output mix | Worker uptime and scaling mode | GPU, CPU, memory and plans |
Vendor capabilities are based on official product descriptions, not hands-on SXF tests. Together and Fireworks also sell dedicated compute; Modal offers Shared Endpoints; Runpod products vary by deployment type. See Together, Fireworks, Runpod and Modal.
SEARCH DEFINITION
What is open-weight LLM hosting?
Direct answer: Open-weight LLM hosting runs downloadable model parameters on external infrastructure and exposes an inference API. Depending on the product, the customer is billed per token, dedicated throughput or billable GPU time. Open-weight does not automatically mean every license qualifies as open source.
Choosing model weights, installing a runtime and running an internet-facing production service are different engineering tasks. An open-model catalogue helps pick models and licenses; local LLM deployment helps size a workstation; this guide answers the commercial question of where to operate an API at a measurable cost.
UNDERSTANDING THE BILL
Per-token LLM API pricing vs GPU hourly pricing: which is cheaper?
Direct answer: Neither billing model is universally cheaper. A token API charges for input and generated output; a GPU worker charges for billable compute time, potentially including warm-idle periods and extra resources. Normalize both to the same completed workload, latency and model quality.
Token cost = (input tokens × input price + output tokens × output price) / 1,000,000. GPU cost = billable worker hours × hourly GPU rate plus CPU, RAM, storage, network, replica and operations cost where applicable. A large monthly token count alone says little about GPU utilization or capacity.
Calculate cost per 1,000 independently accepted responses with the same prompt distribution, rather than comparing $/million tokens against $/GPU-hour or dividing by raw HTTP 200 responses.
For very sporadic jobs, avoiding provisioned idle workers may favor a managed token API. For sustained demand, a custom serving engine may become attractive—but only if measured throughput satisfies the workload. Longer prompts, output-heavy tasks and concurrency change the break-even point.
SAME-MODEL TOKEN PRICES
Together AI vs Fireworks AI: which is cheaper for GPT-OSS 120B?
Direct answer: Both vendors publicly list $0.15 per million input tokens and $0.60 per million output tokens for gpt-oss-120b on the checked serverless tier. This is a price-list tie for one model, not an observed comparison of accuracy, time to first token or reliability.
The model IDs are openai/gpt-oss-120b on Together and accounts/fireworks/models/gpt-oss-120b on Fireworks. At 100,000 requests with 1,000 input and 250 output tokens each, the illustrative combined monthly list-price bill is $30.00 on either service, before volume contracts, cache effects and taxes.
At Fireworks the model card also distinguishes a much lower cached-input rate. Therefore workload caching can alter the comparison. Do not treat marketing latency, unspecified reasoning configurations or model-name parity as independently controlled experiments. See the Together model card and Fireworks model card.
HOURLY GPU ECONOMICS
Runpod vs Modal inference: which GPU hosting option costs less?
Direct answer: For the specific published examples checked, Runpod's H100 PRO Serverless worker costs $5.39 per GPU-hour, while Modal's H100 SXM5 rate is $0.001097 per GPU-second, or $3.9492 per hour. These are different GPU and product configurations, not a measured like-for-like inference benchmark.
At 120 billed GPU hours, the published GPU component is $646.80 on that Runpod Serverless tier versus $473.90 on Modal. Modal can bill CPU and memory separately; Runpod's Pods have separate price bands from its Serverless workers. Load times, autoscaling delays, minimum workers, storage, security requirements and available hardware must be included in an invoice-level comparison.
Modal also provides token-priced Shared Endpoints, so describing Modal exclusively as GPU-second billed is incorrect. Similarly, Together and Fireworks offer dedicated deployment products. The choice is between specific offerings, not arbitrary vendor labels. Sources: Runpod and Modal.
DATED PRICE LEDGER
How much does open-source LLM hosting cost in November 2026?
Direct answer: On the checked public list prices, GPT-OSS 120B serverless token rates are $0.15 input and $0.60 output per million at both Together and Fireworks. H100-class GPU examples are $5.39/hour for Runpod Serverless and $3.9492/hour for Modal GPU time before ancillary costs.
| Product | Unit | Verified list example | Exclusions | Source |
|---|---|---|---|---|
| Together AI gpt-oss-120b | 1M input/output tokens | $0.15 / $0.60 | Batch, cache, contracts | Model card |
| Fireworks AI gpt-oss-120b | 1M input/output tokens | $0.15 / $0.60 | Cached input, deployment tier | Model card |
| Runpod H100 PRO Serverless | GPU-worker hour | $5.39 | Worker time, start/stop and storage | Pricing |
| Modal H100 SXM5 | GPU-second | $0.001097 = $3.9492/hr | CPU, RAM, storage and plan terms | Pricing |
USD, reviewed November 12, 2026. Prices may change. The GPU SKUs differ; do not infer equivalent benchmark throughput or compute hardware from this table.
ORIGINAL SXF COST MODEL
LLM hosting cost calculator: when does renting a GPU beat a token API?
Direct answer: The break-even GPU-hour budget is the token-API bill divided by the all-in modeled hourly GPU cost. That threshold is meaningful only if the GPU can actually complete the requested outputs and meet your latency budget.
This original SXF model runs locally in your browser. The starting scenario uses verified token and GPU rates; request volume, GPU hours, overhead and acceptance are hypothetical, not vendor performance measurements. Change any assumption to inspect sensitivity.
SXF / AI — INFERENCE ECONOMICS FRONTIER
Calculate a workload, not a marketing price
Illustrative scenario. Actual GPU capability and performance are not measured.
Formula: API bill = requests × (input tokens × input price + output tokens × output price) / 1M. GPU bill = hours × GPU rate × (1 + assumed overhead). Break-even hours = API bill / modeled GPU hourly cost. Required completion throughput = output tokens / (break-even hours × 3600), excluding prefill, queues and model loading. Accepted cost = monthly cost × 1000 / accepted requests.
Why idle GPU hours change the result
| GPU hours | Runpod GPU-only | Modal GPU-only | 100K-call API example |
|---|---|---|---|
| 10 | $53.90 | $39.49 | $30.00 |
| 120 | $646.80 | $473.90 | $30.00 |
| 730 (always warm) | $3,934.70 | $2,882.92 | $30.00 |
Scenario table includes GPU rates only; does not include the calculator's hypothetical 15% overhead. It does not imply the workload is feasible within 10 worker hours.
SXF BUYER DECISION PATH
Should you use a serverless LLM API or dedicated GPU hosting?
Direct answer: Start with quality and license requirements, then check whether you need custom weights. If not, two token APIs establish a useful baseline; if yes, benchmark a dedicated endpoint or controllable GPU worker under real target concurrency.
1. Can you use a listed model?
Choose a token API for a quick baseline when the model, context window and safety controls fit. Verify model ID, region and quotas.
2. Do you need your own checkpoint?
For quantized weights, a custom tokenizer or fine-tuned model, evaluate dedicated deployment or container-friendly GPU services.
3. Is traffic steady or bursty?
Count idle worker time and cold starts; compare demand by hour, not only monthly totals. Test capacity and latency at peak concurrency.
4. Does the response pass?
Keep schema validity, human acceptance, retries and cost of operational errors in the denominator of the final business decision.
If you also need multi-provider routing, fallback and cached responses, the SXF LLM Gateway guide covers that separate layer.
REAL WORLD PERFORMANCE
How should you compare LLM inference latency, throughput and cold starts?
Direct answer: Use one model, one prompt mix and recorded concurrency. Report p50/p95 latency, time to first token for streamed runs, output tokens per second, 429/5xx failures, billable worker time and answer quality. Do not turn vendor marketing metrics into independent test scores.
In production, a lower $/GPU-hour can lose to a faster or more highly utilized deployment. Prompt prefill, long context, tool-call format, reasoning configuration, streaming overhead and batch scheduling affect outcomes. Test at concurrency 1, 4 and 16 and publish medians plus tail latencies—never just a single favorable screenshot.
SXF has not spent tokens on paid provider APIs for this page and does not claim measured cross-provider speed. The replication kit below provides a preregistered method and safe opt-in probe so measurements can be published only after they exist.
REPRODUCIBLE SXF RESEARCH
How can you reproduce an open-weight LLM hosting benchmark?
Direct answer: Pin the same model and output budget, repeat identical synthetic task sets at controlled concurrency, record raw errors and invoices, and report both successful and rejected answers. Test cold and warm starts separately and never claim a throughput result from a latency-only script.
The original SXF Hosting Protocol v1 is published as a Markdown research plan and CSV scorecard. The optional Python standard-library probe makes no network requests unless a user explicitly supplies a trusted HTTPS endpoint, model ID and environment-held API key. It measures non-streaming end-to-end timing and reported token usage—not TTFT, GPU throughput or accepted-answer quality. Never use customer data or commit credentials.
- Register the model ID, checkpoint, tokenizer and engine.
- Use short, long-context and output-heavy synthetic prompts.
- Repeat controlled trials in matching regions.
- Separate cold starts from warmed inference.
- Measure tail latency, failures and accepted output independently.
- Use dated invoices or exact worker-time billing.
- Include non-GPU overhead and service limits.
- Publish anonymized raw data and uncertainty before ranking vendors.
See the SXF benchmark methodology to understand why a test score does not equal production reliability.
PRODUCTION DUE DILIGENCE
What hidden costs and security risks matter when hosting open LLMs?
Direct answer: The overlooked costs are idle workers, additional CPU and memory, storage, network, engineering maintenance, caching behavior, failed outputs and compliance. The exact model license and data boundary matter more than the provider's marketing label.
| Risk | Before committing |
|---|---|
| License and provenance | Pin weight revision, source, license, acceptable uses and redistribution rights. |
| Data locality | Confirm region, logging, subprocessors, retention and data-processing terms. |
| GPU idle burn | Inspect worker minimums, scale-to-zero and deployment startup costs. |
| Operational availability | Measure p95, throttling, retries, SLA and incident response. |
| Model drift | Pin checkpoints and test output acceptance after provider engine updates. |
| Surprise bill | Get actual token and GPU usage; count CPU, memory, egress and support. |
For locally hosted model memory, compare 4-bit vs 8-bit quantization and test with the SXF VRAM Finder.
EXTEND THE SHORTLIST
What alternatives exist to Together AI, Fireworks, Runpod and Modal?
Direct answer: Hugging Face Inference Endpoints and Baseten offer additional commercial model-serving paths. The correct shortlist depends on the exact checkpoint, regions, operations support and billing contract—not a universal ranking of cloud brands.
Hugging Face Inference Endpoints documents endpoint compute rates, useful when the team is already working from Hugging Face model repositories. Baseten Model APIs offers hosted model API products alongside deployment options. Evaluate them with the same SXF test protocol rather than treating the four headline providers as the entire market.
DIRECT SEARCH ANSWERS
Open-source LLM hosting: frequently asked questions
Direct answer: Choose a model and billable service tier first, then compare token economics, GPU-worker time and accepted-output quality. None of those units is interchangeable on its own.
What is the best open-source LLM hosting platform?
For ready-made managed model APIs, compare Together AI and Fireworks AI. For custom GPU/container control, evaluate Runpod and Modal. Match model availability, measured latency, traffic pattern, license and the full bill before choosing.
Is Together AI cheaper than Fireworks AI?
For the gpt-oss-120b public serverless model price checked on November 12, 2026, both list $0.15 per million input tokens and $0.60 per million output tokens. That is a list-price tie, not a measured performance tie.
How much does it cost to host gpt-oss-120b?
A hypothetical 100,000 monthly requests averaging 1,000 input and 250 output tokens cost $30 on the listed Together or Fireworks serverless rates before credits, caching and additional charges. GPU-hosted deployments require measured worker hours and additional resources.
Is Runpod cheaper than Modal for inference?
Not inherently. Runpod offers Pods and Serverless worker tiers; Modal charges GPU time and may charge CPU, memory and other resources. Their published H100 examples are different product configurations, so compare actual workload costs rather than bare hourly numbers.
What is the difference between token billing and GPU hourly billing?
With token billing, the provider charges for input and generated tokens. With GPU billing, you pay for billable worker compute time, potentially including idle periods or extra resources. Neither price describes cost per accepted answer without workload measurements.
When should I self-host an LLM rather than use a serverless API?
Self-host or use custom GPU deployment when you need custom weights, special inference controls, stricter runtime boundaries or sustained utilization. For unpredictable low volume, a per-token API often reduces idle-capacity exposure.
Can I use a custom Hugging Face model?
Custom weight support varies by provider and product. Check the exact model license, supported serving engine, hardware, region and deployment documentation. A serverless model API does not automatically accept arbitrary weight files.
How should I benchmark inference hosting platforms?
Pin the same model and prompt distribution, set identical output limits and concurrency, test cold and warm starts, and record p50/p95 latency, token usage, errors, actual cost and answer acceptance. Only compare TTFT if streaming is measured.
Does schema markup guarantee a Google ranking?
No. Accurate Article and BreadcrumbList markup can help search systems understand the page, but indexing, ranking and special result features are determined independently by the search engine.
RESEARCH TRANSPARENCY
How did SXF research this LLM hosting comparison?
Direct answer: SXF reviewed primary-source provider documentation on November 12, 2026, normalized the published gpt-oss-120b serverless prices, calculated GPU-hour scenarios and provided an independent research protocol. No live paid-provider latency or GPU-capacity experiment was performed.
Evidence tiers: officially published prices and product characteristics; SXF interpretations of deployment fit; illustrative modeling using editable assumptions; and explicitly unmeasured latency and accepted-output performance. This is a source-led purchasing analysis with an original workload-normalization method, not a claim of independent vendor benchmarking.
Target search terms include best open source LLM hosting, Together AI vs Fireworks, Runpod vs Modal inference, local LLM, LLM hosting cost comparison and open-source LLM inference. Some adjacent local-model queries appear in SXF Search Console exports; this is not evidence of measured search volume for every target term, nor a guarantee of rankings.