Baseten vs Replicate (2026): H100 at $6.50 vs $5.49/hr, Per-Minute vs Per-GPU-Second Billing

Who Wins Per Workload

Match the row to what you are actually building. The winner flips on workload, not on a single spec.

Quick Comparison

Baseten optimizes for production reliability; Replicate optimizes for breadth and speed-to-ship. Morph is in the last column only as a reference point for one narrow case, applying model-generated code edits, where neither general host is the right tool.

GPU Pricing: Per-Minute vs Per-GPU-Second

Raw hardware is close. The billing granularity separates the bill. Baseten bills per minute of active compute with idle excluded; Replicate bills per GPU-second of run time and charges only while a prediction is running.

On A100 80GB, Baseten ($4.00/hr) undercuts Replicate ($5.04/hr) by about 21%. On H100 80GB, Replicate ($5.49/hr) is about 16% cheaper on the sticker. The catch is what you are billed for. Replicate bills per GPU-second of run time including setup, model load, and cold boots on private models, but only while a prediction runs, so a scaled-to-zero public model costs nothing while idle. Baseten bills per minute of active compute and explicitly excludes idle time. Replicate publishes no B200 or H200 rate; multi-GPU configs (2x H100 at $10.98/hr, 4x A100 at $20.16/hr) require a committed contract.

Per-Token Model API Pricing

Baseten publishes token-priced Model APIs on always-warm endpoints. Replicate prices a few language models per token (DeepSeek R1 at $3.75/M input and $10/M output, Claude 3.7 Sonnet at $3.00/M input and $15/M output) but bills most models by runtime, and image and video models by output: $0.04 per output image, $0.09 per second of output video.

A token-priced endpoint that stays warm sidesteps both the cold-boot wait and the cost of renting a whole GPU you may not saturate. For steady LLM traffic this is usually cheaper than either provider's dedicated GPU rate. If you only call a model occasionally, Replicate's scale-to-zero per-second billing on a shared GPU can be cheaper than holding a Baseten replica warm.

Cost on a Real Workload

Take one dedicated H100 running an LLM around the clock and ask which platform bills less for the same GPU. The arithmetic uses only the H100 rates on this page, so you can redo it.

The numbers above are arithmetic on the published GPU rates, not a throughput benchmark. Actual tok/s, cold-boot length, and the exact break-even depend on your model and traffic shape.

Cold Starts, Scale-to-Zero & Autoscaling

Both providers scale to zero by default, and on both a large model's first request after idle pays a cold boot measured in minutes. The difference is what that idle period costs you and how you configure warm capacity.

Replicate turns a model off when it has not been used for a little while; cold boots can take several minutes on large models. Because only running prediction time is charged on public models, those cold boots do not add cost, but a user waiting on the first request still feels the wait. Replicate's Deployments let you set a minimum instance count that stays warm to eliminate the cold start, and a maximum to cap spend, scaling from zero to hundreds of instances based on traffic.

Baseten defaults to min_replica=0, so a scaled-to-zero deployment incurs no charges, but the next request triggers a cold start that can take minutes for large models, and during wake-up billing is per minute even before the replica serves a response. Baseten's docs recommend min_replica of 2 or more for production to eliminate cold starts. Autoscaling defaults: max replicas 1, concurrency target 1 request per replica (model-specific guidance ranges from 1 for image generation to 256 for batched Whisper), and a 60-second autoscaling window configurable from 10 to 3600 seconds.

Neither is built for the coding-agent apply loop; if applying model-generated code edits is the bottleneck, that is a different tool (Morph Fast Apply, ~10,500 tok/s, with published benchmarks).

Rate Limits

Neither provider publishes a fixed per-model rate-limit table the way some serverless competitors do. On Baseten, concurrency is governed by your autoscaling config: the default concurrency target is 1 request per replica, so total throughput scales with max_replica. On Replicate, throughput scales with the instance count in a Deployment, from zero to hundreds of instances based on traffic. For a guaranteed fixed ceiling on either platform, you provision dedicated capacity rather than relying on a published shared limit. Serverless-first providers differ here: Fireworks caps at 6,000 requests per minute with a payment method, DeepInfra at 200 concurrent requests per account, and Together scales limits dynamically per model with sustained traffic.

The Serving Stack

Baseten gives you a tuned engine; Replicate gives you a container.

Baseten lets you pick the serving engine per workload, choosing between TensorRT-LLM, SGLang, and vLLM, and exposes token-priced Model APIs in an OpenAI-compatible format with function calling. Chains wires multiple models into one pipeline.

Replicate's serving surface is Cog: it generates a Docker image with sensible defaults (Nvidia base images, dependency caching, pinned Python) and exposes an HTTP API. You bring the framework. There is no built-in engine-selection layer the way Baseten ships one; you get a clean container boundary and a 50,000+ model catalog instead.

Deploying Your Own Model

Replicate is faster to publish; Baseten gives more control over how it runs.

On Replicate, you write a cog.yaml, define a predict function, and run cog push. The model gets an interactive GUI, a public or private HTTP API, and rolling updates with no downtime. Cog is open source, so you can run the same container on your own infrastructure. Fine-tuning is a first-class flow: Cog's training API lets users bring their own data to create derivative fine-tunes (SDXL on images, Llama on structured text).

On Baseten, you package with Truss and pick your serving engine, then tune autoscaling, concurrency, and hardware. Chains lets you wire multiple models into one pipeline. Baseten also ships a training product that keeps your weights portable. The tradeoff: more knobs and more setup than a single cog push.

Reliability & Compliance

Baseten is the heavier compliance story; Replicate covers the basics.

Baseten states on its pricing page that it is SOC 2 Type II certified and HIPAA compliant, the posture regulated and revenue-critical workloads need. Replicate provides SOC 2 but does not market the same HIPAA story. For healthcare or finance workloads with strict compliance needs, Baseten is the safer default.

How They Compare to Other Inference Providers

Baseten and Replicate are two points on a wider map. If you are price-shopping raw GPUs or per-token serverless inference, the cheapest provider is often neither of them.

DeepInfra publishes the cheapest dedicated H100 at $1.79/hr, and Modal's per-second billing lands near $3.95/hr. On serverless tokens, Baseten's GPT-OSS 120B at $0.10/$0.50 undercuts Fireworks, Together, and Groq. For a full provider-by-provider breakdown, see the related comparisons below.

One caveat if you are running DeepSeek: cheap serverless rates usually come from quantizing activations to fp8, which moves output away from the reference weights. Morph Open Source Models serve DeepSeek with 16-bit (bf16) activations and do not quantize activations to fp8, so output matches the reference weights when fidelity matters. morph-dsv4flash (DeepSeek V4 Flash) is $0.139 per 1M input tokens and $0.278 per 1M output tokens. For coding agents specifically, Morph adds codegen-tuned speculative decoding and custom low-level inference kernels built for code generation, which makes it the fastest and highest-quality option for that workload. See Morph pricing.

When to Use Baseten

• High-volume LLM serving. Token-priced Model APIs on an always-warm, OpenAI-compatible endpoint (GPT-OSS 120B at $0.10/$0.50 per M in/out) beat renting a whole GPU once traffic is steady.
• Regulated workloads. SOC 2 Type II plus HIPAA make Baseten the default for healthcare, finance, and other compliance-bound deployments.
• Performance-sensitive serving. Per-workload engine selection across TensorRT-LLM, SGLang, and vLLM, plus Chains for multi-model pipelines.
• Cost control on steady traffic. Per-minute billing that excludes idle time, with A100 80GB at $4.00/hr undercutting Replicate's $5.04/hr by about 21%.
• Warm production latency. Setting min_replica >= 2 removes cold starts for user-facing traffic that cannot wait minutes for a model to load.

When to Use Replicate

• Prototyping and shipping fast. A 50,000+ model catalog behind one API request, no serving stack to build.
• Image, video, and audio features. Models bill per output ($0.04 per image, $0.09 per second of video), a few lines of code to add a generative feature.
• Publishing your own model publicly. Cog plus cog push gives your model a GUI, an HTTP API, and a shareable page.
• Open-source portability. Cog is open source, so the same container runs on Replicate or on your own infrastructure with no lock-in.
• Bursty or low-volume traffic. Scale-to-zero plus per-GPU-second billing means an idle model costs nothing; you pay only while a prediction runs.

Frequently Asked Questions

Is Baseten or Replicate cheaper?

It depends on utilization. Replicate's H100 is $5.49/hr ($0.001525/s) versus Baseten's $6.50/hr ($0.10833/min), so the H100 sticker is lower on Replicate. On A100 80GB it flips: Baseten is $4.00/hr versus Replicate's $5.04/hr. Replicate bills per GPU-second only while a prediction runs, so a scaled-to-zero model costs nothing while idle, which wins for bursty traffic. Baseten bills per minute of active compute with idle excluded. For steady, high-throughput LLM serving, Baseten's token-priced Model APIs (GPT-OSS 120B at $0.10/$0.50 per M in/out) beat renting either GPU outright.

Does Replicate have cold starts and how long are they?

Yes. Replicate scales to zero by default and turns a model off after it sits idle; cold boots can take several minutes on large models. Only running prediction time is charged on public models, so cold boots do not add cost, but the user waiting on the first request still feels the wait. Replicate's Deployments set a minimum instance count that stays warm to remove the cold start, at the cost of an idle-GPU bill. Baseten also scales to zero (min_replica=0 default) but its docs warn the cold start can take minutes for large models and recommend min_replica >= 2 for production.

What is the difference between Baseten and Replicate?

Replicate, now owned by Cloudflare, is a model catalog: 50,000+ models behind one API request, one-command deployment via the open-source Cog tool, and per-GPU-second billing. It is built for prototyping and shipping generative features fast. Baseten is a production inference platform: engine selection across TensorRT-LLM, SGLang, and vLLM, token-priced Model APIs, per-minute dedicated GPUs, and SOC 2 Type II plus HIPAA. It is built for steady inference at scale.

Can I deploy a custom model on both?

Yes. Replicate uses Cog to package your model into a container and generate an API server, published with cog push, and the same container runs off-platform. Baseten uses Truss and lets you pick the serving engine plus Chains for multi-model pipelines. Baseten gives more control over the serving engine and autoscaling; Replicate is faster to publish and share publicly.

Which inference provider has the cheapest H100?

Among providers that publish a rate, DeepInfra is cheapest at $1.79/hr for a dedicated H100 80GB, then Modal at about $3.95/hr. Replicate and Together's GPU cluster are both $5.49/hr, Baseten is $6.50/hr dedicated, and Fireworks is $7.00/hr. For serverless LLM token serving instead of a raw GPU, compare per-token rates: GPT-OSS 120B runs $0.10/$0.50 per M in/out on Baseten versus $0.15/$0.60 on Fireworks, Together, and Groq.

Does Cloudflare owning Replicate change anything?

Cloudflare announced the acquisition of Replicate in November 2025 and is folding its model library into Workers AI, pushing the combined catalog past 50,000 models. For most users the product is unchanged: Cog packaging, the public catalog, and per-GPU-second billing all still apply. The direction is tighter Workers integration and edge-adjacent inference, not the dedicated, compliance-heavy production posture Baseten sells. If you want tuned serving engines and HIPAA, Baseten fits; if you want breadth of models and Cloudflare-native deployment, Replicate does.