Inference platforms in 2026: how Modal, Replicate, Together, and Fireworks divide the category

Modal’s $355 million Series C, announced May 21, 2026, is the largest financing round any GPU inference platform has raised. It is also a useful prompt to take stock of the broader category. The four platforms most teams evaluate (Modal, Replicate, Together AI, and Fireworks AI) compete on overlapping problems but solve them in genuinely different shapes. Picking the right one starts with understanding which shape fits the workload.

How the category divides

The four platforms can be sorted along two axes: how much code you write yourself, and which modality you optimize for.

On the custom-code-versus-managed-API axis, Modal sits at one end: you write Python, you deploy that Python to a GPU, you control what happens on the box. Replicate, Together, and Fireworks sit at the other end: you call an API, the platform handles inference. Modal is infrastructure; the other three are products.

On the modality axis, Replicate is broad (image, video, voice, LLM all behind one API). Together and Fireworks are LLM-focused (text-in, text-out, with growing TTS support). Modal is modality-agnostic by definition since you write the code.

Most teams pick one platform per workload type, not one platform for everything. The real comparison is between Together and Fireworks (direct peers), with Modal and Replicate occupying their own lanes.

Modal: the Python runtime

Modal’s wedge is full Python control. You write a function, decorate it, specify the hardware, and deploy. The model loads in-process inside your code rather than living behind a hosted API. That makes Modal the right fit for custom inference pipelines, GPU-accelerated batch jobs, fine-tuning loops, and any workload where you need to combine model inference with surrounding logic in a single execution context.

Per-second billing runs from T4 ($0.000164/s) to B200 ($0.001736/s), with $30/month in free credits on the Starter tier. The Series C announcement signaled a deepening commitment to three areas: low-latency inference primitives, an “open inference stack”, parallel sandbox execution for agent workloads, and reinforcement learning infrastructure. Customers should expect those investments to show up as platform changes over the next two quarters.

The trade with Modal is operational. You write the inference code. You handle batching. You manage cold starts. For teams already running custom AI pipelines or that need primitives the managed APIs do not expose, the operational cost is worth paying. For teams that just need a Llama endpoint, it is not.

Replicate: the multimodal home

Replicate covers image (Flux, Recraft, SDXL), video (WAN, Veo, Sora-style models), voice (ElevenLabs-style synthesis), and LLM inference behind a single API. The Cog packaging format lets teams bring their own models when the catalog does not cover their need. Per-second GPU billing on T4 through H100 with no minimum commit.

The right shape for Replicate is the application that mixes modalities: a chat product that generates images, an agent that produces video, a creator tool that combines voice and image. The single-API surface across modalities is the wedge.

For pure LLM inference, Together and Fireworks are faster and cheaper. Image generation at $0.04 to $0.09 per output and video at $0.09 to $0.25 per second of output are competitive but specialized vendors like Fal undercut Replicate on image-only workloads.

Together AI: the OSS LLM workhorse

Stackmaven’s editor’s pick in the inference category in 2026 is Together. The reasoning is workhorse-driven: when a team needs to serve Llama 3.3 70B, Qwen3, DeepSeek V4, or one of 200-plus other open-source models reliably at known per-token costs without operating GPUs, Together is the default.

The numbers anchor the position. Llama 3.3 70B runs at $0.88 per million tokens. Qwen3 235B A22B input runs at $0.20 per million. Dedicated endpoints start at $6.49 per hour for H100 capacity when serverless scales out. Together shipped Mamba-3 SSM, FlashAttention-4, and the ThunderAgent runtime in 2026, plus a 600-plus voice TTS catalog and managed fine-tuning. It is the broadest serverless inference platform in the OSS LLM category.

The trades are real but narrow. Groq is faster than Together on a small subset of models. Modal is the better choice for custom Python. Replicate is broader for multimodal. For OSS LLM serving specifically, Together is the platform most teams should reach for first.

Fireworks AI: Together’s performance-focused peer

Fireworks is the closest direct competitor to Together. Broad OSS model coverage, competitive per-token pricing, dedicated endpoints, fine-tuning workflows. The team’s pitch is performance: claims of 250 percent higher throughput and 50 percent faster than open-source serving engines, which matters most for chat UX where tail latency is visible to users. On-demand GPU pricing runs from $7 per hour (H100/H200) to $12 per hour (B300). Embedding models are tiered from $0.008 to $0.10 per million tokens depending on model size.

The decision between Together and Fireworks usually comes down to two questions. Which platform has the specific model you need at the price you want? And how visible is tail latency in your application? For most teams the answer is “either works, run the benchmark on your traffic.” Together’s broader catalog tilts the default; Fireworks’ performance claim tilts the trial.

What you would actually pick

The decision tree, simplified to one read:

1. Custom Python code that runs on a GPU. Modal.
2. Image, video, or voice generation, or a multimodal app. Replicate.
3. OSS LLM inference at scale, default choice. Together.
4. OSS LLM inference where tail latency is the bottleneck. Fireworks.

The platforms are not fully substitutable. A team running a Llama endpoint on Together cannot simply move to Modal without rewriting the inference layer. A team using Replicate for image generation would not pick Modal unless they have a strong reason to manage the GPU code themselves. The category divisions are real.

Where the category is heading

Modal’s Series C, the size of the round, and the explicit investment in low-latency inference primitives, sandbox capacity, and reinforcement learning infrastructure suggest the company sees room to expand its lane upward (more agent-platform workloads, more training-plus-inference loops) rather than competing directly with Together or Fireworks on managed LLM endpoints.

The managed-LLM category itself is consolidating in pricing. Together and Fireworks are within 10 to 20 percent of each other on most flagship models; Replicate is the multimodal premium. The shape teams should watch over the next two quarters is which platform first builds a credible continual-learning loop, since that is the lane LangChain Labs just publicly staked out and the inference platforms are the natural infrastructure layer to ship it on. Fireworks and Baseten are named partners on LangChain Labs; Together is not.

That partnership map is worth tracking. It hints at which inference platforms are positioning to become more than managed-API shops as agent workloads grow.

Sources cited

1. Modal: Series C announcement (May 21, 2026) modal.com
2. Together AI: Inference and pricing www.together.ai
3. Fireworks AI: Pricing and performance fireworks.ai
4. Replicate: Pricing and model catalog replicate.com