Together AI is the default serverless inference provider for open-source LLMs in 2026, broadest model selection (200+), competitive pricing across the cost curve, and a dedicated- endpoint path for production scale. The pick is workhorse- driven: when you need to serve Llama 3.3 70B, Qwen3, or DeepSeek V4 reliably at known per-token costs without operating GPUs, Together is the platform most teams reach for. The trade is generality, Groq is faster on a subset of models, Modal is better for custom Python, Replicate is broader for multimodal.
- 200+ open-source models with serverless per-token pricing
- Llama 3.3 70B at $0.88/M tokens; Qwen3 235B A22B at $0.20 input
- Dedicated endpoints from $6.49/hr (H100) for production scale
- Mamba-3 SSM, FlashAttention-4, ThunderAgent shipped 2026
- 600+ voices in TTS, fine-tuning, code sandboxes all in one platform
- Groq beats Together on raw speed for supported models
- No first-party closed-source models (use Anthropic / OpenAI direct)
- Dedicated endpoints add operational complexity vs serverless
- Frontier-model pricing (DeepSeek V4 Pro) approaches closed-source costs
- Smaller ecosystem of fine-tuning recipes than Hugging Face
Together AI is a serverless inference platform for open-source LLMs and generative models. The wedge is breadth + reliability: 200+ models from Llama and Qwen to DeepSeek and Mamba-3, all exposed through a unified per-token pricing model, with dedicated-endpoint upgrades when serverless economics stop making sense at scale.
Where it fits
Together AI is the right pick when you’re building on open-source models and want a single provider that covers the whole catalog without operating GPUs yourself. The serverless tier is shaped for variable workloads, pay per token, scale to zero, no minimum commitment. The dedicated tier (H100 from $6.49/hr) is the upgrade path for production apps where guaranteed throughput matters more than serverless flexibility.
For LLM-heavy workloads specifically, Together is competitive with Fireworks on most models, sometimes faster, sometimes cheaper. The Mamba-3 release (SSM-based, faster decode than Transformers) and FlashAttention-4 reflect the team’s investment in inference performance research that flows back into the managed platform.
Avoid Together when you need pure speed on a small set of models (Groq’s LPU is dramatically faster), when you’re running custom Python at the GPU (Modal is the better shape), or when you need the broadest multimodal catalog including image/video (Replicate covers that better).
Pricing in practice
Serverless: Llama 3.3 70B at $0.88/M input + $0.88/M output; Qwen3.5 9B at $0.10 input / $0.15 output; Qwen3 235B A22B at $0.20 input / $0.60 output. Frontier models (DeepSeek V4 Pro) land at $2.10 input ($0.20 cached) / $4.40 output, approaching closed-source pricing but with full open-weights control.
Dedicated endpoints: H100 80GB from $6.49/hr, scaling to HGX B200 at $11.95/hr. The crossover from serverless to dedicated is typically around 1B tokens/month sustained, below that, serverless is cheaper.
How it compares
Groq, Dramatically faster on supported models via LPU hardware (800+ TPS on Llama 3.1 8B). Pick when latency dominates the experience.
Fireworks AI, Comparable serverless platform with focus on speed and fine-tuning. Pick when you want a Together alternative with similar coverage.
Replicate, Broader multimodal catalog (image, video, voice) with per-second GPU billing. Pick when you need more than LLM inference.
OpenRouter, Router across 400+ models from 60+ providers, including closed-source. Pick when you want one API that spans Anthropic, OpenAI, and OSS.
What changed recently
Together shipped Mamba-3 in 2026 (SSM architecture, faster than Transformers at decode, stronger than Mamba-2), DeepSeek V4 serving with million-token context (compressed KV layouts + prefix caching), and FlashAttention-4 / ThunderAgent / together. compile at NVIDIA GTC 2026. Voice Finder shipped for searching 600+ TTS voices via natural language. The Pearl Research Labs partnership explored proof-of-useful-work for AI inference cost reduction on crypto-powered workloads, an experimental direction worth watching.
Sources
- Together AI Blog, together.ai, 2026
- Together AI Pricing, together.ai
- Mamba-3 Release, together.ai, 2026
- Together AI Docs, together.ai
- launch · 2026-07-12
Together AI puts a fixed price on reserved open-model capacity with Provisioned Throughput
Together AI's Provisioned Throughput sells reserved inference capacity for open models by the minute, with a 99% uptime SLA and predictable cost. It is a bet that teams running open weights in production want a bill they can forecast, not just a cheap per-token rate.
- beat · 2026-07-02
Together AI raises $800M at a reported $8.3B valuation as open-model inference demand compounds
Together AI's $800M Series C, reportedly at an $8.3B valuation, is one of the largest neocloud rounds yet. The raise reads as a bet that cheaper open-model inference keeps pulling workloads off frontier-token pricing.
- beat · 2026-05-30
Groq raising $650M to fund the inference neocloud pivot, Axios reports
The Axios scoop pegs Groq at up to $650M from existing investors, with Disruptive and Infinitum committing to backstop the round. Smaller than 2025's Series E and with no disclosed valuation, it reads as a Groq 2.0 raise.
- comparison · 2026-05-25
Inference platforms in 2026: how Modal, Replicate, Together, and Fireworks divide the category
After Modal's $355M Series C, the GPU inference category divides cleanly across four platforms: Modal as a Python runtime, Replicate for multimodal, Together as the OSS LLM workhorse, and Fireworks as Together's performance-focused peer.
- beat · 2026-05-24
Modal closes $355M Series C at $4.65B valuation
Modal raised $355 million at a $4.65 billion post-money valuation, led by General Catalyst and Redpoint. The round, which came together in two tranches, lands seven months after a $1.1 billion Series B and follows a fivefold revenue jump to roughly $300 million ARR.