Modal is the serverless GPU for Python, write a function with a decorator, specify the hardware, deploy. The wedge is full Python control: you're not consuming a model behind an API, you're running your own code on a GPU with the model loaded in-process. Per-second billing on every GPU tier from T4 to B200 with $30/month free credits on the Starter tier. The trade is operational: this is infrastructure, not a managed model API, you write the inference code, handle batching, manage cold starts. For teams building custom AI pipelines, Modal is the right shape.
- Python-native serverless, decorate a function, deploy to GPU
- Per-second GPU billing, T4 ($0.000164/s) to B200 ($0.001736/s)
- $30/mo free credits on Starter, $100/mo on Team
- Volumes, secrets, cron, web endpoints all built in
- Up to $10K in free compute for academic researchers
- Infrastructure, not a managed model, you write the inference code
- Cold starts on first invocation after idle, needs warm pool config
- Steeper learning curve than consuming a model API
- Python-only, Node / Go / Rust workloads need a different platform
- Team tier starts at $250/mo before usage
Modal is a serverless compute platform optimized for Python
workloads that need GPUs on demand. Write a function, add a
decorator declaring the hardware you need (@modal.function(gpu="H100")),
and Modal handles container orchestration, scaling, and per-
second billing. It’s the right shape when you want full control
over the inference code, not just consumption of a hosted model.
Where it fits
Modal is the right pick for custom inference, pipelines that wrap Hugging Face models with bespoke preprocessing, fine-tuning workflows that need GPU on demand, embedding generation at scale, evaluation harnesses, batch jobs. Anywhere you’d otherwise spin up your own Kubernetes cluster, Modal collapses that to a decorator.
For AI app backends specifically, Modal’s web endpoint primitive lets a function double as an HTTP handler, you get a serverless FastAPI-style backend with GPUs available on demand, no idle cost when traffic is zero, no manual scaling configuration.
Avoid Modal when you just want to call a model API (Together, Fireworks, Groq are easier), when your inference workload is in a non-Python language (Node, Go, Rust), or when the operational overhead of writing the inference code isn’t worth the control.
Pricing in practice
GPU per-second: B200 at $0.001736/s, H200 at $0.001261/s, H100 at $0.001097/s, A100 80GB at $0.000694/s, A100 40GB at $0.000583/s, L4 at $0.000222/s, T4 at $0.000164/s. CPU at $0.0000131/core/s. Memory at $0.00000222/GiB/s. Storage at $0.09/GiB/month (1 TiB free included).
Starter is $0/month with $30/month free credits, 100 containers, 10 GPU concurrency. Team is $250/month with $100/month credits, 1,000 containers, 50 GPU concurrency. Enterprise is custom with volume discounts. Academic researchers can apply for up to $10K in free compute.
How it compares
Together AI, Hosted models with no code to write. Pick when you’re consuming OSS LLMs, not building custom pipelines.
Replicate, Hosted multimodal models with Cog for custom deployment. Pick when you want hosted APIs across modalities with light custom support.
Fireworks AI, Hosted serverless LLM inference. Pick when you’re consuming OSS LLMs, not running Python pipelines.
Latest news
Modal closed a $355M Series C in May 2026 at a $4.65B valuation, one of the largest funding rounds in the serverless GPU category to date (2026-05-21). Two days earlier, Anthropic announced Claude Managed Agents on Modal Sandboxes, giving long-running agent workflows isolated per-task environments with on-demand GPU access (2026-05-19). The pairing positions Modal as a Python-native execution layer beneath frontier-model agent platforms. H200 and B200 hardware coverage and per-second billing across the GPU tier remain the operational wedge for teams running custom inference pipelines rather than consuming hosted model APIs.
Sources
- Modal Pricing, modal.com, 2026
- Modal Docs, modal.com
- Modal Examples, modal.com
- Modal GPU, modal.com
- beat · 2026-07-02
Together AI raises $800M at a reported $8.3B valuation as open-model inference demand compounds
Together AI's $800M Series C, reportedly at an $8.3B valuation, is one of the largest neocloud rounds yet. The raise reads as a bet that cheaper open-model inference keeps pulling workloads off frontier-token pricing.
- launch · 2026-07-01
Claude Science is a vertical product built from orchestration, not a new model
Anthropic launched Claude Science in beta on June 30, an AI workbench for researchers built on connectors, domain skills, and a fact-checking agent. It runs on ungated Claude models, making it a case study in productizing the agent runtime, not a new capability.
- launch · 2026-06-30
Modal launches Auto Endpoints to put transparent LLM inference behind a single CLI command
Modal's Auto Endpoints deploy production-ready LLM inference in one command, with pre-optimized GPU configurations, engine-level observability, and a serving stack you can inspect and modify.
- beat · 2026-05-30
Groq raising $650M to fund the inference neocloud pivot, Axios reports
The Axios scoop pegs Groq at up to $650M from existing investors, with Disruptive and Infinitum committing to backstop the round. Smaller than 2025's Series E and with no disclosed valuation, it reads as a Groq 2.0 raise.
- comparison · 2026-05-25
Inference platforms in 2026: how Modal, Replicate, Together, and Fireworks divide the category
After Modal's $355M Series C, the GPU inference category divides cleanly across four platforms: Modal as a Python runtime, Replicate for multimodal, Together as the OSS LLM workhorse, and Fireworks as Together's performance-focused peer.