The Python AI Backend
A Python-first stack for teams whose product is the model, not the web app. Async API, a database that branches per experiment, serverless GPUs, and open-weight models you host yourself for control and cost.
- 01FABackend FrameworksAPI framework
FastAPI
Async Python framework with typed request models and auto-generated OpenAPI docs. The default for serving models and ML endpoints in production.
- 02NEDatabasesDatabase
Neon
Serverless Postgres with branching, so every experiment or PR gets an isolated database copy. pgvector is built in for embeddings.
- 03MOInferenceModel serving
Modal
Serverless GPUs that scale to zero between requests. Run open models or custom jobs without standing up and babysitting a cluster.
- 04LLAI ModelsOpen model
Llama
Open-weight model you host yourself for data control and per-token cost that beats hosted APIs once volume is high.
- 05DEAI ModelsOpen model
DeepSeek
Strong open-weight reasoning at low cost. A second option behind the same serving layer so you can route by task and price.
- 06FLHostingApp hosting
Fly.io
Runs the FastAPI app close to users with simple scaling, while Modal handles the GPU-bound calls as a separate concern.
Why this combo
This stack serves the audience the JavaScript SaaS path ignores: engineers whose product is the model itself. FastAPI is where serious Python API work lives, Neon gives a Postgres that branches per experiment so you are not sharing one database across every test, and Modal turns GPU serving into a serverless concern instead of a cluster you operate. Hosting open-weight models behind that layer keeps data under your control and makes per-token cost a function of your own infrastructure rather than a vendor price list. Fly runs the app tier; Modal runs the expensive part. The split keeps the cheap, always-on web layer separate from the costly, bursty GPU layer.
This is the stack for people who reach for Python before TypeScript and whose hard problem is the model, not the front end. It optimizes for experiment velocity and for keeping inference cost and data under your own roof.
What this costs to start
The web and database tiers start near zero: FastAPI is free, and Neon and Fly both have usable free tiers for a pre-launch project. The real cost is GPU time, and the design is built around controlling it. Modal scales to zero, so you pay for inference only while it runs, and hosting open-weight models means the per-token math is your hardware utilization rather than a fixed API price. At low volume a hosted API is often cheaper; this stack wins specifically when volume is high enough that owning inference beats renting it.
What to swap when you scale
- Inference throughput: add a dedicated-inference provider for your highest-volume route while keeping Modal for bursty or experimental jobs.
- Model mix: the two open models behind one serving layer let you route by task, sending cheap requests to the smaller model and reserving the larger one for hard reasoning.
- Database: Neon’s branching is great for development; production read load can move to its autoscaling tier or a dedicated Postgres without an ORM rewrite.
Where it is not the right call
If your product is a standard web app that happens to call an AI API occasionally, this is the wrong shape: a hosted model behind a TypeScript app is simpler and cheaper at that scale. Reach for this when inference is your core cost center and you have the appetite to operate model serving yourself.