Open source AI is not free. Here's what it actually costs.

I have been building AI products for years, and this is the misconception I run into more than any other. A lab like DeepSeek or Moonshot puts their weights online, someone reads the headline, and the conclusion lands instantly: the model is free now.

The download is free. Everything after that costs money, and in a lot of cases self-hosting is the single most expensive option on the table.

What you are actually paying for

Take DeepSeek. The full open-weight checkpoints run to roughly 671 billion parameters. That does not fit on your laptop, or on one GPU, or on the server sitting in your office cupboard. You need a rack.

In practice you want something like an eight-card H200 box to hold the whole model in memory at FP8. Eight of those cards gives you around 1,150 GB of VRAM, which is about what a model that size demands before you have served a single request.

Then the rent. An eight-GPU H100 node goes for roughly $16 to $30 an hour on demand depending on who you rent from. An eight-GPU H200 node is closer to $24 to $36. Leave one of those running for a month and you land somewhere around $12,000 to $26,000. Want a second node so that one machine dying does not take your product down with it? Double it.

Here is the part that catches teams out. That meter runs whether or not anybody uses your product. GPUs bill by the hour, not by the request. A dead Sunday costs exactly what your busiest Monday costs.

I have watched teams underestimate this and then spend the next year explaining the burn rate to their board. The model was free. The infrastructure bill was not, and nobody worked it out before the decision got made.

What we do at Phot.AI instead

We run a lot of models at Phot.AI. Text to image, image to image, text to video, image to video, and a stack of GAN models underneath all of that.

If we hosted every one of those on our own GPUs, we would be paying rent on a dozen machines that sit idle for most of the day, because no two of those models get used at the same rate. The video models spike. The older GAN models tick along quietly. Sizing hardware for the peak of each one means buying a lot of silence.

So we do not. We pay per use, and the rent becomes somebody else's problem.

It is the difference between buying a plane ticket and buying the plane. You want to get to the destination. You do not want to own the aircraft, employ the crew, and lease the hangar it sleeps in.

So what does open source actually buy you

Choice.

When a model is closed, there is one shop. Anthropic sells Claude, OpenAI sells GPT, and the price on the sticker is the price you pay.

When the weights are open, anyone with GPUs can serve that model. Now you have a market: several providers running the same thing, competing on price, speed and which region they run in. You can move between them in an afternoon. You can host it yourself if you have a real reason to.

That is the benefit. Not zero cost. Leverage.

closed weights

Claude GPT Gemini

One seller each. Anthropic sells Claude, OpenAI sells GPT. You can switch model, but you cannot switch who you buy it from.

open weights

DeepSeek Kimi Qwen Mistral Llama

Anyone with GPUs can serve these. Several hosts run the same model and compete on price, speed and region. Moving between them takes an afternoon.

Everyone rents. The smarter companies just rent cheaper.

Cheaper per token is not cheaper per job

This trap deserves its own section, because it caught a good chunk of the market in July.

On 16 July, Moonshot AI released Kimi K3, a 2.8 trillion parameter model and the largest set of open weights anyone has shipped. It went straight to the top of the Frontend Code Arena leaderboard with 1,679 points, ahead of Claude Fable 5 on 1,631 and GPT-5.6 Sol on 1,618. It took first place in six of seven categories, dropping only Gaming.

The pricing undercut Anthropic hard: $3 per million input tokens and $15 per million output, against $10 and $50 for Fable 5. Roughly a third of the price for the top spot on a coding leaderboard.

Chip stocks wobbled. Nvidia slid about 3% intraday on 17 July before recovering some of it, and the Philadelphia Semiconductor Index fell around 10% across that week, dropping into technical bear market territory.

Now the part I actually care about, which almost nobody covered.

Artificial Analysis put K3 through their Intelligence Index. It scored 57. Claude Fable 5 scored 60 and GPT-5.6 Sol scored 59. Slightly behind on capability, at a third of the token price. Still sounds like an easy win.

Except that running the same evaluation, K3 burned through 130 million output tokens. The median for models on that index is 63 million. It talks roughly twice as much to arrive at a slightly lower score.

A price per token is not a price. It is one half of a multiplication. The other half is how many tokens the model actually spends getting your work done, and a chatty model can eat the whole discount before you notice.

So: two numbers, every time. What you pay per token, and how many tokens the job takes. A model that is three times cheaper and twice as talkative is not three times cheaper.

The four levels

The setup that works for a five-person startup is wrong for a five-thousand-person company. Most people get this backwards in one of two directions. They over-engineer on day one, or they stay on a setup that quietly bleeds money for a year after they outgrew it.

Here is the map I use.

choosing your AI stack · by company stage

OpenAI

Claude

DeepSeek

NVIDIA

rent → mix → route → own

01

early startup

Rent

Frontier closed models through an API. Pay per use, ship this week, own nothing.

02

growing startup

Mix

Frontier models for the hard reasoning. Hosted open models for the cheap repetitive volume.

03

large company

Route

A router in front of several models. Every request goes to the cheapest one that can handle it.

40 to 60% off the bill

04

large and regulated

Own

Your own GPUs. Only with the volume, the compliance need, and the team to run it.

$20k+/month

cheapest to startmost control, most rent

1Most companies belong one level to the left of where they think they are.

▶ Replay

Level one, the early startup. Use the frontier closed models through an API. ChatGPT, Claude, Gemini. You pay per use, you ship quickly, and you build no infrastructure. Almost every company should start here and a surprising number should stay.

Level two, the growing startup. Now you mix. Frontier models for work that genuinely needs reasoning, hosted open models for the cheap repetitive volume. Classification, extraction, tagging, the thousand small calls that do not need your most expensive model. Most companies I talk to should be at this level and are not.

Level three, the large company. Put a router in front of several models so each request goes to the cheapest one capable of handling it. Done properly this takes 40 to 60% off the bill, which sounds like a sales pitch until you look at how much of your traffic is simple classification being answered by a frontier model at frontier prices.

Level four, large and heavily regulated. This is where hosting on your own cloud starts to make sense. Not before.

The spread between the top frontier models and the budget end of the market is enormous. Depending on which two you put side by side, it runs into the hundreds of times. That gap is the entire argument for routing, and it is why level three exists at all.

When self-hosting is genuinely right

I am not against it. I am against doing it by default, or doing it because open weights felt like a free lunch.

You want volume high enough that a permanently running GPU works out cheaper than metered access. You want a compliance or data residency requirement that actually rules out sending data to somebody else's servers, not a vague preference for keeping things in house. And you want the people to run it, because a GPU cluster is not something you set up once and forget.

If you only have the last one, you are building infrastructure for the fun of it. If you only have the compliance requirement, check whether your provider offers a private or dedicated deployment first. Most of them now do, and it is usually cheaper than doing it yourself.

Self-hosting is not the mature end state that everyone graduates to. It is a specific answer to a specific problem, and most companies do not have that problem.

Where this leaves you

The free download is the least interesting thing about open weights. What you get is the right to shop around, and the obligation to do the arithmetic properly: price per token multiplied by tokens actually spent, against the cost of a machine that bills you while it sits idle.

Everyone rents. Smart companies just rent cheaper.

Both halves of this started as shorts, if you would rather watch: why open source AI is not free and the four-level map.