Open models vs closed models

Open models! Open models! Open models!

One of the first things you hear when you get into AI is people talking about open models and closed models.

You will also hear people call closed models proprietary models. Same idea. Different wording.

Before going deeper into AI systems, it helps to understand this distinction because it affects almost every architecture decision you make:

Where does the model run?
Who controls the weights?
Where does the data go?
How much does inference cost?
Can you fine-tune it?
Can you deploy it offline?

Ok, let's start with the easy one: closed, or proprietary, models.

Closed, or proprietary, models

Closed, or proprietary, models are models you cannot download and run yourself.

The model weights are owned by the company that trained the model. You access the model through an API, subscription product, or hosted service.

Common examples include products and model families from:

OpenAI
Anthropic
Google

For example, when you use ChatGPT, Claude, or Gemini, you are not running the model on your laptop.

You send a prompt over the internet. The company runs the model on its own infrastructure, then sends the response back.

You never directly own or run the model yourself.

This is not automatically bad. In fact, closed models are often among the most capable models available.

The companies behind them spend enormous amounts of money on research, data, training infrastructure, evaluation, safety work, and inference infrastructure.

The trade-off is that you get convenience and frontier performance, but you give up some control.

Open models

Open models are different.

With open models, the company or research group releases the model weights publicly, so people can download and run the model themselves.

Most open models are hosted on Hugging Face, which has become the largest public hub for AI models.

Some common open model families you will hear about are:

Llama
Qwen
Mistral
DeepSeek
Gemma
Phi
Kimi

Instead of calling a hosted API, you can download the model and run it with tools like:

Ollama
llama.cpp
vLLM
LM Studio
Text Generation Inference

Now the model can run on your own machine, your own cloud server, or your own data center.

Small note: open model often means open weights, not always fully open source.

The license still matters.

Some models are permissive. Some have commercial restrictions. Some allow research use but limit production use. So before building a product on top of an open model, always check the license.

Open models are getting very good

Today, some open models perform surprisingly close to proprietary models on coding, reasoning, and agentic tasks.

That does not mean every open model is better than every closed model.

It means the gap is smaller than it used to be, and for many use cases the best open models are good enough to be practical.

Now you might wonder:

If open models exist, why does everyone not just use them?

Good question.

The answer is cost, capability, and convenience.

Why are closed models usually better?

Building a state-of-the-art language model is incredibly expensive.

A capable model depends on much more than the neural network architecture.

You need:

massive datasets
thousands of GPUs
months of training
world-class researchers
continuous experimentation
evaluation infrastructure
inference infrastructure

Training one frontier model can cost tens or even hundreds of millions of dollars.

Then, after training, the company still has to serve the model for millions of users.

That is why companies charge for access. They need to recover the cost of building, improving, and operating the model.

Closed models also tend to improve quickly because the company controls the full stack:

training
post-training
safety tuning
evaluation
serving
product feedback

That feedback loop is powerful.

What are parameters?

You will constantly hear models described like this:

7B
14B
32B
70B
400B

The B means billion parameters.

A parameter is a learned number inside the neural network.

During training, the model adjusts billions of these numbers until it learns patterns about language, reasoning, coding, math, images, and much more.

A simple way to think about parameters:

Parameters are part of how the model stores what it learned during training.

More parameters usually give the model more capacity to learn complex relationships.

But more parameters do not automatically mean a better model.

A smaller model trained on better data can outperform a much larger model trained poorly.

Here is a tiny example:

Example
 
2 input neurons
3 hidden neurons
1 output neuron
 
Weights:
(2 x 3) + (3 x 1) = 9
 
Biases:
3 + 1 = 4
 
Total parameters = 9 + 4 = 13

Real language models are the same idea, just scaled up dramatically.

Instead of 13 parameters, they have billions.

Distillation

Training a large language model from scratch is expensive.

It can take months of training, thousands of GPUs, massive datasets, and a lot of research.

So instead of training another huge model from scratch, researchers often create a smaller model that learns from an already capable one.

This is called distillation.

Think of it like a senior engineer teaching a junior engineer.

The junior engineer will not know everything the senior knows. But after learning from them, they will become much more capable than if they had started from zero.

Models work in a similar way.

A large model, called the teacher, generates many examples and responses.

A smaller model, called the student, is then trained to imitate those responses.

The result is a smaller model that keeps a surprising amount of the teacher model's capability.

That is why you see names like:

Those are DeepSeek-R1 distilled into Qwen-based models.

You may also see:

Those are DeepSeek-R1 distilled into Llama-based models.

The biggest advantages are:

fewer parameters
faster inference
lower cost
smaller GPU requirements

Of course, distilled models are usually not as capable as the original teacher model.

But for many tasks, the difference can be surprisingly small.

Quantization

People often confuse distillation with quantization, but they solve different problems.

Distillation makes the model smaller.

Quantization makes the model lighter.

What does that mean?

Remember that a model is made of billions of parameters.

Each parameter has to be stored somewhere in memory.

By default, those numbers are stored with relatively high precision.

Quantization stores those same numbers using fewer bits.

For example, instead of storing every parameter with 16 or 32 bits, you might store them with 8 bits or even 4 bits.

The model can still have the same number of parameters.

You are just storing those parameters more efficiently.

Because of that, the model:

uses less VRAM
loads faster
can run faster
can fit on cheaper hardware

The trade-off is that you lose some precision.

That can slightly reduce quality, although modern quantization methods are surprisingly good.

You will often see names like:

The lower the number, the less memory it uses.

Usually, that also means a small drop in quality.

Distillation vs quantization

A simple way to remember the difference:

Distillation changes the model.

Quantization changes how the model is stored.

Distillation creates a smaller student model.

Quantization takes a model and stores its parameters with less precision.

Different techniques. Different goals.

Can you use both?

Absolutely.

In fact, this is very common.

A company might first distill a 70B model into an 8B model, then quantize that 8B model to Q4.

Now you have a model that is much smaller, uses much less memory, and is cheaper to run, while still keeping a large part of the original model's capability.

That is why, when you browse Hugging Face, you often find many versions of the same model:

the original model
a distilled version
a Q4 quantized version
a Q5 quantized version
a Q8 quantized version
a GGUF version for local inference tools

For example:

DeepSeek-R1-Distill-Llama-8B-GGUF

That name tells you a lot:

it is related to DeepSeek-R1
it is distilled into a Llama-based model
it has 8B parameters
it is packaged in GGUF format

Model names look scary at first, but once you know the pieces, they start to read like labels.

What happens after training?

Even after a model has been trained, someone still has to run it.

That is called inference.

Every time you ask a question:

your prompt is tokenized
the tokens are loaded into GPU memory
the model processes them through transformer layers
new tokens are generated one at a time

This happens for every user, every prompt, every day.

Inference is one of the biggest ongoing costs for AI companies.

Training is expensive once.

Inference is expensive forever.

Why use open models?

Open models are not just about saving money.

Sometimes they are the better architectural choice.

For example:

private company data
healthcare
government
banking
defense
on-premise deployments
air-gapped environments

Since the model can run on your own infrastructure, your data does not have to leave your servers.

That is one of the biggest reasons enterprises are investing in open models.

It gives them more control over:

data privacy
deployment
latency
compliance
model customization
long-term cost

Deployment

Running an open model does not necessarily mean running it on your laptop.

Many companies deploy open models on cloud providers like AWS, Azure, or GCP.

In that case, you are still trusting another company with the infrastructure, but you maintain more control over the model and application.

Some organizations go further and run GPU clusters inside their own data centers.

That gives maximum privacy and control, but it also means taking responsibility for:

GPUs
networking
scaling
monitoring
updates
maintenance
hardware failures

There is always a trade-off between convenience and control.

Closed model APIs give you convenience.

Self-hosted open models give you control.

Cost

One of the biggest reasons companies evaluate open models is cost.

Closed models usually charge per token.

LLM API Providers Leaderboard

For companies making millions of requests every day, these costs can become large.

Open models do not have a per-token API fee.

Instead, you pay for the hardware needed to run them.

This shifts the cost from:

paying another company per token

to:

operating your own inference infrastructure

Whether this is cheaper depends on:

traffic volume
latency requirements
model size
GPU costs
engineering resources
uptime requirements

Open models can be cheaper at scale, but they are not automatically free.

You still pay for the machine.

Fine-tuning

Another advantage of open models is customization.

Instead of using a general-purpose model exactly as it was released, you can fine-tune it on your own data.

For example:

legal documents
medical knowledge
company documentation
customer support conversations
internal coding standards

This allows the model to specialize in a domain without training an entirely new model from scratch.

I will not go deeper into fine-tuning here.

That deserves its own blog.

When should you choose open models?

Open models are a strong choice when:

privacy is important
you need complete control
you are deploying on-premise
you want to fine-tune the model
you have large request volume
you need offline deployment
you can manage infrastructure

Closed models are usually a better choice when:

you want the best reasoning performance available
you do not want to manage infrastructure
you are building quickly
your usage is relatively small
you need the latest frontier capabilities
you want a simple API

The short version:

Closed models:
better convenience, usually stronger frontier performance
 
Open models:
more control, more privacy, more deployment flexibility

Neither option is always better.

The right choice depends on what you are building.