Open-weight models

People say open models a lot.

Sometimes they mean open source. Sometimes they mean open weights. Sometimes they mean "this can actually run outside someone else's API."

The useful split is simpler.

Proprietary models:
Call someone else's model through an API.
 
Open-weights models:
Download the weights and run the model directly.

This choice quietly changes almost everything:

Where does the model run?
Who controls the weights?
Where does the data go?
What does inference cost?
Can you fine-tune it?
Can it run offline?

Start with the hosted/proprietary side.

Closed, or proprietary, models

Closed models, also called proprietary models, are models that cannot be downloaded and run directly.

The company owns the weights. You access the model through an API, hosted app, or subscription product.

Common examples:

OpenAI
Anthropic
Google

When someone uses ChatGPT, Claude, or Gemini, the model is not running on their laptop. The prompt goes to the company's servers, runs there, then comes back as a response.

The nice part: convenience, strong performance, and usually the newest frontier capabilities.

The trade-off: control. The model belongs to the provider, the infrastructure belongs to the provider, and data usually leaves the application unless there are special enterprise controls.

Open models

Open models are the other side.

With open-weights models, the weights are public. The model can run on a local machine, a cloud server, or a GPU cluster.

Most of them live on Hugging Face. Common model families include:

Llama
Qwen
Mistral
DeepSeek
Gemma
Phi
Kimi

Tools that show up a lot:

Ollama
llama.cpp
vLLM
LM Studio
Text Generation Inference

Small but important detail: open model does not always mean fully open source.

Often it means open weights. The license still matters. Some models are permissive, some have commercial restrictions, and some are research-only.

Open models are catching up

Open models used to feel like the budget option.

Not anymore.

The best ones are now good enough for real coding, reasoning, and agentic workflows. They don't always beat proprietary frontier models, but the gap is not what it used to be.

Obvious thought:

If open models exist, why does everyone not just use them?

Because the model is only one piece of the stack.

Why are closed models usually better?

A top proprietary model is not just a model file sitting somewhere.

It comes with:

massive datasets
thousands of GPUs
months of training
world-class researchers
continuous experimentation
evaluation infrastructure
inference infrastructure

Training a frontier model can cost tens or hundreds of millions of dollars. Then it has to be served for millions of users every day.

That's why proprietary models are usually ahead. The company controls the full loop:

training
post-training
safety tuning
evaluation
serving
product feedback

That loop compounds fast.

What are parameters?

Model names often include sizes like:

7B
14B
32B
70B
400B

The B means billion parameters.

A parameter is basically a learned number inside the model.

During training, the model adjusts these numbers until it captures patterns in language, code, math, images, and other data.

Simple version:

Parameters are where a model stores a lot of what it learned.

More parameters usually mean more capacity.

But bigger is not automatically better. A smaller model trained well can beat a larger messy one.

Tiny toy example:

Example
 
2 input neurons
3 hidden neurons
1 output neuron
 
Weights:
(2 x 3) + (3 x 1) = 9
 
Biases:
3 + 1 = 4
 
Total parameters = 9 + 4 = 13

Same idea at LLM scale, just with billions of learned numbers instead of 13.

Distillation

Distillation: make a smaller model learn from a bigger one.

The senior-engineer-to-junior-engineer analogy fits pretty well.

The junior won't know everything the senior knows, but they pick up useful shortcuts. Models work similarly: a large teacher model generates examples, then a smaller student model learns to imitate them.

Result: a smaller model that keeps a decent amount of the teacher's ability.

This is why these names exist:

Those are DeepSeek-R1 distilled into Qwen-based models.

Same idea with Llama:

Those are DeepSeek-R1 distilled into Llama-based models.

Why this matters:

fewer parameters
faster inference
lower cost
smaller GPU requirements

They usually lose some quality, but for many tasks the trade-off is worth it.

Quantization

Quantization is a different trick.

Distillation makes the model smaller.

Quantization makes the model lighter.

A model is billions of numbers. Those numbers take memory. Quantization stores them with fewer bits.

Instead of storing each parameter with 16 or 32 bits, you might store it with 8 bits or 4 bits.

Same model shape. Same parameter count. Less memory.

This helps because the model:

uses less VRAM
loads faster
can run faster
can fit on cheaper hardware

Trade-off: less precision.

That can reduce quality a bit, but modern quantization is honestly pretty good.

You'll see labels like:

The lower the number, the less memory it uses.

Usually, lower also means a small quality hit.

Distillation vs quantization

Useful shortcut:

Distillation changes the model.

Quantization changes how the model is stored.

Distillation creates a smaller student model.

Quantization stores an existing model more efficiently.

Can you use both?

Yes. This is common.

A company might first distill a 70B model into an 8B model, then quantize that 8B model to Q4.

Now there is something much cheaper to run, while still keeping a decent chunk of the original capability.

This is why Hugging Face model names can look chaotic:

the original model
a distilled version
a Q4 quantized version
a Q5 quantized version
a Q8 quantized version
a GGUF version for local inference tools

For example:

DeepSeek-R1-Distill-Llama-8B-GGUF

That name tells you a lot:

it is related to DeepSeek-R1
it is distilled into a Llama-based model
it has 8B parameters
it is packaged in GGUF format

Once the pieces are visible, the names stop looking cursed and start reading like labels.

What happens after training?

Training is not the end of the bill.

Every time a model is asked something, inference happens:

the prompt is tokenized
the tokens are loaded into GPU memory
the model processes them through transformer layers
new tokens are generated one at a time

This happens for every user, every prompt, every day.

Training is expensive once.

Inference is expensive forever.

Why use open models?

Open models are not just "the cheaper option."

Sometimes they are the better option.

For example:

private company data
healthcare
government
banking
defense
on-premise deployments
air-gapped environments

If the model runs on private infrastructure, the data does not have to leave private servers.

That means more control over:

data privacy
deployment
latency
compliance
model customization
long-term cost

Deployment

Running an open model does not mean everything has to run on a laptop.

Deployment options:

locally
on AWS, Azure, or GCP
inside an internal data center

Cloud still means trusting someone else's infrastructure, but the model and app are more directly controlled by the team running them.

An internal data center gives the most control, but it also brings the annoying parts: GPUs, scaling, monitoring, updates, failures, all of it.

Cost

Cost is where open models get interesting for real.

Proprietary APIs usually charge per token.

LLM API Providers Leaderboard

At high volume, token bills add up fast.

Open models don't have a per-token API fee. The cost moves to hardware.

This shifts the cost from:

paying another company per token

to:

operating internal inference infrastructure

Whether this is cheaper depends on:

traffic volume
latency requirements
model size
GPU costs
engineering resources
uptime requirements

So open models can be cheaper at scale. Not free. The bill just moves from API tokens to infrastructure.

Fine-tuning

Open models are also nicer when customization matters.

Fine-tuning data could be:

legal docs
medical knowledge
company docs
support conversations
internal coding standards

That lets the model specialize without training a new model from zero.

Fine-tuning is its own topic, so this stops at the basic idea.

When should you choose open models?

Open models make sense when:

privacy is important
complete control matters
on-prem deployment matters
fine-tuning matters
request volume is high
offline deployment matters
the team can manage infrastructure

Proprietary models make sense when:

the best reasoning performance matters most
managing infrastructure is not worth it
speed matters
usage is still small
latest frontier capabilities matter
a simple API is enough

Short version:

Proprietary models:
better convenience, usually stronger frontier performance
 
Open-weights models:
more control, more privacy, more deployment flexibility

Neither option is always better.

The choice usually comes down to the data, the latency/cost shape, and how much infrastructure pain is worth owning.