Open Source LLM: What It Means, How to Run One, and What It Costs

TL;DR

An open source LLM is a large language model whose trained weights are published for anyone to download, run, fine-tune, and serve. Almost none meet the OSI Open Source AI Definition, because the training data stays private, so the accurate term is open weight. The license on the checkpoint, not the label, decides what you are allowed to ship.

Published June 18, 2026; updated September 7, 2026 with Hugging Face download data, host-fidelity measurements from Moonshot's vendor verifier, a quantization benchmark on Qwen3.8 27B, and the interconnect limit on eight-GPU self-host builds.

This page defines the category, verifies the license on every model that matters, and prices the three ways to run one. For a ranked answer to which model is best, read best open source LLM, which ranks the field by benchmark, size, and use case.

What Counts as an Open Source LLM in 2026

The Open Source Initiative published the Open Source AI Definition 1.0 in October 2024. It requires three components under OSI-approved terms: the parameters (weights, including intermediate checkpoints and optimizer states), the complete source code used to train and run the system, and "sufficiently detailed information about the data used to train the system so that a skilled person can build a substantially equivalent system." It also grants four freedoms: use for any purpose, study, modify, and share.

No model in the table below meets that bar. Every one ships weights. Most ship inference code and a technical report. None publish the training corpus or enough data information to rebuild the model. Three ship under custom licenses that fail the "use for any purpose" freedom outright: Kimi K3, GLM-5.3, and Llama 4 each carry a revenue or user threshold above which you need permission from the vendor.

So the working definition is open-weight: the checkpoint is public, the license is whatever the model card says, and the training data is private. Read "open source LLM" as that unless the page cites OSAID explicitly. The models that do pass the strict test (OLMo, Pythia) are research artifacts, not the models topping coding leaderboards.

Nathan Lambert made the same argument before OSAID existed, in an open-source LLM (March 2024), and proposed three tiers instead of a binary: openly trained models (OLMo, Pythia), permissible usage models (Llama, Mistral, Gemma), and closed models. His line was that "an open-source LLM was assumed to be one with every part of the stack accessible and without any usage restrictions." That taxonomy still holds, but the middle tier has since split again: Apache 2.0 and MIT checkpoints with no conditions on one side, and checkpoints with revenue or user thresholds on the other.

What People Actually Download

Every list of open source LLMs, including the tables on this page, ranks frontier models. The download data says most teams run something else. Hugging Face published State of Open Models: Summer 2026 on August 14, 2026, counting the whole Hub rather than a leaderboard.

Three things follow. First, the frontier open models are a rounding error in downloads and the bulk of the ecosystem is small models pulled as quantized GGUF files, which is why quantization quality is a production question and not a hobbyist one. Second, the permissive licenses won on volume: of 178 Chinese releases above 20B parameters this year, 59% carry Apache 2.0 and 22% carry MIT, and the report notes almost none carry non-commercial restrictions, so the revenue thresholds on Kimi K3 and GLM-5.3 are the exception rather than the trend. Third, on size, the report states that "in almost every month of 2026, the largest and most performant open model from a Chinese lab was larger than any model an American lab released," with the Chinese monthly ceiling running from 754B to 2.78T parameters against American releases under 130B in five of seven months.

License Table, Verified Against the Model Cards

Every row below was read from the license file or model card on Hugging Face, or the vendor's license page, on September 1, 2026. Two things changed from the 2025 landscape: Gemma moved from the custom Gemma Terms of Use to Apache 2.0 with Gemma 4, and the Chinese labs split, with Z.ai and Moonshot attaching revenue thresholds to their flagships while keeping the smaller models on plain MIT.

The practical reading. If you are building a product that embeds a model behind a feature, every row above is usable, including Kimi K3 and GLM-5.3, whose thresholds target companies reselling inference. If you are an inference provider, Kimi K3 needs a Moonshot agreement at $20M of revenue and Llama 4 is off the table at hyperscale. If you want zero conditions, the MIT and Apache 2.0 rows cover the whole capability range: DeepSeek V4 Pro at the top, GLM-5.3-Flash and DeepSeek V4 Flash in the efficient middle, Qwen 3.8 27B and Gemma 4 31B on a single GPU.

Independent Evidence: Index Scores and Measured Throughput

Vendor benchmarks on open models run high; the llm-stats tracker lists 0 of 104 SWE-bench Verified entries as independently verified. Two independent sources are worth more than any launch table. Artificial Analysis runs every model through the same harness and publishes an Intelligence Index. SemiAnalysis InferenceX runs the same serving workload on the same GPUs and publishes tokens per second per GPU.

The four highlighted rows are the models Morph serves. Two observations fall out of the table. First, the gap between the best open model and the best closed model is eight points, and the open model costs $14.00/M output against Anthropic's Opus-class rates. Second, GLM-5.3-Flash at 42 sits above the previous generation's flagship (GLM-5.2, 39) with 18B active parameters and $0.35/M output, which is why the small MIT models now do most of the volume work.

Throughput per GPU, measured by InferenceX

Index scores say what a model can do. Throughput says what it costs to serve. InferenceX measured Kimi K3 on B300 GPUs with vLLM at FP4 on the AgentX coding-agent trace: 6,083 tokens per second per GPU at 50 tokens per second per user, falling to 2,027 at 200 per user. At the 50 tok/s operating point InferenceX prices that at $0.10 per million tokens, using a $2.26 per GPU-hour hyperscaler rate, which is far below what anyone rents a B300 for on demand.

The two DeepSeek rows show the trade every self-hoster makes: one user on a B300 gets 216 tok/s and the GPU produces 216 tok/s total; 64 users on a B200 each get 40 tok/s and the GPU produces 2,560 tok/s. Cost per token is set by the second number. Interactive latency is set by the first. You do not get both from one box, which is the whole argument for sizing dedicated capacity against a target per-user speed rather than a GPU count.

How Far Behind Is Open, Really

A six-point index gap is one snapshot. The more useful question is how long open weights take to reach a capability the closed labs already shipped, and the answer depends entirely on which benchmark you measure. Jamie Dborin ran that analysis at Doubleword on June 22, 2026, measuring the lag in months rather than in points.

The conclusion is narrower than the headline: open weights caught up on coding, and coding is where most of the total benchmark improvement has landed, which makes a single composite index read like convergence everywhere. Dborin's own summary is that "open source LLMs are consistently 5 months behind close source, and that the gap might be growing." If you are picking a model for a coding agent, the open option is roughly current. If you are picking one for a task nobody optimizes for, budget for a lag.

The argument that has nothing to do with benchmarks

The 250-comment thread on that analysis spent most of its length on durability rather than scores. The case for open weights, in one commenter's words, is that "whatever capabilities they reach, those stay forever," while hosted models get retired on the provider's schedule. The counterargument in the same thread is that a frozen checkpoint ages: a model that has never seen a library change is worth less every quarter, and continued training or distillation is then your cost, not the lab's. Both are true, and together they are the actual reason to care about the license. A checkpoint you may legally keep, run, and fine-tune is insurance against a deprecation notice. A checkpoint you may only rent is not.

What the local-model community argues about

The largest study of that community is an academic one. Woohyeuk Lee, James Howison, Min Kyung Lee, and Hanlin Li analyzed 52,518 posts and 731,615 comments from r/LocalLLaMA in Open AI in the Wild (June 2026). Their finding is the split this page opened with: purists who want full replicability against pragmatists who accept partial releases that offer, in the paper's phrase, tangible utility, with both camps conceding that retraining a foundation model from scratch is infeasible anyway. The adoption drivers they identify are privacy through local inference, low marginal cost after a one-time hardware purchase, and control over a vendor's roadmap. The barriers are a sharp learning curve, fragmented tooling, and resource asymmetry on large workloads. That is the honest case for calling an API instead: not that the model is better, but that the second list is somebody else's problem.

Same Weights, Different Model: Hosts and Quantization

An open source LLM is not one artifact. It is a checkpoint plus whoever serves it, at whatever precision they chose. Two teams calling "Kimi K3" through two providers are not necessarily running the same model, and the difference shows up in tool calls before it shows up in prose.

Host fidelity is measurable, and hosts differ

Moonshot publishes K2-Vendor-Verifier, which sends an identical request set to every vendor serving its weights and compares two things against the official API: whether a response triggers a tool call at all, and whether the emitted JSON satisfies the schema. On the K2 0905 run at temperature 0.6 the official API produced schema-valid tool calls on all 1,274 successful calls, several third-party hosts matched it, and the weakest deployments in the same table scored 76% and 73.13%. On the K2-Thinking run at temperature 1.0 the official API and hosts including Fireworks scored 100%, and the lowest scored 83.05%. No harness exists yet for K3.

The usual cause is undisclosed quantization. OpenRouter now ships an :exacto variant that sorts providers by quality rather than price, described as "a virtual model variant that explicitly applies quality-first provider sorting" and ranked on tool-calling success measured from real traffic. Practitioners reached the same conclusion by hand: one commenter on the OpenRouter Series B thread wrote that providers quantize silently, "so you think for example you're using Kimi k2.6 but behind the scenes, it's the 4b or 8b quantized versions," and now routes real work to the model author's own endpoint. Another, on the Intelligence vs Cost thread, put the economics plainly: filter by quantization and most providers converge on the same price, and anything materially cheaper implies a trick on the provider's side. Before you standardize on a host, ask which precision it serves and whether the checkpoint revision is pinned, then run your own tool-call suite against it. On Morph, a dedicated deployment pins the revision for you.

Where quantization actually breaks

Quantization is not a slider from good to bad. It is flat and then it is a cliff. Quesma benchmarked Qwen3.8 27B across five precisions on August 26, 2026, spending about $3,000 of rented GPU time, $2,308 of it on Terminal-Bench 2.1 alone.

The author's conclusion is that 4-bit Q4_K_M matches the full model while fitting a consumer 24 GB GPU, and that "compression eventually hits a cliff. At 1 bit, the model performs around random chance on GPQA Diamond." Two practical consequences. If you self-host a small model, 4-bit is free capability and 2-bit costs you agentic tasks first, because multi-step tool use degrades before single-turn knowledge does. If you buy an open model from a host, the precision it serves is the model you are actually buying, which is why the fidelity question above matters more than a price difference of a few cents per million tokens.

Run Open Source LLMs Through One API

Morph serves the four highlighted models on one OpenAI-compatible endpoint, https://api.morphllm.com/v1, each at the full 1M-token context the vendor publishes. Rates below are the live list prices per million tokens, read from the same file that bills you.

from openai import OpenAI

client = OpenAI(
    base_url="https://api.morphllm.com/v1",
    api_key="YOUR_MORPH_API_KEY",
)

resp = client.chat.completions.create(
    model="morph-glm53flash",  # or morph-kimik3, morph-glm53-744b, morph-dsv4flash
    messages=[
        {"role": "system", "content": "You are a senior engineer reviewing a pull request."},
        {"role": "user", "content": "Find the race condition in this handler and propose a fix."},
    ],
)
print(resp.choices[0].message.content)
print(resp.usage.prompt_tokens_details.cached_tokens)  # tokens billed at the cached rate

Prefix caching is on by default

Every request that shares a prefix with earlier traffic (system prompt, tool definitions, conversation history) skips prefill on those tokens and bills at the cached rate: 84% off on GLM-5.3, 80% on GLM-5.3-Flash, 88% on Kimi K3, 75% on DeepSeek V4 Flash. No configuration and no cache-write surcharge. Matching is exact-prefix on roughly 1K-token blocks, so put stable content first and keep it byte-identical between turns. A prompt_cache_key per conversation pins every turn to the worker that already holds its prefix, and cache_ttl (5m to 24h, sliding) controls retention. The mechanism is explained in prompt caching; the API details are in the caching docs.

Standby: 50% off when nobody is waiting

Send service_tier: "standby" and every leg bills at half the list rate on all four models. Standby runs on spare capacity: it queues behind default traffic and sheds a fast 429 with Retry-After: 15 when a region is busy. Shed requests bill nothing. Cached standby input stacks both discounts.

Batch API: the same rate with a 24-hour window

Upload a JSONL file of chat-completion requests, create a batch, and collect the results within 24 hours. It is the OpenAI Batch API, so the official SDKs work unchanged against api.morphllm.com/v1. Every completed line bills at the standby rate above. Eval runs, dataset generation, nightly summarization, and backfills belong here; anything a person is waiting on belongs on the synchronous endpoint. See the Batch API docs and standby docs.

Dedicated: your own B200 or B300 capacity, by the minute

When you need isolated capacity, a pinned checkpoint revision, or a speculator trained on your own traffic, dedicated inference reserves single-tenant B200 (list $9.98 per GPU-hour, billed by the minute) or B300 capacity behind the same API. The calculator compares monthly dedicated cost against per-token spend for your traffic shape, and the benchmark page sizes GPU count from a target per-user speed.

API vs Self-Host vs Dedicated: The Crossover

The self-host argument is that GPUs are a fixed cost and tokens are free once you own them. That is true, and it hides the real variable: utilization. A rented GPU costs the same at 3 a.m. as at peak. The API bills only tokens served. So the question is not "which is cheaper per token at full load" (self-host, always) but "how many tokens per second must I sustain, every second of the month, before the fixed cost pays for itself."

The crossover table prices a minimal self-hosted deployment of each Morph model on RunPod B200s at $6.79 per GPU-hour,730 hours a month, and asks how many output tokens that same money buys on the API. Weight sizes assume one byte per parameter (FP8), except Kimi K3, whose checkpoint ships in MXFP4. GPU counts are the smallest tensor-parallel layout that holds the weights with room for KV cache; Kimi K3 spans two nodes. Input tokens are ignored, which favors self-hosting, so treat the break-even as a floor.

Read the last column against the InferenceX rows above. Two B200s serving DeepSeek V4 Flash need to produce 9,439 output tokens per second around the clock to match the API rate. InferenceX measured the larger V4 Pro at 2,560 tok/s per B200 at 64 concurrent users, so a two-GPU Flash deployment would have to run near its throughput ceiling every hour of the month, with no headroom for bursts, to break even. For the Flash-class models the API is cheaper at any realistic utilization.

Kimi K3 is the opposite case. Its API rate tracks Moonshot's list price rather than compute cost, so 16 B200s break even at 2,156 tok/s against a measured 6,083 tok/s per B300 on the AgentX trace. If you have that traffic and the team to run a 2.8T-parameter, two-node, Kimi-Delta-Attention deployment, self-hosting K3 pays. Most teams have one of the two.

# Serve an MIT-licensed open model on your own GPUs
vllm serve zai-org/GLM-5.3-Flash \
  --tensor-parallel-size 2 \
  --max-model-len 262144

# Your client code is unchanged from the Morph example above:
# base_url="http://your-host:8000/v1", model="zai-org/GLM-5.3-Flash"

Model Family Directory

Each page below covers one model or family end to end: architecture, vendor and independent benchmarks, license, how to self-host, and how to call it. The four Morph serves are highlighted.

Comparisons: GLM-5.2 vs Kimi K3, Kimi K3 vs Claude, GLM-5.3 vs Claude, GLM-5.2 vs Claude Opus. This page covers what an open source LLM is, what each license permits, and what running one costs. For the ranking of which model wins, read best open source LLM and best open source coding model 2026. Beyond LLMs, open source AI models covers vision-language, embedding, speech, and image models by category.

FAQ

What is an open source LLM?

A large language model whose weights are published for download, so you can run, inspect, and fine-tune it on your own hardware. Strictly, the OSI Open Source AI Definition 1.0 also requires training code and data information, which no leading model provides; in practice the term means open-weight.

What is the difference between open source and open weight?

OSAID 1.0 requires parameters, complete training and inference code, and enough data information to rebuild the system, all under OSI-approved terms. Open-weight models release the parameters under a license of their own choosing and keep the data private. Both can be self-hosted and fine-tuned.

Which open source LLMs can I use commercially?

All of the leading ones, with conditions that differ by vendor. MIT: GLM-5.3-Flash, DeepSeek V4 Pro, DeepSeek V4 Flash. Apache 2.0: Qwen 3.8, gpt-oss, Gemma 4. Kimi K3 needs a Moonshot agreement above $20M of model-as-a-service revenue. GLM-5.3 needs a Z.ai security review above $10B. Llama 4 needs a Meta license above 700M monthly active users. The license table above has the exact clauses.

What is the best open source LLM in 2026?

GLM-5.3 scores 45 and Kimi K3 44 on Artificial Analysis Intelligence Index v4.3, the highest of any model with public weights, checked September 7, 2026. For a coding-specific ranking see best open source coding model 2026; for the field across use cases see best open source LLM.

How much does it cost to run an open source LLM through an API?

On Morph, per million tokens: Kimi K3 $2.50 in / $14.00 out, GLM-5.3 $1.19 / $3.74, GLM-5.3-Flash $0.10 / $0.35, DeepSeek V4 Flash $0.141953 / $0.399625. Cached input and standby each cut those further; the tables above list every leg.

When is self-hosting cheaper than the API?

When your sustained throughput clears the break-even row for that model. For DeepSeek V4 Flash on two rented B200s that is about 9,439 output tokens per second every second of the month; for Kimi K3 on 16 B200s about 2,156. Below that, the API is cheaper. Between the two, dedicated inference gives reserved capacity without the ops.

How do I run an open source LLM locally?

Ollama or llama.cpp pull a quantized checkpoint with one command and serve it at localhost:11434 on an OpenAI-compatible API. The best Ollama models page ranks the current library by VRAM, context, and coding quality.

How do I deploy an open source LLM in production?

Pick the precision first, then the layout. Quesma's benchmark found 4-bit Q4_K_M matches BF16 on Terminal-Bench 2.1 for Qwen3.8 27B at 17 GB instead of 55 GB, while 2-bit loses agentic tasks first. Then run vLLM or SGLang with tensor parallelism sized to hold weights plus KV cache, add a second replica for failover, and pin the checkpoint revision. Budget engineer time for revision churn.

What is the smallest open source LLM I can deploy?

Smaller than the leaderboards suggest. Models under 1B parameters take 83% of all-time Hugging Face downloads and models above 100B take 1%. At the useful end, Qwen3.8 27B in Q4_K_M is 17 GB and fits a consumer 24 GB GPU with no measured loss against BF16. See best Ollama models for the local library by VRAM.

What is the largest open source LLM?

Kimi K3, at 2.8T total parameters with 104B active. Hugging Face reports the Chinese monthly ceiling ran from 754B to 2.78T parameters through 2026 while American open releases stayed under 130B in five of seven months.

Do all providers serve the same open source LLM identically?

No. On Moonshot's K2-Vendor-Verifier the official API produced schema-valid tool calls on all 1,274 successful calls of the K2 0905 run, while the weakest deployments in the same table scored 76% and 73.13%. Undisclosed quantization is the usual cause. Ask which precision a host serves and pin the checkpoint revision.

What are the best open source LLM directories?

Hugging Face Hub is the registry, at 2.96 million public model repositories. Artificial Analysis maintains a scored open-weights leaderboard with price and speed. Curated GitHub lists go stale fast; eugeneyan/open-llms, still ranking on page one for this query, stops at early 2025.

Do open source LLMs support 1M-token context?

Kimi K3, GLM-5.3, GLM-5.3-Flash, DeepSeek V4 Pro, and DeepSeek V4 Flash all publish 1M windows, and Morph serves each at the full 1M. Qwen 3.8 27B is 262,144 tokens natively, Gemma 4 31B is 256K. See LLM context windows for the full comparison.

Related Resources

• Open Source AI Models by Category: LLMs, Vision, Embeddings, Speech, Image
• Best Open Source LLMs (2026), Ranked
• Best Open Source Coding Model (2026)
• LLM API Providers Compared by Price
• LLM Inference: Cost, Speed, Caching, and GPU Capacity
• Dedicated Inference and the cost calculator
• Fireworks AI Alternative, Together AI Alternative, OpenRouter Alternatives
• LLM Router and LLM Gateway
• LLM Context Windows Compared