Open Source LLM Models to Run Locally

The practical shortlist of open source LLM models to run locally in late 2026 is Qwen3 / Qwen3.8, Gemma 4, gpt-oss, and DeepSeek-R1 distill tags, chosen by how much memory you have. Install Ollama, pick a tag that fits your machine, and run it with one command. File sizes below come from the Ollama library pages; licences come from the official model cards.

Most people searching this phrase want downloadable weights they can run offline, not a lecture on licences. Still, "open source" and "open weight" are not the same thing. Weights you can download are usually open weight; only a few releases (for example Ai2's OLMo) also publish training data and training code under OSI style terms. Our open weight vs open source guide covers that spectrum. Here we focus on models you can actually pull and chat with today.

What's new (October 22, 2026): Ornith-1.5-122B, a 122B mixture of experts model that extends Ornith-1.0 (itself built on Qwen3.5 and Gemma4), went up on Hugging Face under the MIT license in mid October. The Ollama library lists the ornith-1.5:122b tag at 73 to 77 GB with a 256K context window, next to smaller 9b (6.6 GB) and 35b (23 GB) tags. The model card says the full model serves on a single 80 GB GPU, so treat 122b as a workstation or server model; the 9b and 35b tags are the ones that fit ordinary desktops. Run it with ollama run ornith-1.5:122b.

What's new (October 24, 2026): MiniMax published the weights of MiniMax-M3.1-Flash on Hugging Face this week: a multimodal mixture of experts model with about 428B total and 23B active parameters, text, image and video input, and up to 1M tokens of context. The card points to SGLang and vLLM for serving, so this is a server model, not a desktop one. Its MiniMax Community License is a custom licence that requires a "Built with MiniMax M3.1-Flash" notice for commercial use and written permission above $20 million in yearly revenue, so read it before you ship anything.

Pick by memory first

Memory, not a leaderboard score, decides whether a model feels usable. Weights need room; context needs more. For the general method, see how much VRAM you need for an LLM. As a starting point, match the Ollama download size to your GPU VRAM or Mac unified memory, then leave a few gigabytes free for the KV cache.

Those sizes are the published Ollama builds: qwen3, qwen3.8, gemma3, and gpt-oss. Raising context length adds memory on top; Ollama's own context length docs default to only 4K tokens on GPUs under 24 GiB of VRAM.

Best all rounders: Qwen3 and Qwen3.8

Qwen is the default recommendation for most local users who want Apache 2.0 weights.

• Qwen3-8B is a dense 8.2B model with thinking and nonthinking modes, multilingual support, and an Apache 2.0 licence on its model card. Ollama's qwen3:8b tag is a 5.2 GB download.
• Qwen3.8-27B is the denser, vision capable follow up: 27B parameters, native 262,144 token context (extendable with YaRN), thinking on by default, Apache 2.0, per the Qwen3.8-27B card. Ollama's default qwen3.8 / qwen3.8:27b tag is an 18 GB Q4_K_M build with image input and a 256K context window.

ollama run qwen3:8b
ollama run qwen3.8

For VRAM tiers, quant choices and thinking controls on the 27B, use our dedicated run Qwen 3.8 locally guide. Qwen is strong when you want one model for coding, chat and multilingual work without a custom licence. For a dedicated coding model, see how to run Recursive Flint locally.

Pros: permissive Apache 2.0 licence; excellent Ollama and Hugging Face support; 27B adds vision. Cons: thinking mode can be slow if you leave it at full effort; the 27B wants roughly 16 to 19 GB of total memory at 4 bit.

Best multimodal laptop pick: Gemma 3 and Gemma 4

Google DeepMind's Gemma line is the other high quality single GPU family.

• Gemma 3 ships 270M through 27B sizes. Ollama lists gemma3:4b at 3.4 GB, gemma3:12b at 8.2 GB (text + image), and gemma3:27b at 17 GB, with a 128K context window on the multimodal tags (gemma3 library page). Older Gemma 3 weights use the Gemma Terms of Use, not a plain Apache grant: read that page before commercial redistribution.
• Gemma 4 moves to Apache 2.0 on the Hugging Face cards (for example google/gemma-4-12B-it). Ollama's gemma4 page lists tags such as 12b (about 8 GB) and 26b / 31b (about 19 to 20 GB), aimed at laptop to server local runs with multimodal inputs.

ollama run gemma3:12b
ollama run gemma4:12b

Pick Gemma when you care about image (and, on Gemma 4 smaller tags, audio/video) understanding on one machine. For embedding only Google models, see EmbeddingGemma 2 locally. For local image generation rather than image understanding, see FLUX 3 Klein in ComfyUI. For the smallest machines, our SmolLM vs Gemma guide compares the two tiny model families.

Pros: strong small and mid sizes; Gemma 4 is Apache 2.0 on the published cards. Cons: Gemma 3 still carries custom terms; always check the exact card for the tag you pull.

Best Apache reasoning models: gpt-oss

gpt-oss is OpenAI's open weight reasoning pair. The gpt-oss-20b card and Ollama's gpt-oss page agree on the essentials:

Both use Apache 2.0, configurable reasoning effort, and MXFP4 MoE weights. They are text only with a 128K context window.

ollama run gpt-oss:20b
ollama run gpt-oss:120b

Full setup across Ollama, LM Studio, llama.cpp and vLLM is in how to run gpt-oss locally.

Pros: permissive licence; strong tool use and reasoning story; 20b fits many 16 GB machines. Cons: no vision; 120b is workstation class hardware.

Best local reasoning distillations: DeepSeek-R1

The full DeepSeek-R1 checkpoint is a 671B MoE (37B active) under MIT on its model card. Powerful, but the Ollama 671b tag is a 404 GB download, so it is not a laptop default. What most local users actually run are the distill sizes Ollama publishes under deepseek-r1 (step by step: how to run DeepSeek locally with Ollama):

ollama run deepseek-r1:14b

Licence nuance matters: the R1 card says the main R1 weights are MIT, but the Qwen based distills inherit Apache 2.0 from Qwen, and the Llama based distills inherit the Llama community licence. Check the specific distill card before shipping a product.

Pros: excellent math and coding reasoning for the size; MIT on the flagship weights. Cons: full 671B is impractical locally; distill licences follow their base models.

Other solid local options

• Llama 3.2 small tags (llama3.2:1b / 3b, about 1.3 to 2.0 GB on Ollama) are handy on phones and tiny boxes, but Meta's Llama licence is not Apache: acceptable for many personal uses, stricter for large commercial deployments.
• Mistral classic 7B (mistral, 4.4 GB on Ollama) remains a clean, well supported chat baseline.
• SmolLM 2 (smollm2:1.7b, 1.8 GB) is Apache 2.0 on its model card and useful for on device experiments.
• Extremely compressed specialty builds such as ternary Bonsai 2 27B can fit in roughly 6 to 7 GB, but need Prism's fork rather than stock Ollama.
• IBM Granite models such as Granite 4.0 H Small are Apache 2.0 on their Hugging Face model card, and IBM keeps publishing new Granite checkpoints there.
• Falcon from TII includes the Falcon H1 hybrid models, which use the separate Falcon LLM license rather than Apache 2.0, so read its terms first.
• For speech to text rather than chat, OpenAI's Whisper large v3 turbo is MIT licensed per its model card and runs locally.
• For the reverse, text to speech, Ollama added speech models on October 24, 2026, starting with Gradium's Phonon (ollama run phonon). It runs on macOS only for now. The English phonon model is Apache 2.0, while the 40M language tags are CC BY-NC 4.0 and noncommercial only (Ollama blog).
• Liquid AI's LFM2.5 24B MoE (~2.45B active) is aimed at on device agents; plan for about 14.5 GB of Q4_K_M weights plus KV cache.

How to run them (three common tools)

Ollama (fastest start)

1. Install Ollama from ollama.com.
2. Run a tag from the tables above.
3. Point any OpenAI compatible client at http://localhost:11434.

Compare runners in Ollama vs llama.cpp vs LM Studio.

LM Studio (GUI)

LM Studio browses Hugging Face GGUF and MLX builds, estimates fit for your machine, and exposes a local API. Prefer it when you want a desktop UI and Mac MLX paths; see MLX vs GGUF on Mac.

llama.cpp / GGUF (full control)

llama.cpp is the engine behind most GGUF apps. Download a quant from Hugging Face, then serve with llama-server or your preferred frontend. Quant names (Q4_K_M and friends) are explained in GGUF quantization explained. For multiuser GPU serving, compare vLLM vs llama.cpp.

Who this shortlist is for

Skip giant MoE flags (DeepSeek-R1 671B, huge Qwen MoE tags) unless you have server memory. Skip custom ternary forks until stock runners already feel too big: start with a standard GGUF or Ollama tag first. For a concrete Qwen setup, see how to run Qwen 3.6 locally. To wire a local model into your editor, read how to use a local LLM in VS Code. If you do have workstation class memory, our Muse Spark Open local guide shows what Meta's 816B MoE needs at each quant.

From Ahzy Capital (our own products, labeled as such): Get the free AI Agent Cheat Sheet: 24 copy and paste AI agent setups for 8 types of small business, with prompts and workflows. We make and sell these, so they aren't independent picks.

FAQ

What are the best open source LLM models to run locally right now?

For most people: Qwen3-8B on small machines, gpt-oss-20b or Gemma 3/4 12B on 16 GB, and Qwen3.8-27B on about 24 GB. Those picks balance download size, licence clarity and tooling support (Ollama, LM Studio, llama.cpp).

How do I use open source LLM models without a GPU?

CPU only inference works with llama.cpp and Ollama, but a dense 8B+ model will be slow. Prefer tiny tags (smollm2, gemma3:1b, qwen3:1.7b) or a Mac with enough unified memory. Treat anything above about 14 GB of weights as "possible but painful" on CPU alone.

Are Llama and Gemma really open source?

They are downloadable open weight models. Llama uses Meta's community licence. Gemma 3 uses Google's Gemma Terms of Use; Gemma 4's published Hugging Face cards mark Apache 2.0. If you need a classic permissive grant, prefer Qwen, gpt-oss, SmolLM2 or OLMo, and still read the exact card.

How do I deploy an open source LLM locally for an app?

Run Ollama or llama.cpp as a local HTTP server and call it with an OpenAI compatible client. For several concurrent users on one GPU, prefer vLLM or SGLang. Keep context length explicit: defaults are often much shorter than the model's maximum.

Which free open source LLM models are safe for commercial products?

Apache 2.0 and MIT weights (Qwen3/3.8, gpt-oss, SmolLM2, DeepSeek-R1's MIT card, Gemma 4 per its HF cards) are the usual starting point. Llama and Gemma 3 add extra conditions. This is not legal advice: read the licence file shipped with the exact revision you ship.

Should I fine tune these models or just prompt them?

Prompt and tool call first. Fine tune when you have a narrow style or domain that prompting cannot fix. Local fine tuning options are covered in how to fine tune an LLM locally and LoRA vs QLoRA.