// category · 20 tools · 18 open source
ML Frameworks
The numerical foundations: tensor libraries, autodiff engines, compilers and inference runtimes that every model above them depends on.
The layers of the stack
Everything in this category sits underneath the models. Tensor libraries such as PyTorch and JAX provide arrays, automatic differentiation and GPU kernels. Model libraries such as Hugging Face Transformers turn those into ready architectures. Inference engines such as vLLM, SGLang, llama.cpp, Ollama and LM Studio take trained weights and serve them as fast as the hardware allows.
Current versions (checked October 21, 2026)
These release numbers come from PyPI and GitHub on the date above:
- PyTorch 2.14.1, uploaded September 30, 2026.
- JAX 0.12.0, uploaded October 19, 2026.
- Transformers 5.20.2, uploaded October 19, 2026.
- vLLM 0.32.0, uploaded October 18, 2026.
- SGLang v0.5.22, released October 19, 2026, with what the project describes as 10x faster model loading and a PyTorch backend for Apple Silicon Macs.
These projects ship often, so treat any version number as a snapshot and check the project page before you pin a dependency.
Training framework or inference engine?
If you are building or training models, the choice is mostly between PyTorch and JAX. PyTorch has the larger ecosystem and is the default for most open model code; JAX is popular for TPU work and research that benefits from its function transformations. Our PyTorch vs JAX comparison, TensorFlow vs PyTorch vs Keras and best ML frameworks for Python cover the tradeoffs, and tinygrad vs PyTorch looks at a minimalist alternative.
If you only want to run a model, pick an inference engine instead. For one person on a laptop or Mac, llama.cpp based tools and MLX are the natural fit; see MLX vs GGUF on Mac. For a server handling many requests, vLLM and SGLang are the two to benchmark against each other, as explained in vLLM vs llama.cpp vs SGLang. Newer engines such as DeepSeek's DeepInfer are covered in our DeepInfer explainer. Each tool page below links to the project's own documentation and repository.
Ollamaollama.comOpen source local model runner with a one line pull and run CLI, a desktop app and a local HTTP API. Uses llama.cpp-based and MLX engines; optional paid cloud models.ML FrameworksFreemium★ 185.1k
vLLMvllm.aiHigh throughput open source inference and serving engine built around PagedAttention and continuous batching, with an OpenAI compatible server that scales from one GPU to clusters.ML FrameworksOpen source★ 95.7k
PyTorchpytorch.orgThe dominant deep learning framework in research, with eager execution, torch.compile and a huge ecosystem.ML FrameworksOpen source★ 105.1k
ComfyUIcomfy.orgNode based open source app for running image, video and audio generation models locally, with shareable JSON workflows and support for quantized checkpoints.ML FrameworksOpen source★ 143.6k
tinygradtinygrad.orgDeliberately small deep learning framework that aims to make it easy to add new accelerators.ML FrameworksOpen source★ 33.8k
scikit-learnscikit-learn.orgPython's standard library for classical machine learning: one fit/predict interface for linear models, trees, clustering, preprocessing, cross validation and metrics.ML FrameworksOpen source★ 68k
PyTorch Lightninglightning.aiLightweight wrapper that removes PyTorch training boilerplate and scales the same code to many GPUs.ML FrameworksOpen source★ 31.4k
TensorFlowtensorflow.orgGoogle's end to end machine learning platform, still widely deployed in production and on mobile via LiteRT.ML FrameworksOpen source★ 201.3k
Transformershuggingface.coHugging Face's library defining thousands of model architectures, the de facto reference implementation for open models.ML FrameworksOpen source★ 168.3k
llama.cppgithub.comPlain C/C++ LLM inference engine that popularised GGUF quantised models running on everyday hardware.ML FrameworksOpen source★ 132.6k
Keraskeras.ioHigh level deep learning API that runs on JAX, TensorFlow or PyTorch backends.ML FrameworksOpen source★ 64.4k
SGLangsglang.ioOpen source (Apache 2.0) server for language, vision and diffusion models. RadixAttention reuses KV cache across prompts that share a prefix; v0.5.22 is current as of October 2026.ML FrameworksOpen source★ 38.3k
JAXdocs.jax.devComposable transformations of NumPy style programs: automatic differentiation, JIT compilation and parallelism on GPUs and TPUs.ML FrameworksOpen source★ 36.5kDiffusershuggingface.coHugging Face's Python library of pipelines and building blocks for diffusion and flow models, covering image, video and audio generation, with LoRA training support.ML FrameworksOpen source★ 34.8kStratagithub.comMIT licensed local inference app and terminal that runs very large mixture of experts models on one consumer GPU by keeping busy experts in RAM and streaming the rest from SSD.ML FrameworksOpen source★ 34.8k
MLXml-explore.github.ioApple's array framework for machine learning on Apple silicon, with unified memory and a NumPy style API.ML FrameworksOpen source★ 29k
XGBoostxgboost.readthedocs.ioGradient boosted decision tree library for tabular data, with a scikit learn style Python API and CUDA-accelerated training and prediction.ML FrameworksOpen source★ 28.9k
LightGBMlightgbm.readthedocs.ioMicrosoft's tree based gradient boosting framework, designed for fast training and low memory use, with parallel, distributed and GPU learning.ML FrameworksOpen source★ 18.9kmlx-lmgithub.comPython package from Apple's MLX team for generating text with, quantizing, serving and fine tuning (LoRA or full) language models on Apple silicon, using Hugging Face Hub models.ML FrameworksOpen source★ 7.5k
LM Studiolmstudio.aiDesktop app for macOS, Windows and Linux to find, download and chat with local models, running GGUF on llama.cpp and MLX on Apple Silicon, with a local API server.ML FrameworksFreemiumHN 3// guides
Guides to ML frameworks
Best ML Frameworks for Python in 2026
The best ML frameworks for Python in 2026, ranked by job: PyTorch, scikit-learn, JAX, Keras 3, XGBoost, LightGBM, TensorFlow, MLX and more, with versions.
Tinygrad vs MLX: Which Should You Use?
Tinygrad vs MLX compared for Apple silicon and beyond: hardware support, lazy graphs, JIT and compile, LLM tooling, install and which framework fits your project.
How to Run Muse Spark Open Locally (816B MoE)
Can you run Muse Spark Open locally? Real memory needs by quant (GGUF, MLX, NVFP4), llama.cpp and MLX commands, Mac Studio setups, licence terms and limits.
SGLang vs vLLM vs llama.cpp: Which to Use?
SGLang vs vLLM vs llama.cpp compared: what SGLang is, how RadixAttention works, Docker setup, independent benchmarks, hardware and when to pick each.
KV Cache Quantization: Fit Longer Context in VRAM
KV cache quantization explained: how q8_0, q4_0 and FP8 caches cut memory, the settings for llama.cpp, Ollama, LM Studio, vLLM and MLX, and when quality drops.
How to Run LFM2.5 24B Locally
Run Liquid AI LFM2.5-24B-A2B locally with Ollama, llama.cpp or MLX: VRAM by quant, Q4_K_M sizes, Ollama 0.17.1 caveat, sampling settings and agent tips.
How to Run Qwen 3.8 27B Locally (VRAM + Setup)
Run Qwen 3.8 27B locally with Ollama, llama.cpp, LM Studio or vLLM: which quant fits 8, 16 or 24 GB, setup commands, thinking controls and context memory.
Ollama vs llama.cpp vs LM Studio: 2026 Guide
Ollama vs llama.cpp vs LM Studio compared for 2026: what each one is, ports, licences, Mac and GPU support, the context length trap, and which to pick.
What Is Safetensors? Safe Model Files Explained
What is safetensors? A plain guide to the Hugging Face model file format: why it is safer than pickle, how it compares to GGUF, and how to load or convert it.
Ollama vs vLLM: Which Should You Run?
Ollama vs vLLM compared: setup, model formats, speed for one user or many, hardware, Mac support and the signs it is time to move from Ollama to vLLM.
RTX 3090 BF16 at Full Rate: What It Means for AI
RTX 3090 BF16 runs at half rate, and a new Linux mod restores it on RTX 30 cards. What it speeds up for local LLMs, what it doesn't, the risks and setup.
SmolLM vs Llama: Which 3B Model Should You Run?
SmolLM vs Llama compared on benchmarks, licence, context, languages, memory and local setup, so you can choose between SmolLM3 and Llama 3.2 for your project.
SGLang vs vLLM on DGX Spark: Which to Use?
SGLang vs vLLM on DGX Spark compared on setup, memory, NVFP4 models, speed and clustering, so you can pick the right server for your GB10 desktop.
Recursive Flint: Run the 40B Coding Model Locally
Recursive Flint is an Apache 2.0 coding model with 40B total and 2.5B active parameters. How to run Flint locally with Ollama, llama.cpp, rec or Unsloth.
SmolLM vs Gemma: Which Small Model to Use?
SmolLM vs Gemma compared on size, licence, context, languages, vision and audio, local setup and fine tuning, so you can pick the right small model in 2026.
FLUX Open Weights: Every Model and Its License
FLUX open weights explained: every Black Forest Labs model you can download, its license, file size and VRAM, plus how to run FLUX 3 Klein in ComfyUI.
Tinygrad vs PyTorch Performance
Tinygrad vs PyTorch performance: when TinyJit and BEAM can outrun eager PyTorch, when torch.compile and the ecosystem win, and how to compare them fairly.
TensorFlow vs PyTorch vs Keras in 2026
TensorFlow vs PyTorch vs Keras in 2026: Keras 3 multibackend, which is easier to learn, open weight LLM ecosystem fit, and who should pick each stack.
How to Run Qwen 3.6 Locally (VRAM + Setup)
Run Qwen 3.6 locally with Ollama, llama.cpp or LM Studio: pick 27B or 35B-A3B by VRAM, thinking mode, vision tips, MTP speedups and sampling settings.
How to Run Gemma 4 Locally (VRAM + Setup)
Run Gemma 4 locally with Ollama, llama.cpp or LM Studio: pick E2B, E4B, 12B, 26B-A4B or 31B by memory, thinking mode, vision tips and sampling settings.
MLX vs GGUF on Mac: Which Should You Use?
MLX vs GGUF on a Mac: what each format is, how Ollama and LM Studio use them, speed tradeoffs, memory limits, fine tuning with mlx-lm, and which to download.
How to Run gpt-oss Locally (20b and 120b)
Run gpt-oss locally with Ollama, LM Studio, llama.cpp or vLLM: memory needs for 20b and 120b, setup commands, reasoning effort, harmony format and fine tuning.
How to Fine Tune an LLM Locally in 2026
Fine tune an LLM locally, step by step: when it pays off, picking a base model, formatting data, Unsloth vs Axolotl vs TRL vs mlx-lm, training and GGUF export.
How Much VRAM Do I Need for an LLM?
How much VRAM do you need for an LLM? A simple formula for weights plus KV cache, worked examples from real model configs, and what fits in 8, 16, 24 or 32 GB.