sglang.io · ML Frameworks
Open source (Apache 2.0) server for language, vision and diffusion models. RadixAttention reuses KV cache across prompts that share a prefix; v0.5.22 is current as of October 2026.
SGLang is one of 20 ML framework tools on Axiomi. See all ML framework tools or how we check facts.
// guides
Ollama vs vLLM compared: setup, model formats, speed for one user or many, hardware, Mac support and the signs it is time to move from Ollama to vLLM.
SGLang vs vLLM on DGX Spark compared on setup, memory, NVFP4 models, speed and clustering, so you can pick the right server for your GB10 desktop.
Recursive Flint is an Apache 2.0 coding model with 40B total and 2.5B active parameters. How to run Flint locally with Ollama, llama.cpp, rec or Unsloth.
MOSS-Image is an Apache 2.0 open weight 5B text to image model built for text and layouts. How to run MOSS-Image in ComfyUI, file sizes and best settings.
DeepInfer is DeepSeek's open source engine for DeepSeek-V4.1-Flash. What it does, the hardware it needs, what is missing, and when vLLM or SGLang fits better.
FLUX open weights explained: every Black Forest Labs model you can download, its license, file size and VRAM, plus how to run FLUX 3 Klein in ComfyUI.
// related
vLLMvllm.aiHigh throughput open source inference and serving engine built around PagedAttention and continuous batching, with an OpenAI compatible server that scales from one GPU to clusters.ML FrameworksOpen source★ 95.7k
Transformershuggingface.coHugging Face's library defining thousands of model architectures, the de facto reference implementation for open models.ML FrameworksOpen source★ 168.3k
llama.cppgithub.comPlain C/C++ LLM inference engine that popularised GGUF quantised models running on everyday hardware.ML FrameworksOpen source★ 132.6kStratagithub.comMIT licensed local inference app and terminal that runs very large mixture of experts models on one consumer GPU by keeping busy experts in RAM and streaming the rest from SSD.ML FrameworksOpen source★ 34.8k
XGBoostxgboost.readthedocs.ioGradient boosted decision tree library for tabular data, with a scikit learn style Python API and CUDA-accelerated training and prediction.ML FrameworksOpen source★ 28.9k
Ollamaollama.comOpen source local model runner with a one line pull and run CLI, a desktop app and a local HTTP API. Uses llama.cpp-based and MLX engines; optional paid cloud models.ML FrameworksFreemium★ 185.1k