github.com · ML Frameworks
MIT licensed local inference app and terminal that runs very large mixture of experts models on one consumer GPU by keeping busy experts in RAM and streaming the rest from SSD.
Strata is one of 20 ML framework tools on Axiomi. See all ML framework tools or how we check facts.
// guides
RTX 3090 BF16 runs at half rate, and a new Linux mod restores it on RTX 30 cards. What it speeds up for local LLMs, what it doesn't, the risks and setup.
Can you run DeepSeek V4.1 Flash locally? Real RAM, VRAM and disk needs by GGUF, Strata on one 12 GB GPU, the llama.cpp PR, vLLM, Ollama and DGX Spark.
// related
llama.cppgithub.comPlain C/C++ LLM inference engine that popularised GGUF quantised models running on everyday hardware.ML FrameworksOpen source★ 132.6k
Ollamaollama.comOpen source local model runner with a one line pull and run CLI, a desktop app and a local HTTP API. Uses llama.cpp-based and MLX engines; optional paid cloud models.ML FrameworksFreemium★ 185.1k
vLLMvllm.aiHigh throughput open source inference and serving engine built around PagedAttention and continuous batching, with an OpenAI compatible server that scales from one GPU to clusters.ML FrameworksOpen source★ 95.7k
ComfyUIcomfy.orgNode based open source app for running image, video and audio generation models locally, with shareable JSON workflows and support for quantized checkpoints.ML FrameworksOpen source★ 143.6k
Transformershuggingface.coHugging Face's library defining thousands of model architectures, the de facto reference implementation for open models.ML FrameworksOpen source★ 168.3k
SGLangsglang.ioOpen source (Apache 2.0) server for language, vision and diffusion models. RadixAttention reuses KV cache across prompts that share a prefix; v0.5.22 is current as of October 2026.ML FrameworksOpen source★ 38.3k