github.com · Training & Fine tuning
PyTorch native library of hackable recipes for fine tuning and experimenting with LLMs.
torchtune is one of 9 training and fine tuning tools on Axiomi. See all training and fine tuning tools or how we check facts.
// guides
How to fine tune an LLM for code generation: pick a coder base model, build an instruction or FIM dataset, train a LoRA with TRL or Unsloth, test with pass@k.
Fine tune an LLM locally, step by step: when it pays off, picking a base model, formatting data, Unsloth vs Axolotl vs TRL vs mlx-lm, training and GGUF export.
// related
Unslothunsloth.aiFine tuning toolkit that speeds up LoRA and full training while cutting GPU memory use, with free notebooks.Training & Fine tuningOpen source★ 78.7k
PEFThuggingface.coParameter efficient fine tuning methods such as LoRA, letting large models adapt on modest hardware.Training & Fine tuningOpen source★ 21.9k
Axolotlaxolotl.aiConfig driven tool for fine tuning open models across many architectures and training methods.Training & Fine tuningOpen source★ 12.6k
DeepSpeeddeepspeed.aiLibrary for large scale distributed training and inference, known for the ZeRO memory optimisation family.Training & Fine tuningOpen source★ 43.3kTRLhuggingface.coHugging Face library for post training models with supervised fine tuning, DPO, GRPO and other preference optimisation methods.Training & Fine tuningOpen source★ 19.6kMegatron-LMgithub.comNVIDIA's open source codebase for training transformers at scale: training scripts plus the Megatron Core library with tensor, pipeline, data, expert and context parallelism.Training & Fine tuningOpen source★ 18.3k