TRL

huggingface.co · Training & Fine tuning

Open source

#4 in Training & Fine tuning

Hugging Face library for post training models with supervised fine tuning, DPO, GRPO and other preference optimisation methods.

#rlhf#dpo#grpo#post-training

TRL is one of 9 training and fine tuning tools on Axiomi. See all training and fine tuning tools or how we check facts.

// guides

Guides that cover TRL

SmolLM vs Llama: Which 3B Model Should You Run?
SmolLM vs Llama compared on benchmarks, licence, context, languages, memory and local setup, so you can choose between SmolLM3 and Llama 3.2 for your project.
SmolLM vs Gemma: Which Small Model to Use?
SmolLM vs Gemma compared on size, licence, context, languages, vision and audio, local setup and fine tuning, so you can pick the right small model in 2026.
LLM Distillation vs Quantization: Key Differences
LLM distillation vs quantization: distillation trains a smaller student model, quantization stores the same weights in fewer bits. When to use each, or both.
How to Fine Tune an LLM for Code Generation
How to fine tune an LLM for code generation: pick a coder base model, build an instruction or FIM dataset, train a LoRA with TRL or Unsloth, test with pass@k.
LoRA vs QLoRA vs DoRA: Which Fine Tuning Method?
QLoRA vs LoRA explained, plus DoRA and full fine tuning: how each works, real VRAM figures, quality tradeoffs, recommended settings, and when to choose which.
How to Fine Tune an LLM Locally in 2026
Fine tune an LLM locally, step by step: when it pays off, picking a base model, formatting data, Unsloth vs Axolotl vs TRL vs mlx-lm, training and GGUF export.

// related