Training Hub: continuous learning for LLMs

Language model post-training transforms raw, pre-trained models into functional, instruction-following assistants through targeted optimization. Supervised fine-tuning utilizes message-formatted data and chat templates to enable multi-turn dialogue capabilities. To address catastrophic forgetting during continual learning, Orthogonal Subspace Fine-Tuning (OSFT) preserves critical weight components while updating lower-rank subspaces. Memory-efficient techniques like LoRa approximate full weight updates using low-rank matrix decomposition, significantly reducing computational overhead. Furthermore, reinforcement learning methods such as Group Relative Policy Optimization (GRPO) employ rules-based verifiers to refine reasoning and alignment by rewarding optimal solution paths. These methodologies are accessible via the open-source Training Hub library, which provides a unified, function-based abstraction for implementing state-of-the-art post-training algorithms across various hardware configurations.