Open Documentation
Fine-Tuning Cookbooks
& Engineering Guides
A growing repository of utility scripts, bug fixes, tutorials, and architectural frameworks to support the community navigating model fine-tuning and agentic deployments.
How to See What a Tiny Transformer Actually Learned
Three ways to see what a tiny character-level transformer learned: watch the rules emerge during training, test whether it generalized or memorized, and ablate the attention head doing the work.
Implementing LoRA from Scratch
Learn exactly how Low-Rank Adaptation works by writing the A and B matrices in vanilla PyTorch, training them, and merging the weights.
Knowledge Distillation in Practice
Learn how to extract a teacher model's true reasoning topology by implementing temperature scaling and Kullback-Leibler divergence in PyTorch.
Gradient Checkpointing with LoRA: The Detached Tensor Problem
Wondering why your training loop no longer crashes with a detached tensor error? We break down the infamous Issue #137, Hugging Face's silent PEFT patch, and how to fix the computational graph manually in custom architectures.
RAG vs. Fine-Tuning: When to use which
A technical framework for deciding between RAG for dynamic knowledge retrieval and SFT for behavioral logic.
Implementing a KV Cache From Scratch
Stop recomputing past tokens. Learn how to implement a transformer Key-Value (KV) cache in pure PyTorch to turn an O(n³) decoding bottleneck into an optimized inference loop.