# Sabr Research

> Sabr Research builds specialized small language models and reasoning architectures, and publishes technical cookbooks and research on fine-tuning, model internals, and deployment.

The cookbooks below are grouped the same way they are on the site itself: https://sabrresearch.com/cookbooks, organized by where each piece sits in the fine-tuning pipeline, from data preparation through deployment.

## Data & Evaluation

- [Power Analysis for Benchmark Design | SR Cookbooks](https://sabrresearch.com/cookbooks/power-analysis-for-benchmark-design): Calculate statistical power and minimum detectable accuracy gaps for LLM benchmarks using Python to avoid reporting sampling noise as signal.
- [Deduplication at Scale: MinHash & LSH in Python | SR Cookbooks](https://sabrresearch.com/cookbooks/pure-python-minhash-lsh-deduplication): Implement MinHash and Locality-Sensitive Hashing (LSH) in pure Python to eliminate the quadratic bottleneck of near-duplicate detection for ML dataset curation.
- [How to Check a Fine-Tuning Dataset Before You Train | SR Cookbooks](https://sabrresearch.com/cookbooks/evaluate-fine-tuning-dataset): Avoid common fine-tuning failures. Learn how to validate chat templates, prevent silent truncation, verify loss masking, and check for data leakage before spending GPU hours.

## Post-Training Techniques

- [The Math of DPO, Explained | SR Cookbooks](https://sabrresearch.com/cookbooks/math-of-dpo-explained): A step-by-step mathematical derivation of Direct Preference Optimization (DPO), showing how the partition function cancels and verifying the implicit reward gradient in Python.
- [Fixing EOS Errors: Why Fine-Tuned Models Talk to Themselves | SR Cookbooks](https://sabrresearch.com/cookbooks/fix-eos-token-infinite-generation): A technical guide to fixing the infinite generation bug in SFT by properly mapping EOS tokens and chat templates in Hugging Face.
- [PEFT Explained: LoRA vs. QLoRA | SR Cookbooks](https://sabrresearch.com/cookbooks/lora-vs-qlora-explained): Understand the architectural differences between LoRA and QLoRA, and learn when to use each Parameter-Efficient Fine-Tuning technique based on your VRAM limits.
- [How to Train Custom Tokens with LoRA | SR Cookbooks](https://sabrresearch.com/cookbooks/train-custom-tokens-with-lora): Learn how to fix untrained embedding errors when adding custom tokens to an LLM vocabulary during PEFT and LoRA fine-tuning.
- [Catastrophic Forgetting in Fine-Tuning | SR Cookbooks](https://sabrresearch.com/cookbooks/catastrophic-forgetting-in-fine-tuning): What catastrophic forgetting is, why it happens when you fine-tune an LLM, what it costs you, and how to spot it, plus when a specialized small model can safely ignore it.
- [Implementing LoRA from Scratch in PyTorch | SR Cookbooks](https://sabrresearch.com/cookbooks/implementing-lora-from-scratch-pytorch): Learn how to write a custom LoRA wrapper in pure PyTorch. Freeze a base model, train low-rank A and B matrices, and merge the weights without using PEFT.
- [LoRA & Gradient Checkpointing: The Phantom Bug | SR Cookbooks](https://sabrresearch.com/cookbooks/lora-gradient-checkpointing-detached-tensor-fix): Wondering why combining LoRA with gradient checkpointing no longer crashes your script? A deep dive into PyTorch detached tensors, Hugging Face PEFT fixes, and manual graph attachment.
- [Knowledge Distillation from Scratch: Pure PyTorch | SR Cookbooks](https://sabrresearch.com/cookbooks/pure-pytorch-knowledge-distillation): Learn the underlying math of Knowledge Distillation. Move beyond standard SFT by implementing temperature scaling and Kullback-Leibler divergence from scratch in PyTorch.

## Model Architecture & Internals

- [RAG vs SFT: When to Use Which | SR Cookbooks](https://sabrresearch.com/cookbooks/rag-vs-sft): A technical breakdown of when to use Retrieval-Augmented Generation (knowledge) versus Supervised Fine-Tuning (behavior) in enterprise AI pipelines.
- [Speculative Decoding From Scratch | SR Cookbooks](https://sabrresearch.com/cookbooks/speculative-decoding-from-scratch): Implement speculative decoding from scratch using NumPy to verify losslessness via rejection sampling and calculate wall-clock speedup bounds.
- [Inspecting What a Tiny Transformer Actually Learned | SR Cookbooks](https://sabrresearch.com/cookbooks/what-a-tiny-transformer-learned): A technical guide to probing a character-level PyTorch Transformer. Learn how to measure rule acquisition, test generalization, and ablate attention heads.
- [Implementing a KV Cache From Scratch: Pure PyTorch | SR Cookbooks](https://sabrresearch.com/cookbooks/pure-pytorch-kv-cache): Learn the mechanics of autoregressive LLM optimization. Implement a transformer Key-Value (KV) cache from scratch in PyTorch to massively speed up decoding.
- [Tokenizers From Scratch: BPE vs MaxMatch | SR Cookbooks](https://sabrresearch.com/cookbooks/pure-python-tokenizers-bpe-maxmatch): Stop treating tokenization as a black box. Learn how Byte Pair Encoding (BPE) and MaxMatch split text differently, and what mathematically happens when you add new tokens for fine-tuning.

## Deployment & Infrastructure

- [Self-Hosting an Open-Weight Coding Agent on AWS | SR Cookbooks](https://sabrresearch.com/cookbooks/self-hosting-open-weight-coding-agent-aws): A complete guide to self-hosting open coding models on AWS EC2 with vLLM, connecting VS Code Continue, running VPC-isolated MCP tools, and analyzing team infrastructure costs.
- [Building Local MCP Servers for VS Code & Cursor with FastMCP | SR Cookbooks](https://sabrresearch.com/cookbooks/building-local-mcp-servers-fastmcp-vscode): A step-by-step technical guide to building Python MCP servers with FastMCP, configuring VS Code and Cursor mcp.json, and debugging stdio JSON-RPC streams.
- [What is the Model Context Protocol (MCP)? | SR Cookbooks](https://sabrresearch.com/cookbooks/what-is-mcp): A technical architecture guide to the Model Context Protocol (MCP). Learn how it solves the M x N integration problem for AI agents and tool calling.

## Research & Case Studies

- [Stealth Content Injection in LLMs](https://sabrresearch.com/blogs/llm-stealth-injection): Your AI agent does not always see the same thing as you.
- [Language Models are not Thinking Machines](https://sabrresearch.com/blogs/llm-thinking): When semantic prior overrides user constraints.
- [Specializing SLMs for Appellate Law with Proprietary SFT](https://sabrresearch.com/blogs/chains): SR-AppellateLaw - SLM Reasoning
- [Decoding LLM Hallucinations](https://sabrresearch.com/blogs/hallucination): A Technical Review of LLM Errors and Attribution Frameworks
- [SAGE AI: 70% Accuracy in Legal Outcome Prediction](https://sabrresearch.com/blogs/legal): Benchmarking SAGE AI on the AnnoCaseLaw dataset.
- [RAL: Retrieval Augmented Logic](https://sabrresearch.com/blogs/ral): What are RAL and how are they useful?
- [SAGE AI: A Generic Decision Engine](https://sabrresearch.com/blogs/sage): Engineering the Architecture of Organizational Decision Making.
- [Enhancing LLM Reasoning with Agentic Systems](https://sabrresearch.com/blogs/agentic-reasoning): How structure helps enhance reasoning capabilities in complex environments.
- [Case Study: High-Performance Backtesting Infrastructure](https://sabrresearch.com/blogs/backtest-framework): Delivering a distributed, cloud-native framework to accelerate research loops for institutional partners.
- [Case Study: Engineering a Financial Data Factory](https://sabrresearch.com/blogs/data-infra): Architecting a serverless 'Zero-Touch' pipeline to handle large-scale ingestion for a quantitative client.
- [Case Study: Dynamic Sector Analysis for Risk Modeling](https://sabrresearch.com/blogs/correlation-clusters): Deploying unsupervised learning engines to identify 'empirical sectors' and hidden correlations in real-time.
- [Case Study: Validating Alpha with Synthetic Data](https://sabrresearch.com/blogs/simulation-framework): Moving beyond historical backtesting by generating synthetic market universes to quantify strategy robustness.

## Optional

- [Home](https://sabrresearch.com/): company overview
- [Enterprise](https://sabrresearch.com/enterprise): specialized SLM deployment for enterprise
- [Datasets](https://sabrresearch.com/datasets): reasoning trace and evaluation datasets
- [Illusion of Reasoning](https://sabrresearch.com/illusion-of-reasoning): an in-browser, interactive small model demo
- [The Case for Specialized Small Language Models](https://sabrresearch.com/whitepapers/specialized-slms): whitepaper