Chapter 1: Ingest & Clean Domain Knowledge
Data Purity Gate, Continual Pre-Training (CPT), and Tokenizer Fertility Auditing
Before an AI can answer questions, it must understand specialized domain vocabulary. During Continual Pre-Training (CPT), the AI reads millions of words of raw text line-by-line without Q&A templates, practicing predicting every single next word to absorb native terminology into its memory.
Raw web data is full of dialect intrusions, corrupted encodings, and foreign copulas. Blindly throwing text away shrinks the dataset, while feeding dirty text pollutes the model. The Purity Gate uses Repair-not-Reject logic (matching person, number, and tense) to purify text before it reaches training.
📐 Mathematical Formulation: Causal Pretraining Objective
Continual Pre-Training optimizes autoregressive maximum likelihood over the unstructured token sequence $X = (x_1, x_2, \dots, x_T)$:
$$\mathcal{L}_{\text{CPT}}(\theta) = -\sum_{t=1}^T \log P_\theta(x_t \mid x_1, x_2, \dots, x_{t-1})$$Every token position contributes equally ($L=1.0$), ensuring the model learns transition probabilities between domain entities, technical terms, and regional grammatical auxiliaries.
⚠️ Production War Story: The Tokenizer Extension Trap
> 2.8 tok/word) shattered common verbs into 3–4 meaningless subword character fragments, forcing attention layers to spend 70% of their representational capacity reassembling letters rather than learning dialect grammar and vocabulary.
^[\u0900-\u097F]+$), and warm-starting new token embeddings as the centroid of constituent subtokens:
Chapter 2: Adapt with LoRA & Q&A Masking
Prompt Masking, Coverage Matrices, Low-Rank Sidecars, and the Softmax Next-Word Race
Retraining a 4-billion parameter brain is slow and burns millions of dollars. Instead, LoRA (Low-Rank Adaptation) freezes the giant base model and attaches tiny, high-precision adapter sidecars. When fine-tuning on Q&A flashcards, we only grade the AI on its answers (masking the questions so it doesn't waste memory).
In Supervised Fine-Tuning, user instructions and system prompts are loss-masked (L = 0.0). The model is not penalized for what the user wrote; backpropagation runs strictly on the gold response (L = 1.0).
The adapter produces a hidden state vector nudge $\Delta h$. When multiplied by the language modeling head ($W_{\text{vocab}}$), this nudge shifts output logits and the Softmax probability distribution: $P(w_i) = \frac{\exp(z_i)}{\sum \exp(z_j)}$, elevating authentic domain terms over generic baseline words.
Step 0: Matrix B = 0. The model currently favors generic word "था" (42%) over target word "छो" (6%). Click "Train 1 Step" or "Auto-Train" to see the adapter learn!
Enterprise teams do not generate synthetic training data randomly. They maintain a 2D Coverage Matrix (e.g. Person × Tense for languages, or Document Type × Disclosure for Finance). An LLM self-repair loop verifies generation against a strict gate and retries up to 3 times to fill matrix gaps.
📐 Mathematical Formulation: LoRA vs. DoRA Decomposition
LoRA: Adds a low-rank adapter $\Delta W = \frac{\alpha}{r} B \cdot A$ to the frozen base weight $W_0$, where $B \in \mathbb{R}^{d \times r}$ and $A \in \mathbb{R}^{r \times k}$:
$$h = W_0 x + \Delta W x = W_0 x + \frac{\alpha}{r} B A x$$DoRA: Decomposes weights into magnitude $m \in \mathbb{R}^{1 \times k}$ and directional matrix $V$:
$$W = m \frac{W_0 + \Delta W}{\|W_0 + \Delta W\|_c} = m \frac{W_0 + \frac{\alpha}{r}BA}{\|W_0 + \frac{\alpha}{r}BA\|_c}$$DoRA mirrors full fine-tuning dynamics by decoupling directional updates from norm scaling.
⚠️ Production War Story: The Attention-Only Fallacy
q_proj, v_proj) while leaving feed-forward MLP layers (gate_proj, up_proj, down_proj) frozen in an attempt to conserve VRAM during supervised fine-tuning.
--lora-layers 16) completely eliminated dialect auxiliary agreement failures.
gate/up/down) to ensure grammatical markers and domain knowledge are fully internalized.Chapter 3: Train & Optimize
Loss Dynamics, SFT Step Budgeting (1.1 × Formula), and Apple Silicon RAM Headroom
Fine-tuning is an optimization journey. We want error to drop steadily until the model produces authentic domain answers. But training too long causes catastrophic overfitting, while stopping too early leaves the model half-trained.
--grad-checkpoint)
--grad-checkpoint holds forward activations in memory, avoiding redundant recomputations.
⚠️ Production War Story: The Ultra-Early Checkpoint Illusion
< 0.35 epochs) solely because a small validation sample yielded an artificially low loss number before the model had processed the full corpus.
⚠️ Production War Story: The Silent Deduplication Disaster
Chapter 4: Align, Evaluate & Deploy
SimPO vs DPO, Held-Out Generalization Checks, and Standalone Checkpoint Weight Fusion
Knowing facts isn't enough; the AI must have good judgment. Preference Alignment trains the model on pairs of answers (Winner vs. Loser), rewarding concise, accurate responses while penalizing hallucinations and verbose fluff.
Directive 7: Purity-gate pass rate and training loss are not evidence of generalization. An overfit adapter can drive loss to 0.05 while failing completely on unseen native phrasing.
Deploying adapters as sidecars introduces runtime compute overhead. In production, we run scripts/fuse_checkpoint.py to bake adapter matrices directly into the base weights (W_fused = W_0 + (α/r) · B · A), yielding a standalone model for native serving.
📐 Mathematical Formulation: SimPO Objective
SimPO directly aligns policy $\pi_\theta$ using length-normalized log-likelihood margins without requiring an extra reference model $\pi_{\text{ref}}$ in GPU memory:
$$\mathcal{L}_{\text{SimPO}}(\theta) = -\log \sigma \left( \frac{\beta}{|y_w|} \log \pi_\theta(y_w \mid x) - \frac{\beta}{|y_l|} \log \pi_\theta(y_l \mid x) - \gamma \right)$$where $\gamma > 0$ enforces a target margin penalty and $|y|$ prevents the model from winning purely by generating longer sentences.