## 🧠 Frameworks, Methodologies & Knowledge Base

### Model Lifecycle Framework

```
Problem Definition → Data Audit → Baseline Selection → Training/Fine-Tuning
    → Evaluation Harness → Optimization (quant/distill) → Staging Deploy
    → Production Monitor → Drift Detection → Retrain Trigger
```

### Architecture Decision Matrix

| Scenario | Recommended Starting Point | Rationale |
|---|---|---|
| General chat / assistant | API (GPT-4o, Claude) or Llama-3.1-70B + LoRA | Strong baselines, fast iteration |
| Domain Q&A (docs < 1M tokens) | RAG + small embed model (e.g., bge-large) | Cheaper, updatable, auditable |
| Domain style/tone adaptation | SFT or DPO on 1K-50K curated examples | Behavior shaping without full retrain |
| Code generation | Code-specialized models (DeepSeek-Coder, Codestral) | Better syntax/logic than generalists |
| On-device / edge | Quantized 1-3B models (Phi-3-mini, Gemma-2B) | Latency and privacy constraints |
| Multimodal (image+text) | LLaVA, InternVL, GPT-4V API | Task-dependent; API for prototyping |

### Fine-Tuning Methodology

**LoRA / QLoRA Recipe (default for teams < 8 GPUs):**
- Rank: 8-64 (higher for complex style shifts)
- Target modules: `q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj`
- Learning rate: 1e-4 to 2e-5 with cosine decay
- Epochs: 1-3 with early stopping on held-out eval
- Tools: `peft`, `trl`, `axolotl`, `LLaMA-Factory`

**Full SFT (multi-GPU):**
- Use FSDP or DeepSpeed ZeRO-3
- BF16 mixed precision; gradient checkpointing enabled
- Sequence length packing for throughput
- Save every N steps; keep best + last 3 checkpoints

**Preference Optimization (DPO/IPO):**
- Requires 5K+ high-quality preference pairs
- Beta tuning critical (0.1-0.5 typical)
- Always compare against SFT-only baseline

### Evaluation Harness Design

```python
# Eval suite structure (conceptual)
eval_suite = {
    "capability": ["MMLU-subset", "GSM8K", "HumanEval"],
    "domain": ["custom-held-out-QA", "adversarial-prompts"],
    "safety": ["toxicity-classifier", "jailbreak-suite"],
    "regression": ["golden-set-v2.3"],  # pinned production cases
    "latency": ["p50-p99-throughput-bench"]
}
```

- **Golden set**: 200-500 real production queries with expected outputs; version-controlled.
- **LLM-as-judge**: Calibrate against human labels monthly; use chain-of-thought rubrics.
- **Statistical significance**: Report confidence intervals, not single-run deltas.

### Inference Optimization Toolkit

| Technique | Typical Gain | Trade-off |
|---|---|---|
| INT4 GPTQ/AWQ | 3-4x throughput | Minor quality loss on rare tokens |
| FP8 (H100) | 2x throughput | Hardware-specific |
| Speculative decoding | 1.5-2.5x latency reduction | Draft model tuning needed |
| Continuous batching (vLLM) | 2-10x vs. static batching | Memory fragmentation management |
| KV-cache quantization | 30-50% memory savings | Long-context quality impact |

**Serving stack recommendations:**
- **High throughput API**: vLLM or TensorRT-LLM
- **Multi-framework**: Triton Inference Server
- **Edge**: ONNX Runtime, llama.cpp, MLX (Apple Silicon)

### Data Engineering Standards

- **Deduplication**: MinHash/LSH at document level before training
- **Quality filtering**: Perplexity filtering, heuristic rules, classifier-based
- **Format**: JSONL with `messages` schema for chat; include metadata (source, date, license)
- **Splits**: Temporal splits for production data; never random-split time-series user logs

### MLOps Checklist

- [ ] Experiment tracking (W&B, MLflow, Neptune)
- [ ] Model registry with stage promotion (dev → staging → prod)
- [ ] Automated eval gate in CI (block deploy if golden-set regression > 2%)
- [ ] Feature/data versioning (DVC, lakeFS)
- [ ] Production logging: input hash, model version, latency, token counts
- [ ] Drift alerts: embedding distribution shift, output length anomalies
- [ ] Rollback playbook documented and tested

### Key Papers & References (conceptual anchors)

- Attention Is All You Need (Vaswani et al.) — Transformer foundation
- LoRA (Hu et al.) — Parameter-efficient fine-tuning
- DPO (Rafailov et al.) — Direct preference optimization
- Scaling Laws (Kaplan, Chinchilla) — Compute-optimal training
- RAG (Lewis et al.) — Retrieval-augmented generation