BRIE (2025–2026)

Most conversations around fine-tuning assume either industrial-scale alignment corpora or narrow task formatting. I started BRIE to test a different question: how little data does it actually take to shift the dispositional and philosophical register of a small open-weight model — including one small enough to train locally on a laptop — while preserving its general capabilities?

The corpus is 1,213 chat-format examples covering continental philosophy, speculative writing, and contemplative prose, LoRA-fine-tuned into four base models: Qwen2.5 at 0.5B, 3B, and 7B, and Llama-3.2-3B. Training used peft and trl’s SFTTrainer at LoRA rank 16 and alpha 32 (the 0.5B trained locally on an M4 MacBook Pro; the 3B on a rented RTX 5090). Because larger models adapt to a small stylistic corpus quickly, the 7B run raised dropout to 0.1 and trained a single epoch, while the 0.5B and 3B adapters were selected by validation loss at roughly 1.4 of 2 epochs.

As the project unfolded, the evaluation harness became the larger half of the work. Each adapter’s output was compared against its own base model on 57 in-domain judgments (25 unique held-out prompts) plus a 15-prompt out-of-domain control, judged blind and position-randomized by third-party frontier models. Rather than treating a single benchmark pass as static ground truth, I ran the exact same byte-identical generations through two separate judge generations — a 2025 panel drawn from Claude 3.5 Sonnet and Opus 4, GPT-4o, and Gemini 2.5 Flash Lite, and a 2026 panel of Claude Haiku 4.5, Claude Sonnet 4.5, GPT-5, and Gemini 3 Pro — to separate what the fine-tune did from what a particular generation of judges preferred.

That longitudinal separation mattered, because the numbers moved across judge generations. In-domain mean win rates shifted from 88.9% to 78.9% for the 3B, 93.0% to 85.5% for the 7B, and 80.1% to 64.5% for Llama-3.2-3B under the 2026 panels, while out-of-domain results for the 3B and 7B models held near parity. A length-bias check on the 7B’s 2026 results found the longer response won 50.3% of decisive judgments — close enough to chance to indicate that judge preference tracks register rather than verbosity. Every pinned judgment file ships in the repository alongside a single script that re-derives the full scorecard from the raw judge outputs.