Gemma 2 · closed-book QA fine-tune

Gemma 2 2B QA

Gemma 2 2B, QLoRA fine-tuned on our 7,141-pair closed-book legal/financial QA set (val ppl 4.26). Same questions as the from-scratch models' QA stage, so you can see what 2.6B pretrained parameters buy over 125M and 500M trained from scratch.

HuggingFace weights → SLM Arena 21: compare all 21 models →

2.6B

parameters

256K

vocab

8,192

context

4-bit

NF4 quantized

QA-SFT

fine-tuned

Validation metrics along the Gemma 2 2B lineage

Each perplexity is measured on that stage's own validation set, so read the trend as 'how well the model fits its own stage's data', not as one curve on one dataset. DPO and RLAIF optimize preferences rather than likelihood, so they log preference margin and reward instead of perplexity. Click a stage to open that model.

Base

n/a

Google's weights, not trained by us

→

QA SFT

ppl 4.26

QA val

→

Instruct

ppl 1.90

instruction val

→

DPO

QLoRA-DPO

preference-trained, no ppl

RLAIF

loss 0.63→0.27

best-of-4 SFT, no ppl

→

RAFT on DPO

ppl 1.22

RAFT val

RAFT on RLAIF

ppl 1.25

RAFT val

ask the QA fine-tune a question

ready

The fine-tuned model's answer will appear here.

What this is pretrained base · not ours

This is google/gemma-2-2b-it, Google DeepMind's instruction-tuned Gemma 2 2B. Unlike our 125M and 500M models (which we trained from random weights), this one arrives already pretrained on Google's corpus and instruction-tuned to answer questions. It is the starting point, not an output, of our Gemma work.

With ~2.6B parameters it holds roughly 20x the usable knowledge of our 125M and 5x the 500M, so it answers general questions fluently out of the box. What it does not know is our legal corpus. That gap is exactly what our fine-tunes close: teaching it to recall specific holdings and filings, or to answer grounded in a retrieved passage.

Served 4-bit quantized (bitsandbytes NF4) on a single GPU, scale-to-zero, so the first question may take ~20–40s while the model wakes from idle. The same endpoint also serves our fine-tuned Gemma, which is what the comparison sites below query.

closed-book Q&A: base vs fine-tuned → legal QLoRA fine-tune → our 125M base → our 500M base →

what this model cost to build

$2.11 total to assemble, on real Modal billing

Fine-tuning (SFT) $2.11

Stage	Cost	Share
Pretraininggoogle/gemma-2-2b-it reused (Google paid for it)	$0	0%
QA dataset (shared)1/9 share of the 7,141-pair closed-book QA set	$0.40	19.0%
Fine-tuning (QA SFT)full fine-tune on H100	$1.71	81.0%
Total to assemble	$2.11	100%

Pretraining was $0: we started from Google's released weights. The entire model cost $2.11 to assemble, versus ~$120 for our from-scratch models of a quarter the size.

Figures are actual invoiced Modal usage (GPU + CPU + memory) for this model’s lineage, not estimates. Shared inputs (the QA and preference datasets) are charged at this model’s share, and their generation also used ~$1–2 of external LLM API calls, included above. Serving is billed separately and scales to zero.