Gemma 4 E4B NLI 10k
LoRA adapter for three-way natural language inference, trained on 10,000 examples
with 2,000 sampled from each listed dataset using seed 0. FEVER inputs use evidence
as the premise and the claim as the hypothesis. Labels are entailment, neutral
and contradiction. The adapter and tokenizer are included; the base weights are
loaded separately at the revision pinned in adapter_config.json.
Training uses one epoch, batch size 8, AdamW with learning rate 0.0001, linear
warmup over 5% of steps, and a 256-token premise cap. QLoRA uses 4-bit NF4 with
double quantization and bfloat16 compute. LoRA rank is 16, alpha 32 and dropout
0.05 on text attention and MLP projections. Loss applies only to answer tokens.
Both adapters use the same training examples and recipe, with their own model
tokenizer and chat template.
The scoring implementation normalizes the logits for the distinct first tokens
of the three label words. It returns probabilities in entailment, neutral,
contradiction order. These are not full-word sequence likelihoods.
Code and scoring commands are in
understanding-probe-robustness.
From its correctness_models directory, score saved generations with
python run_scores.py --artifacts_dir /path/to/artifacts --model llama --dataset sciq --setting ID --correctness_function gemma --correctness_model joestacey/gemma4-e4b-nli-10k
This scores both training and evaluation answers by default without regenerating
answers or hidden states. The saved NLI score uses AlignScore's sentence and
reference aggregation. Evaluation of the trained adapter on held-out NLI datasets
is in progress; no final evaluation results are claimed here.