ovi-distill

pythia-6.9b-ifeval-sft

README

License: apache-2.0

Results

IFEval (541 prompts), scored with lm-eval's IFEval checkers; the prompt under the chat template, greedy decoding, up to 1,280 new tokens, float32.

Table with columns: model, prompt-level strict %, instruction-level strict %, prompt-level loose %
model	prompt-level strict %	instruction-level strict %	prompt-level loose %
this model	47.3	59.4	50.8
Pythia-6.9B base (raw prompt, as lm-eval without a chat template)	16.3	30.2	16.5
allenai/open-instruct-pythia-6.9b-tulu (its own chat format)	16.3	27.9	18.5

Sampling at temperature 1 (8 samples per prompt): 41.3 % of samples follow every instruction; for 67.7 % of prompts at least one of the 8 does.
By the number of instructions in the prompt (greedy, prompt-level strict): one 64.9 % (305 prompts), two 28.5 % (179), three 12.3 % (57). Every training prompt has a single constraint.
500 held-out RLVR-IFeval prompts (one constraint each, scored with the project's verifier): 57.8 %.

Use

The chat format is <|user|>\n{prompt}\n<|assistant|>\n; the markers are plain text to the Pythia tokenizer, and the tokenizer's chat template writes them. Run the model in float32, as it was trained and evaluated (in bfloat16 Pythia's next-token distributions move away from float32). The context is 2,048 tokens.

python

import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer repo = "ovi-distill/pythia-6.9b-ifeval-sft"tok = AutoTokenizer.from_pretrained(repo)model = AutoModelForCausalLM.from_pretrained(repo, dtype=torch.float32, device_map="auto")messages = [{"role": "user", "content": "Describe a rainy afternoon in exactly two sentences, all in lowercase."}]inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt", return_dict=True).to(model.device)out = model.generate(**inputs, max_new_tokens=256, do_sample=False)print(tok.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Training

Data: 14,918 answers by Qwen2.5-7B-Instruct to 8,443 prompts of allenai/RLVR-IFeval (Tulu 3's instruction-following prompts, one verifiable constraint each). For each of 13,500 prompts, 4 answers were sampled (temperature 0.7, top-p 0.95), and an answer was kept only if a verifier confirms the constraint is followed, it ended on its own, and prompt and answer fit 1,024 tokens each; at most 2 answers per prompt.
Held out: 500 RLVR-IFeval prompts (the dev set above) and all of IFEval. No training prompt matches an IFEval prompt or shares a 13-word span with one outside its constraint sentence (the constraint sentences reuse IFEval's instruction templates by design).
Recipe: full fine-tuning with verl's SFT trainer, float32 weights and compute (TF32 matrix multiplications), AdamW, learning rate 2e-5 with 3 % warmup and cosine decay, no weight decay, 2 epochs (466 steps of 64 sequences), loss on the answer tokens only, seed 0; 1.9 hours on two H100 GPUs.

Limitations

A research model: no alignment or safety tuning beyond what the base model has. It can produce false, biased or offensive text. English only.
Trained on prompts with one constraint; prompts with several constraints are followed much less often (see above).
A 2023 base model with a 2,048-token context. 10 % of its greedy IFEval answers run to the 1,280-token limit, usually by repeating themselves.

License and attribution

Apache-2.0, like Pythia-6.9B. The training answers were generated by Qwen2.5-7B-Instruct (Apache-2.0); the prompts come from allenai/RLVR-IFeval (ODC-BY), part of the Tulu 3 release.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

ovi-distill

Model Tree

Base

EleutherAI/pythia-6.9b

Fine-tuned

this model

Input Modalities

Text

Output Modalities