Evaluation
The reported headline score is the best validation mean@16 observed during training.
It is not necessarily the score of the uploaded last checkpoint.
Table with columns: Dataset, Method, Model, Uploaded checkpoint, Best val mean@16, Best step, Final val mean@16| Dataset | Method | Model | Uploaded checkpoint | Best val mean@16 | Best step | Final val mean@16 |
|---|
| biology | SDPO | Qwen3-8B | global_step_100 | 53.87% | 90 | 52.62% |
Raw result files:
results/validation_mean16.csvresults/training_scores.csvartifacts/config.yamlartifacts/wandb-summary.json
Training Setup
- Base model:
Qwen/Qwen3-8B
- Dataset:
biology
- Method:
SDPO
- EMA teacher update rate:
0.05
- Uploaded weights: last checkpoint,
global_step_100
- Validation metric used for headline score:
val-aux/*/mean@16
- Validation sampling:
n=16
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
repo_id = "SeongryongJung/qwen3-8b-biology-sdpo-ema005"
tokenizer = AutoTokenizer.from_pretrained(repo_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
repo_id,
torch_dtype="auto",
device_map="auto",
trust_remote_code=True,
)
Notes
Most best-validation intermediate checkpoints were not retained as full actor checkpoints because
training kept only the latest actor checkpoint. Therefore, this repository publishes the last
checkpoint and records the best validation score separately.
