Training data
Same 1,776 samples (940 harmful / 836 benign) as the ImmuneCoT run's IASD
dataset. Each sample = the untriggered student assistant prefill (naive CoT,
masked out of the loss) + the combined-trigger immune continuation (reasoning
- final answer, loss computed on all completion tokens). Candidates were
generated with the same base model at temperature 0.7 / top_p 0.95 /
repetition_penalty 1.2 and passed structural quality gates.
Hyperparameters
Table with columns: item, value| item | value |
|---|
| fine-tune | full-parameter |
| epochs | 2 |
| effective batch size | 32 (4 per-device x 4 accum x 2 GPU) |
| optimizer | AdamW (fused) |
| learning rate | 5e-6, linear decay, no warmup |
| precision | bf16 |
| max sequence length | 4608 |
| max completion length | 2048 |
| seed | 42 |
| framework | TRL SFTTrainer + DeepSpeed ZeRO-2 (CPU optimizer offload) |
| train loss | 0.58 avg / 0.35 final, token accuracy 88.2% |
Intended use
Research baseline: measures how much of the ImmuneCoT safety benefit is
achievable by plain SFT on the same token budget, without on-policy rollout
or the privileged-trigger context.
This model is released for research on LLM safety only.