Method
On-policy self-distillation (OPSD): the student rolls out from the
untriggered assistant prefill (naive CoT); a frozen copy of the initial
base model scoring the same tokens under the combined detect+response
trigger (privileged context) provides the distillation target. The policy
advantage is the teacher-vs-student sampled-logp gap; a generalized JSD term
(token clip 10) regularizes toward the teacher distribution.
A live-teacher variant collapsed within ~16 optimizer steps (loss → 0,
entropy collapse), consistent with the failure mode reported by the OPSD
authors; the frozen-teacher variant used here is the stable configuration.
Training
Table with columns: item, value| item | value |
|---|
| fine-tune | full-parameter |
| data | same 1,790-sample IASD set as the ImmuneCoT run (940 harmful / 850 benign) |
| epochs | 2 (112 optimizer steps) |
| effective batch size | 32 (4 per-device x 2 accum x 4 GPU) |
| optimizer | AdamW (fused) |
| learning rate | 5e-6, linear decay, no warmup |
| precision | bf16 |
| max length / completion | 4608 / 2048 |
| rollout | vLLM colocate (mem 0.25), temperature 0.7, top_p 0.95, top_k 20 |
| seed | 42 |
| framework | IASD trainer + DeepSpeed ZeRO-2 (CPU optimizer offload) |
Weights correspond to checkpoint-112 (the final training state).
Intended use
Research baseline: measures how much safety benefit on-policy self-distillation
from a privileged-trigger context achieves without the ImmuneCoT PoE fusion.
This model is released for research on LLM safety only.