Checkpoint steps
The folder checkpoints/checkpoint-<step> contains the adapter after <step> training
steps. In one step, the model learns from a batch of 8 conversations. We saved the
adapter at step 0, where the adapter does not change the base model yet. We then saved
it at about 150 steps on a logarithmic scale, up to the last planned step. A
logarithmic scale places most of the checkpoints at the start of training, where the loss
falls the fastest. We also saved the adapter at every evaluation step. Training stopped at step 1716 (epoch 1.21), because the evaluation loss did not improve anymore. The
adapter in the root folder is the final adapter. The final adapter is the same as
checkpoint-1430, which has the lowest evaluation loss. The file checkpoints.csv
lists the epoch and the losses of each checkpoint.
Loading a checkpoint
from peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer base = AutoModelForCausalLM.from_pretrained("unsloth/Qwen3-8B", dtype="bfloat16")tokenizer = AutoTokenizer.from_pretrained("EvilScript/Qwen3-8B-taboo-gold-r10-a16-e2-checkpoints")model = PeftModel.from_pretrained( base, "EvilScript/Qwen3-8B-taboo-gold-r10-a16-e2-checkpoints", subfolder="checkpoints/checkpoint-143")