How this checkpoint was obtained
- Stage: outcome + verified task-state process + error penalties.
- Checkpoint: RL rollout 49 (zero-based; 50 cumulative completed RL batches in this arm). Labels are zero-based and not interchangeable with optimizer steps.
- SFT arm:
B_full, initialized independently from Qwen/Qwen3.5-9B.
- SFT: 59 updates, 3 epochs, LR 5e-6, global batch 4. Full target supervision; no knowledge-body masking.
- Parent of this training stage: YanSong97/E2K-Qwen3.5-9B-tagged-sft-B-full-step58.
- RL arms start from B_full SFT with fresh Adam at the SFT/RL boundary. The process arm's resumptions at69/99/199 keep native optimizer, scheduler, RNG and sampler state; they are not fresh-Adam restarts.
- RL: 40 environment actions per episode, medium reasoning effort, 16K output cap,
recent4/input12K/full current state, no auto compaction, temperature1, LR1e-6constant, TP4/PP1.
- 8 episodes per batch: 4 cd82 + 4 m0r0, environment seed0, starting level0/cap1.
- Training is text/grid-based. The vision tower and MTP auxiliary tensors were not trained in this experiment; frozen inference assets were restored by the export helper.
This continuation stage covers rollout 0 through 49 inclusive (50 batches), with the stage parent named above.
Data and reward
The local observational knowledge set contains29 first-level facts (22 cd82/7 m0r0),
represented by58 question/answer rows. B/C add12 constructed state/application
examples and all arms include8 initial-look protocol examples. The data are
locally derived from OpenCode observations/replay and external editorial work;
they are not a row-for-row mirror of ziyan98's HF knowledge dataset, nor a
collection of native Claude-model teacher thoughts. Semantic evidence is scoped.
Knowledge answers are not injected into RL actor prompts.
Outcome reward is a true configured first-level clear. The correction arm uses
legal random fallback and small call-local error penalties. The process arm
additionally uses independently replay-bound best-so-far task-state progress,
capped0.2, with gamma0.99 over actual actions. This uses privileged reward-side
state/planning and does not score the truth of reasoning. Random terminal
fallback clears get no learner outcome credit. See the exact linked run protocol.
Results and limitations
training_metadata.json records checkpoint ancestry and the sampling cohort, when
available. That cohort was generated before its update: it must not be reported
as an independent evaluation of these exported weights. Group-relative native
rollout/rewards is a reply-weighted mean advantage, not environment reward.
Tags/recitation are not evidence of correct knowledge use. Audits find fabricated
coordinates, cross-game rule mixing, explanation/action mismatch and repeated
generation even in successful episodes. Learning is non-monotonic and dominated
by m0r0. The long process continuation stopped at341all-failed sampling, leaving
340as its last verified update, rather than completing400. The exact peak260..269
weights are not retained; HF249is not labelled the best checkpoint.
Files
Standard HF config/tokenizer/processor assets and60 safetensors shards, including
frozen vision/MTP, are supplied according to model.safetensors.index.json.
provenance.json lists source artifact identity, files, sizes and SHA256 values.
No optimizer, native distributed checkpoint, pickle, RNG or sampler state is
uploaded. HF weights can initialize future work but do not provide exact Adam
continuation. Original native states, where retained, stay on the source server.
Loading
import torchfrom transformers import AutoProcessor, Qwen3_5ForConditionalGenerationmodel = Qwen3_5ForConditionalGeneration.from_pretrained( "YanSong97/E2K-Qwen3.5-9B-tagged-B-process-rl-iter49", torch_dtype=torch.bfloat16, device_map="auto")processor = AutoProcessor.from_pretrained("YanSong97/E2K-Qwen3.5-9B-tagged-B-process-rl-iter49")