This run
Fresh from Qwen3.8-27B on v1 + v2 concatenated (1,982 sequences), lr 1e-5, one epoch = 248 steps. The run stopped at step 244 of 248 when its job ended (2026-11-01); the learning rate at step 244 was 1.07e-8 and the last logged loss 0.447, so these weights are the ZeRO-3 checkpoint-244 consolidated to bf16 (the four missing steps are numerically negligible). Not yet evaluated.
Serving notes
- Load as
Qwen3_5ForConditionalGeneration (the text-only class would initialise random weights);
preprocessor_config.json and video_preprocessor_config.json are included for vLLM.
- Evaluated with thinking enabled and a reasoning parser.
Training setup (all runs)
Full-parameter trajectory-level SFT with train_cca_sft.py from github.com/rifaaQ/long_ctx_train on Aurora (Intel XPU):
one training sequence per agent context generation, loss on the assistant spans only, 128k context, bf16,
DeepSpeed ZeRO-3 + CPU optimizer offload, node-local Ulysses sequence parallelism (8 ranks), AdamW, cosine schedule,
8 sequences per optimizer step, 1 epoch unless noted. Data: CCA agent trajectories from cca-large-v1
(250 training runs, 1,243 sequences, 61.9M supervised tokens) and cca-large-v2 (182 runs, 739 sequences,
47.7M supervised tokens; disjoint runs, open-weight teachers only).
Research artifact. Evaluation protocol and all numbers: longctx/results.md in the project record (github.com/rifaaQ/long_ctx_train).