Configuration
- Architecture: Qwen3_5ForConditionalGeneration; qwen3_5.
- 32 text layers: 24 linear-attention + 8 full-attention; hidden size 2,560.
- Vocabulary 280,320 = 248,320 original rows + 32,000 new rows.
- New IDs: 248320–280319; EOS: 248044; tied input/output embeddings.
- One configured MTP layer. MTP was not adapted by Stage A.
Merge evidence
MERGE_VERIFICATION.json records source revision 7fcfce2243638b1edf98ea7974ff900301f02e44
of the StageA-RealText training artifact and:
old_rows_byte_exact: true
all_other_shard_tensors_byte_exact: true
changed_new_rows: 32000
- Embedding shape
[280320, 2560], stored as BF16.
These are merge/integrity checks, not a quality benchmark. They do not establish improved
Amharic generation, resolved repetition, English retention, or production readiness.
The card update did not change weights, tokenizer, source repositories, or visibility.
Intended next stage
This is the intended starting point for the subsequent full text continued-pretraining stage.
The cleaned packed dataset
contains approximately 1.947B packed tokens; that is a corpus size, not a claim that this model
has already trained on that corpus.
The recorded integration target is transformers 5.5.4. Preserve sensitive FP32 linear-attention
tensors and do not assume an ordinary load/save round trip preserves MTP. For SM120 custom wheel
requirements, usage, and limited measured throughput, see the
wheel card.
Those short training-stack throughput figures are not model-quality results or a sustained
throughput guarantee. FlashQLA remains a planned, unbenchmarked comparison in this runtime.