MooreMuaMu

qwen35-27b-ancient-rl-r32-step100

README

License: apache-2.0

Intended Use

python
from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "MooreMuaMu/qwen35-27b-ancient-rl-r32-step100"
tokenizer = AutoTokenizer.from_pretrained(repo_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    repo_id,
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=True,
)

Training Source

Base model: /blob/models/qwen35-27b-ancient-stage2/checkpoint-227-merged
LoRA checkpoint: /blob/experiments/ancient-swift-rl-stage2/qwen35_27b_ckpt227_uy_mn_bo_ta_full_r32_lr1e4/v0-20260811-070520/checkpoint-100
LoRA rank/alpha: 32/32
Merge dtype: bfloat16
Serialization: safetensors, 5GB max shard size

Evaluation Summary

Evaluation used the independent six-bucket held-out subset: 300 prompts, 50 per bucket, thinking enabled, temperature 0.7, top-p 0.95, n=1, seed 42. Deltas are prompt-paired against the same base-model generations where applicable. Bootstrap confidence intervals use 10,000 paired samples.

Overall semantic metrics:

Table with columns: Metric, Base, Step 100, Delta, 95% CI
Metric	Base	Step 100	Delta	95% CI
Exact	0.3133	0.3400	+0.0267	[-0.0167, +0.0733]
Char-F1	0.5499	0.5579	+0.0080	[-0.0328, +0.0495]
BERTScore-F1	0.7638	0.7657	+0.0019	[-0.0309, +0.0348]
Semantic composite	0.5915	0.6015	+0.0100	[-0.0260, +0.0473]
Has answer	0.9533

SacreBLEU2 answer-extraction metrics use sacrebleu.metrics.BLEU(tokenize="zh", max_ngram_order=2, smooth_method="exp", effective_order=True) after extracting text inside <ANS>...</ANS> and discarding any prefix before </think>.

Table with columns: Metric, Base, Step 100, Delta
Metric	Base	Step 100	Delta
Corpus BLEU-2 (%)	34.30	39.86	+5.56 pp
Sentence BLEU-2 mean (%)	46.12	48.10	+1.98 pp
Nonempty extracted answer	95.0%	94.7%	-0.3 pp

Bucket-level SacreBLEU2:

Table with columns: Bucket, Corpus BLEU-2 base -> step100, Delta, Sentence BLEU-2 base -> step100, Delta, Step100 nonempty
Bucket	Corpus BLEU-2 base -> step100	Delta	Sentence BLEU-2 base -> step100	Delta	Step100 nonempty
Tibetan annotation	59.14 -> 56.73	-2.41 pp	65.40 -> 66.82	+1.42 pp	94.0%
Tibetan translation	15.68 -> 18.15	+2.47 pp	16.43 -> 16.00	-0.43 pp	94.0%
Traditional Mongolian annotation

Decision note: step 100 is an exploratory checkpoint. Its aggregate semantic improvement is small and the confidence interval crosses zero, although SacreBLEU2 corpus score improves, mainly from Traditional Mongolian annotation.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

MooreMuaMu

Model Tree

Base

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality