MooreMuaMu

qwen35-27b-ancient-rl-r32-step200

README

License: apache-2.0

Intended Use

python
from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "MooreMuaMu/qwen35-27b-ancient-rl-r32-step200"
tokenizer = AutoTokenizer.from_pretrained(repo_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    repo_id,
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=True,
)

Training Source

Base model: /blob/models/qwen35-27b-ancient-stage2/checkpoint-227-merged
LoRA checkpoint: /blob/experiments/ancient-swift-rl-stage2/qwen35_27b_ckpt227_uy_mn_bo_ta_full_r32_lr1e4/v0-20260811-070520/checkpoint-200
LoRA rank/alpha: 32/32
Merge dtype: bfloat16
Serialization: safetensors, 5GB max shard size

Evaluation Summary

Evaluation used the independent six-bucket held-out subset: 300 prompts, 50 per bucket, thinking enabled, temperature 0.7, top-p 0.95, n=1, seed 42. Deltas are prompt-paired against the same base-model generations where applicable. Bootstrap confidence intervals use 10,000 paired samples.

Overall semantic metrics:

Table with columns: Metric, Base, Step 200, Delta, 95% CI
Metric	Base	Step 200	Delta	95% CI
Exact	0.3367	0.3100	-0.0267	[-0.0700, +0.0133]
Char-F1	0.5708	0.5758	+0.0050	[-0.0320, +0.0431]
BERTScore-F1	0.7627	0.7901	+0.0274	[-0.0010, +0.0571]
Semantic composite	0.6061	0.6139	+0.0078	[-0.0255, +0.0415]
Has answer	0.9400

SacreBLEU2 answer-extraction metrics use sacrebleu.metrics.BLEU(tokenize="zh", max_ngram_order=2, smooth_method="exp", effective_order=True) after extracting text inside <ANS>...</ANS> and discarding any prefix before </think>.

Table with columns: Metric, Base, Step 200, Delta
Metric	Base	Step 200	Delta
Corpus BLEU-2 (%)	34.59	37.80	+3.21 pp
Sentence BLEU-2 mean (%)	48.42	47.59	-0.82 pp
Nonempty extracted answer	93.7%	98.7%	+5.0 pp

Bucket-level SacreBLEU2:

Table with columns: Bucket, Corpus BLEU-2 base -> step200, Delta, Sentence BLEU-2 base -> step200, Delta, Step200 nonempty
Bucket	Corpus BLEU-2 base -> step200	Delta	Sentence BLEU-2 base -> step200	Delta	Step200 nonempty
Tibetan annotation	65.98 -> 59.98	-6.00 pp	74.86 -> 66.85	-8.01 pp	96.0%
Tibetan translation	14.16 -> 21.22	+7.06 pp	14.20 -> 18.87	+4.67 pp	100.0%
Traditional Mongolian annotation

Decision note: step 200 is an exploratory checkpoint. It improves answer presence significantly and has positive BERTScore/SacreBLEU2 corpus movement, but aggregate semantic confidence intervals still cross zero and translation buckets are mixed.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

MooreMuaMu

Model Tree

Base

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality