Levi980623

Qwen3-8B-Plan-SFT

README

License: apache-2.0

Output format (Qwen3-native thinking container)

text

<think><strategy>...high-level plan...</strategy><execution>...detailed reasoning...</execution></think>\boxed{...}

Prompt (user turn), applied with add_generation_prompt=True, enable_thinking=True:

text

Solve the following mathematics problem.First write a concise high-level plan inside <strategy>...</strategy>.Then follow that plan in <execution>...</execution> with detailed reasoning.Finally provide the answer in \boxed{...}.Do not include the answer or detailed calculations in the strategy.Problem:{question}

Decoding used for all evaluations below: temperature 0.6, top_p 0.95, top_k 20, max 8192 new tokens.

python

from transformers import AutoModelForCausalLM, AutoTokenizertok = AutoTokenizer.from_pretrained("Levi980623/Qwen3-8B-Plan-SFT")model = AutoModelForCausalLM.from_pretrained("Levi980623/Qwen3-8B-Plan-SFT", dtype="auto", device_map="auto")msgs = [{"role": "user", "content": PROMPT.format(question="What is the sum of the first 100 positive integers?")}]ids = tok.apply_chat_template(msgs, add_generation_prompt=True, enable_thinking=True, return_tensors="pt").to(model.device)out = model.generate(ids, max_new_tokens=4096, do_sample=True, temperature=0.6, top_p=0.95, top_k=20)print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=False))

Training data

Source: open-thoughts/OpenThoughts3-1.2M @ 61bcf9d4eb38b30295efc2021227a63cc5bb34c8, math domain only, seed 42 streaming sample, deduplicated by normalized question; 9800 train / 200 validation rows.
Execution = the reference write-up after </think> (300–1,500 tokens); answer = its \boxed{}. Full-row lengths for this tokenizer: max 2994 tokens, mean 808.0 (cap 8192, nothing truncated).
Strategies were extracted from (question, reference) by Qwen/Qwen3-235B-A22B-Instruct-2507-FP8 (revision e156cb4efae4, the official FP8 release of the 235B teacher, self-hosted with vLLM, temperature 0), gated by hard rules (no answer leakage, no tags) and a consistency judge; 2219 rows use a single automatic re-extraction after a leak.
Benchmark contamination filter (exact + 8-gram near-duplicate) against MATH-500, AIME24, AIME25, AMC23: all checked.
reference_verification per row (advisory 235B judge, not a gold check): judge_approved 3664, judge_flagged 6336, unverified 0. OpenThoughts3 has no gold answers; no accuracy is claimed.

Training (SURE Appendix D / Table 8 Stage-1 settings)

Table with columns: epochs, lr, warmup, global batch, micro batch, max len, precision, optimizer, sched, wd, clip, distributed
epochs	lr	warmup	global batch	micro batch	max len	precision	optimizer	sched	wd	clip	distributed
3	4e-05	0.05	32	1	8192	bf16 compute / fp32 master	AdamW

921 optimizer steps, completion-only token-weighted CE (prompt masked), activation checkpointing, wall-clock ≈ 17.6 h. Final train loss 0.048; validation loss 0.501 (epoch-1 ≈ 0.42: the third epoch over-fits in the usual 3-epoch-SFT way; kept as prescribed).

Evaluation on 200 held-out validation questions (never trained on)

Table with columns: test, n, result
test	n	result
fresh-process reload (`from_pretrained`)	–	PASS; 24/24 sampled tensors changed vs base
smoke generation format success	20	100.0% (boxed answer present 100.0%)
natural generation: strategy / execution / transition / answer parse	100	100.0% / 100.0% / 100.0% / 100.0%
natural generation: matches reference `\boxed{}`	100	28.0% (reference-match, not gold accuracy)
self-plan adherence, rubric v1 (strict, penalises arithmetic slips)

Judge for the adherence tests: Qwen/Qwen3-235B-A22B-Instruct-2507-FP8 (non-thinking, temperature 0). Status flag: LOW_PLAN_ADHERENCE — the task's acceptance rule treats adherence below 0.70 as low; rubric v2 gives 61.0%, so this is reported as a finding, not hidden. Caveat: under rubric v1 the judge marked most executions "contradicted" for arithmetic or verification mistakes even when the execution used exactly the strategy's method; rubric v2 asks it to name the method actually used and compare methods only. Both are reported; neither is a ground-truth measurement. Format compliance is near-perfect, answer agreement with the (unverified) references is low — this checkpoint is a Stage-1 initialization for later latent-planning work, not a tuned math solver.

Provenance

Pipeline, tests, dry-runs, decisions and blockers: /PHShome/yl463/sure_stage1_autonomous/ (RESULTS.md, wiki/decisions.md). Card regenerated 2026-10-10T13:05:38-0400 from the evaluation files; weights unchanged since the training export.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

Levi980623

Model Tree

Base

Qwen/Qwen3-8B

Fine-tuned

this model

Input Modalities

Text

Output Modalities

Text

Supported Functionality

Dedicated EndpointsContainer

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

Output format (Qwen3-native thinking container)

text

<think><strategy>...high-level plan...</strategy><execution>...detailed reasoning...</execution></think>\boxed{...}

Prompt (user turn), applied with add_generation_prompt=True, enable_thinking=True:

text

Solve the following mathematics problem.First write a concise high-level plan inside <strategy>...</strategy>.Then follow that plan in <execution>...</execution> with detailed reasoning.Finally provide the answer in \boxed{...}.Do not include the answer or detailed calculations in the strategy.Problem:{question}

Decoding used for all evaluations below: temperature 0.6, top_p 0.95, top_k 20, max 8192 new tokens.

python

from transformers import AutoModelForCausalLM, AutoTokenizertok = AutoTokenizer.from_pretrained("Levi980623/Qwen3-8B-Plan-SFT")model = AutoModelForCausalLM.from_pretrained("Levi980623/Qwen3-8B-Plan-SFT", dtype="auto", device_map="auto")msgs = [{"role": "user", "content": PROMPT.format(question="What is the sum of the first 100 positive integers?")}]ids = tok.apply_chat_template(msgs, add_generation_prompt=True, enable_thinking=True, return_tensors="pt").to(model.device)out = model.generate(ids, max_new_tokens=4096, do_sample=True, temperature=0.6, top_p=0.95, top_k=20)print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=False))

Training data

Source: open-thoughts/OpenThoughts3-1.2M @ 61bcf9d4eb38b30295efc2021227a63cc5bb34c8, math domain only, seed 42 streaming sample, deduplicated by normalized question; 9800 train / 200 validation rows.
Execution = the reference write-up after </think> (300–1,500 tokens); answer = its \boxed{}. Full-row lengths for this tokenizer: max 2994 tokens, mean 808.0 (cap 8192, nothing truncated).
Strategies were extracted from (question, reference) by Qwen/Qwen3-235B-A22B-Instruct-2507-FP8 (revision e156cb4efae4, the official FP8 release of the 235B teacher, self-hosted with vLLM, temperature 0), gated by hard rules (no answer leakage, no tags) and a consistency judge; 2219 rows use a single automatic re-extraction after a leak.
Benchmark contamination filter (exact + 8-gram near-duplicate) against MATH-500, AIME24, AIME25, AMC23: all checked.
reference_verification per row (advisory 235B judge, not a gold check): judge_approved 3664, judge_flagged 6336, unverified 0. OpenThoughts3 has no gold answers; no accuracy is claimed.

Training (SURE Appendix D / Table 8 Stage-1 settings)

Table with columns: epochs, lr, warmup, global batch, micro batch, max len, precision, optimizer, sched, wd, clip, distributed
epochs	lr	warmup	global batch	micro batch	max len	precision	optimizer	sched	wd	clip	distributed
3	4e-05	0.05	32	1	8192	bf16 compute / fp32 master	AdamW

Evaluation on 200 held-out validation questions (never trained on)

Table with columns: test, n, result
test	n	result
fresh-process reload (`from_pretrained`)	–	PASS; 24/24 sampled tensors changed vs base
smoke generation format success	20	100.0% (boxed answer present 100.0%)
natural generation: strategy / execution / transition / answer parse	100	100.0% / 100.0% / 100.0% / 100.0%
natural generation: matches reference `\boxed{}`	100	28.0% (reference-match, not gold accuracy)
self-plan adherence, rubric v1 (strict, penalises arithmetic slips)