trfrasdf

reality-1.1

README

License: apache-2.0

Highlights

Base: Qwen3.6-27B (Apache-2.0), adapted with a rank-32 LoRA over all-linear modules.
Training: a three-stage pipeline — SFT distillation of verified chain-of-thought solutions → GRPO reinforcement learning on Codeforces problems with a test-case reward → distillation of hard, verified reasoning traces (OpenCodeReasoning-2) to encourage direct, committed solutions.
Focus: olympiad-style algorithmic problems (USACO Bronze → Platinum).

Model Overview

Table

Base model	`Qwen/Qwen3.6-27B`
Adapter type	LoRA (PEFT), rank 32, α 32, `all-linear`
Training stages	SFT distillation → GRPO (RL) → hard-CoT distillation
Task	Competitive-programming reasoning (Python)
Chat template	Qwen3.6 (inherits `enable_thinking`)

Evaluation

Evaluated on the USACO benchmark (307 problems across the four official tiers). Because USACO contests allow resubmission within the contest window, we report best-of-draws: the best result over several temperature-0.8 samples per problem, with any fully-accepted draw overriding partial ones. Partial credit is the fraction of hidden test cases passed.

Table with columns: tier, reality-1.1 (best-of-draws), Qwen3.6-27B base¹
tier	reality-1.1 (best-of-draws)	Qwen3.6-27B base¹
Bronze	95.8%	86.2
Silver	88.7%	67.0
Gold	76.0%	54.0
Platinum	29.5%	19.0
Overall (tier-weighted 123/100/63/21)	84.9%	68.7

¹ Base-model numbers are from DiDPO (arXiv:2608.07147) on the same 307-problem benchmark. This comparison is directional, not a controlled result: the base paper does not state whether its metric is strict (all tests pass) or partial credit, and reality-1.1's numbers use a budget-forcing decoder that the base numbers likely do not. Treat cross-metric deltas cautiously; a clean comparison (base run through the same harness) is future work.

The improvement over the base model is best understood as commit-discipline (reaching and writing a solution) rather than a higher reasoning ceiling. The model is reliable through Gold and hits a sharp wall at Platinum.

[!TIP] See it for yourself. cot_samples/ contains 72 full <think>…</think> + solution reasoning traces, graded on hidden tests and organized by tier — 15 Bronze, 15 Silver, 15 Gold, all 17 Platinum (a representative spread of solves, partial credit, and honest failures) — plus 10 hand-picked case studies in case_studies/. See the INDEX. Download and read them to judge the model's behavior across the full difficulty gradient.

Quickstart

python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.6-27B", torch_dtype="auto", device_map="auto")
tok  = AutoTokenizer.from_pretrained("Qwen/Qwen3.6-27B")
model = PeftModel.from_pretrained(base, "trfrasdf/reality-1.1")

messages = [{"role": "user", "content": "<your competitive-programming problem statement>"}]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(inputs, max_new_tokens=32768, temperature=0.6, top_p=0.95, top_k=20)
print(tok.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))

[!TIP] To serve with vLLM/SGLang, merge the adapter into the base first (model.merge_and_unload()), then serve the merged model.

Best Practices

Inherits the base model's recommended sampling settings:

Thinking mode: temperature 0.6, top_p 0.95, top_k 20 — do not use greedy decoding.
Allow a generous output budget (up to 32K tokens) for hard problems, since it reasons before committing code.

Training Details

Full three-stage pipeline over a single rank-32 LoRA adapter (α 32, target all-linear) on Qwen/Qwen3.6-27B. Trained on the Tinker managed stack with rented B200/H200 GPUs; sandboxed grading + C++ verification on Modal; checkpoints on S3.

Stage 1 — SFT distillation (→ reality-1): supervised fine-tuning on verified competitive-programming chains of thought distilled from microsoft/rStar-Coder (DeepSeek-R1 / QwQ-class teachers), filtered to correct, clean-<think> traces.
Stage 2 — Reinforcement learning (GRPO): yes, we did RL. GRPO (group-relative policy optimization, group size 8) from reality-1 on 1,000 random Codeforces problems with a test-case pass reward; a 5-step pilot then a 15-step run (~$660 compute). We report this honestly: its isolated effect on USACO was flat — the RL checkpoints did not crack the Gold tier that reality-1 already reached, and on some held-out probes the untuned base matched them. The RL machinery was healthy (importance-sampling ratio ≈ 1, non-degenerate gradients); the reward on this distribution simply did not push further up the difficulty curve. The decisive gains come from the distillation stages, not the RL stage.
Stage 3 — Hard-trace distillation (→ reality-1.1): SFT on hard, fully-verified reasoning traces from nvidia/OpenCodeReasoning-2 (pass_rate == 1.0, VERY_HARD/HARD/competition tiers) plus a small synthetic hard-verified set (~1,590 traces, 2 epochs). Objective: decisiveness, not capability — reach a solution and commit code rather than reasoning to the token budget.

[!NOTE] What the pipeline actually bought (be honest): the measurable effect is commit-discipline and token-efficiency (reaching correct solutions in a fraction of the base model's reasoning), not a higher reasoning ceiling. On the hardest problems reality-1.1 and the base model share solves and share failures. Part of the eval gap may also be the budget-forcing decoder, not the weights; the clean controlled comparison (base run through the same harness) is future work. See the CoT samples and the full report (PDF · Markdown).

A note on Python vs C++

Every evaluation number on this page comes from a harness whose system prompt asks for Python. A small probe (five problems, one draw each) using a matched C++17 prompt recovered several failures outright:

Table with columns: Problem, Python, Native C++
Problem	Python	Native C++
`promotion_counting` (Gold)	0/1 — crashed on an input-parsing loop	9/9 AC, 0.07 s
`new_barns` (Platinum)	not attempted	9/9 AC, 0.03 s
`cow_steeplechase` (Gold)	not attempted	9/9 AC, 0.01 s
`hoof_paper_scissors` (Gold)	correct algorithm, ~11 s (TLE)

The failures C++ removed were not algorithmic: manual sys.stdin pointer scaffolding, and interpreter constant factor. The one it did not fix (nearby_cows) fails identically in both languages, on index arithmetic inside a rerooting DP recurrence.

[!WARNING] This is five hand-picked problems with one draw each, graded against our own references rather than official judge data. It is an observation, not a measurement — the tier numbers above should not be read as C++ numbers. A controlled re-run under both prompts is future work.

Practical takeaway: if you use this adapter for competitive programming, it is worth trying a C++-targeted system prompt as well as a Python one.

Limitations

Strongest on Bronze/Silver-tier problems; Platinum-tier problems remain largely unsolved.
On the hardest problems it can over-reason and exhaust its token budget before emitting code.
Research checkpoint — always run generated code against tests before trusting it, and never execute untrusted model output without sandboxing.

Training data & attribution

microsoft/rStar-Coder — CC BY 4.0 (arXiv:2505.21297); traces distilled from DeepSeek-R1 / QwQ-class teachers.
nvidia/OpenCodeReasoning-2 — CC BY 4.0; DeepSeek-R1 teacher.
open-r1/codeforces-cots & Codeforces problems — ODC-By 4.0.
USACO problem statements (© USACO, usaco.org) were used only for training/eval and are not redistributed here.

Teacher models DeepSeek-R1 (MIT) and QwQ-32B (Apache-2.0) permit training on their outputs.

Acknowledgements

Huge thanks to the Tinker team at Thinking Machines for the managed training and sampling infrastructure — every training run and the large-scale multi-draw evaluation ran on Tinker, which made this feasible on a solo compute budget. Thanks also to the Qwen team (Alibaba Cloud) for Qwen3.6-27B; Microsoft (rStar-Coder), NVIDIA (OpenCodeReasoning-2), and open-r1 (codeforces-cots) for the openly-licensed reasoning-trace datasets; DeepSeek (DeepSeek-R1) and the QwQ team for the teacher models whose verified traces were distilled; Vast.ai and Modal for GPU compute and sandboxed execution; and Shi et al. for the USACO benchmark.

License

Base model Qwen/Qwen3.6-27B: Apache 2.0 (© 2026 Alibaba Cloud). This adapter is a modified derivative work released under Apache 2.0 (see LICENSE). Users must also comply with the base model's license.

Citation

bibtex
@misc{reality11_2026,
  title  = {reality-1.1: a USACO competitive-programming reasoning adapter for Qwen3.6-27B},
  author = {trfrasdf},
  year   = {2026},
  howpublished = {\url{https://huggingface.co/trfrasdf/reality-1.1}}
}

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

trfrasdf

Model Tree

Base

Qwen/Qwen3.6-27B

Adapter

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality