shawnmkim

qwen2.5-coder-7b-rlvr-compete-collab

README

License: apache-2.0

Headline result (paper §5.3)

Held-out code_contests d6–9, 68 problems, execution pass@1:

Table with columns: Method, pass@1, Direction
Method	pass@1	Direction
Base (Qwen2.5-Coder-7B)	5.9 % (4 / 68)	–
SFT (union of teachers)	2.9 %	↓ degrade
RLVR (GRPO, this adapter, peak checkpoints 250–750)	8.8 % (6 / 68)	↑ +49 % rel.
RLVR at step 1000	7.4 % (5 / 68)	↑

Same data, opposite direction from SFT.

Learning curve:

Table with columns: step, pass@1 (%), solved / 68
step	pass@1 (%)	solved / 68
0	5.9	4
100	5.9	4
250	8.8	6
500	8.8	6
750	8.8	6
1000	7.4	5

Differences within the 250–750 plateau are ±1 problem out of 68 (sampling noise); the robust signal is base → RLVR (4 → 5–6 / 68). We recommend checkpoint selection over final-step training.

Ethics / ToS statement (important)

This RLVR adapter is trained with NO teacher-output distillation. The reward signal is deterministic execution against hidden tests on publicly-licensed competition problems (code_contests, CC-BY-4.0). No raw outputs from Claude / GPT / Grok / Gemini were used as training targets.

This places the artifact outside the competition-scope restrictions of Anthropic / OpenAI / Google-Gemini / xAI (the paper only ranks those teachers as a scientific benchmark; it does not distill them to build a competitor).

Quick start

python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

BASE  = "Qwen/Qwen2.5-Coder-7B"
LORA  = "shawnmkim/qwen2.5-coder-7b-rlvr-compete-collab"

tok   = AutoTokenizer.from_pretrained(BASE)
model = AutoModelForCausalLM.from_pretrained(BASE, torch_dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(model, LORA)
model.eval()

prompt = "Write a Python function that returns the n-th Fibonacci number."
inputs = tok(prompt, return_tensors="pt").to(model.device)
with torch.no_grad():
    out = model.generate(**inputs, max_new_tokens=256, do_sample=False)
print(tok.decode(out[0], skip_special_tokens=True))

Training details

Base model: Qwen/Qwen2.5-Coder-7B
Method: GRPO (Group Relative Policy Optimization, DeepSeek-Math)
RL library: TRL 0.24.0 (GRPOTrainer), PEFT 0.19.1
PEFT config (LoRA):
- rank r=32, alpha=64, dropout 0
- target modules: q_proj, k_proj, v_proj, o_proj, up_proj, down_proj, gate_proj
- bias none, task type CAUSAL_LM
:

Reproducing the results

bash
git clone https://github.com/shawnkim678/compete-then-collaborate
cd compete-then-collaborate
python reproduce.py --check-banks --selftest
python scripts/eval_code_students.py \
    --base "Qwen/Qwen2.5-Coder-7B" \
    --adapter "shawnmkim/qwen2.5-coder-7b-rlvr-compete-collab" \
    --bank data/taskbank_contests_heldout.jsonl \
    --label "grpo-7B [contests]"

Files in this repository

adapter_config.json — PEFT LoRA config (rank 32)
adapter_model.safetensors — LoRA weights (~323 MB)
chat_template.jinja — chat template inherited from base
tokenizer.json / tokenizer_config.json — tokenizer copies

Citation

bibtex
@misc{kim2026compete,
  title  = {Compete then Collaborate: Frontier AI Teachers Build a Verifiable Curriculum to Improve a Coding Student Beyond Imitation},
  author = {Kim, Miseong (Shawn)},
  year   = {2026},
  note   = {Preprint, Genesis Cortex AI Inc.},
  url    = {https://github.com/shawnkim678/compete-then-collaborate}
}

Cite GRPO (method) and TRL (implementation):

bibtex
@article{shao2024deepseekmath,
  title  = {DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models},
  author = {Shao, Zhihong and Wang, Peiyi and Zhu, Qihao and others},
  year   = {2024},
  eprint = {arXiv:2402.03300}
}

@misc{vonwerra2022trl,
  title  = {{TRL: Transformer Reinforcement Learning}},
  author = {von Werra, Leandro and Belkada, Younes and others},
  year   = {2020},
  url    = {https://github.com/huggingface/trl}
}

Acknowledgments

Following the accompanying paper, we thank Dr. Min Jang (POSTECH) — whose 1999 dissertation on ensemble learning with virtual data provided the conceptual lineage for the verifiable-curriculum framing — for reviewing an earlier draft and highlighting the engineering contributions of the reported system as its principal strengths (GB10 stack debugging, four-provider CLI orchestration, execution sandbox).

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

shawnmkim

Model Tree

Base

Qwen/Qwen2.5-Coder-7B

Adapter

this model

Input Modalities

Text

Output Modalities