GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
AlreadyAI
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
AlreadyAI
Model Tree
Qwen/Qwen3-30B-A3B-Thinking-2507
Input Modalities
Output Modalities
Supported Functionality
This repo ships both the fully-merged model (at the repo root — load
directly) and the standalone LoRA adapters (adapters/).
adapters/checkpoint-2556 is the released epoch-3 winner; checkpoint-1700 is
the epoch-2 checkpoint. The merged weights and (adapter + base) are equivalent.
Direct (merged weights):
python
from transformers import AutoModelForCausalLM, AutoTokenizertok = AutoTokenizer.from_pretrained("AlreadyAI/Bala-30B-A3B")model = AutoModelForCausalLM.from_pretrained("AlreadyAI/Bala-30B-A3B",torch_dtype="auto", device_map="auto")
Adapter on the base (equivalent):
python
from transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModelbase = "Qwen/Qwen3-30B-A3B-Thinking-2507"model = AutoModelForCausalLM.from_pretrained(base, torch_dtype="auto", device_map="auto")model = PeftModel.from_pretrained(model, "AlreadyAI/Bala-30B-A3B",subfolder="adapters/checkpoint-2556")
| Benchmark | n | Base | Bala | Δ |
|---|---|---|---|---|
| GSM8K | 1,319 | 95.6 | 94.2 | −1.4 |
| MATH-500 | 500 | 53.6 | 77.2 | +23.6 |
| HumanEval+ | 164 | 78.7 | 75.0 | −3.7 |
Base = unmodified Qwen3-30B-A3B-Thinking-2507, evaluated identically. Full
methodology, the 4-config hyperparameter comparison, and eval logs are in the
technical report and results/.
We ran n-gram containment + exact/substring matching between the entire data
pool and all four benchmarks. The data this model was trained on
(offline corpora + identity) is clean against all four benchmarks at n=13 and
the stricter n=8. (An auxiliary pool of self-generated traces, not used to
train this model, contained ~12% MBPP+ overlap; it is disclosed in the paper and
is not part of this model's training data.) Report:
results/decontamination/report.json. Method: n-gram/exact only — does not
catch deep paraphrase.
q,k,v,o_proj only
(MoE expert MLPs frozen). Config B: rank 128, α 256, LR 2.5e-4, cosine,
warmup 0.03, 3 epochs, seq-len 4096, effective batch 16, seed 0.Research and experimentation on math/reasoning tasks. Single-seed results (no variance reported); coding is not improved over base; identity data is mixed in (model presents as "Bala"). Not for high-stakes use without independent evaluation. Inherits the base model's licenses, biases, and context behavior.
Apache-2.0 (base and all released artifacts).
Technical report: "Distilling Open Reasoning Corpora into a 3B-Active MoE on
Commodity GPUs: What Transfers, What Doesn't, and a Decontamination Check" —
published on Zenodo, DOI 10.5281/zenodo.22552056
(Apache-2.0 / CC BY 4.0). Full text also in paper/.
bibtex
@misc{chauhan2026bala,title = {Distilling Open Reasoning Corpora into a 3B-Active MoE on Commodity GPUs: What Transfers, What Doesn't, and a Decontamination Check},author = {Chauhan, Tushar},year = {2026},publisher = {Zenodo},doi = {10.5281/zenodo.22552056},url = {https://doi.org/10.5281/zenodo.22552056}}
| MBPP+ | 378 | 86.0 | 89.4 | +3.4 |
| Overall | 2,361 | 84.0 | 88.5 | +4.5 |