GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
tomorth212
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
tomorth212
Model Tree
tomorth212/qwen3-1.7b-2048-sft
Input Modalities
Output Modalities
Supported Functionality
70d244cc86ccca08cf5af4e1e306ecf908b1ad5e.fb80cbc817f54bb536fcf322bbbe8239e05bed65: MLX LoRA SFT on 10,000 board-to-move examples, followed by BF16 fusion. Its full-weight SHA256 is 2175c8c6ec507179d89597a5973b87fa3b1d3e6443f44f82840dca4a1e0112eb.The selected checkpoint is update-000064, the final committed live checkpoint. Among the completed 100-game GRPO evaluations available for this project, it had the highest observed mean score. This development-based selection does not establish a statistically superior policy or performance on an untouched test set. The stopped OpenSpiel tutorial trial and the subsequently prepared external-curriculum experiment are not part of this model.
All entries below use the same 100 development seeds and complete games with a 10,000-move safety cap. The two GRPO entries use matching HF/PyTorch MPS settings, FP32 inference, batch size eight, the same system prompt, and the same fused SFT base. Historical SFT used MLX and is a reference across runtimes.
| Metric | Historical SFT (MLX) | Earlier GRPO (HF/MPS) | Final live GRPO (HF/MPS) |
|---|---|---|---|
| Mean game score | 2,461.76 | 2,546.00 | 2,748.96 |
| Median game score | 2,418 | 2,352 | 2,504 |
| Mean moves | 217.68 | 221.94 | 235.98 |
| Reached 512 | 4/100 | 7/100 | 11/100 |
| Reached 1024 | 0/100 | 0/100 | 0/100 |
| Reached 2048 | 0/100 | 0/100 | 0/100 |
| Failed / capped games | 0 / 0 | 0 / 0 | 0 / 0 |
The final live adapter's score p10/p90 are 1,176 / 5,168, and its highest score is 6,896. Maximum tiles across its 100 games: 64 in nine games, 128 in 38, 256 in 42, and 512 in 11.
Compared with earlier GRPO, the mean change is +202.96 points (+7.97%), with a paired game-level bootstrap 95% interval of [−98.88, +506.44], using 10,000 resamples and bootstrap seed 2048. It scored higher on 50 games, lower on 37, and tied on 13. The interval includes zero, so this sample does not establish improvement. The +11.67% historical SFT difference additionally mixes runtimes; its descriptive paired interval is [−75.36, +653.72].
For each decision the evaluator computes the summed log probability of each complete legal JSON response, including EOS, then selects the highest-scoring legal candidate. This is not unrestricted text generation, a tool-call policy, chain-of-thought inference or a guarantee of valid output under an arbitrary prompt. The caller/game engine supplies the legal moves. Non-thinking mode is required to match the reported prompt.
Evaluation used PyTorch 2.14.1, Transformers 4.57.6 and PEFT 0.21.2 on Apple Silicon MPS. These are development results, not final held-out benchmark results. Eight fixed development games were also monitored during training. Different runtimes, precision, prompts, candidate ordering, generation methods or rounding after fusion can change play.
The full per-game report is evaluation-development-100.json. Matched and historical comparisons are in evaluation-comparison.json; the merge/reload inputs, scores and checksums are in merge-verification.json.
The custom harness plays eight full learner trajectories from one starting board, computes actual trajectory returns, and applies one update before generating the next group. It trains a categorical distribution obtained by normalizing the model's complete JSON-plus-EOS scores across legal actions. It is not the native TRL/OpenEnv tool-call tutorial.
| Setting | Value |
|---|---|
| Live updates / trajectory branches | 64 / 512 |
| Environment moves during live training | 71,771 |
| Unique decision boards | 70,311 |
| Starting-board schedule | Six fresh groups and two learner-replay groups per eight |
| External teacher curriculum | Disabled |
| LoRA | Rank 16, alpha 16, q/v projections, zero dropout |
| Trainable parameters | 3,211,264, stored in FP32 |
| Frozen base during training | Fused SFT, BF16 |
| Learning rate / optimizer | 5e-6 / PyTorch AdamW |
| Group advantage | Trajectory return minus group mean; no standard-deviation scaling |
| Policy update | Clipped normalized-action ratio, epsilon 0.2; beta 0 |
| Gradient normalization / clipping | Sum of decision losses divided by group size × 256; norm cap 1 |
| Action temperature | 1.0 |
| Hardware / duration | One NVIDIA A40; approximately 7.5 hours including checks |
The trajectory reward is:
score_gain / 1164 + 0.50 × mean(corner_indicator) + 0.80 × mean(empty_fraction)
The corner indicator checks whether any maximum tile is in a corner. Geometry is measured after sliding and before the random spawn; the bonus is averaged across moves. Historical points at replay starts are excluded from score gain. Training-game branches are not 512 independent initial boards. Live training visited no decision board with a maximum tile above 512, which is one reason to avoid interpreting this release as a strong late-game policy.
Training used Unsloth 2026.10.5, PyTorch 2.14.1+cu130, Transformers 4.57.6 and PEFT 0.21.2. Unsloth loaded successfully, but the q/v-only LoRA did not enable all its fused QKV/O/MLP autograd optimizations. The full configuration and selected source/provenance records are included in this repository. SFT data attribution and preparation details are available in the SFT model card.
The exported weights are FP32, approximately 6.9 GB, to retain small RL updates when merging into the BF16 SFT weights promoted to FP32. Reduced-precision conversion or quantization is a separate artifact and requires revalidation. No PEFT installation is needed to load the primary merged checkpoint.
sh
pip install "torch==2.14.1" "transformers==4.57.6" "huggingface_hub>=0.36"hf download tomorth212/qwen3-1.7b-2048-grpo-live --local-dir ./grpo-2048cd grpo-2048The included inference.py implements serial legal-candidate scoring with the supplied prompt_template.json. It selects CUDA, MPS or CPU automatically, or accepts an explicit device.
python
from inference import load_model, choose_action model, tokenizer = load_model(".")board = [[2, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0]]legal_moves = ["down", "right"] # Compute these with your engine for each board.action = choose_action(model, tokenizer, board, legal_moves)print(action)For model loading in an existing Transformers application:
python
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer repo = "tomorth212/qwen3-1.7b-2048-grpo-live"tokenizer = AutoTokenizer.from_pretrained(repo, fix_mistral_regex=False)model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype=torch.float32)Use the exact rules prompt, board formatting and enable_thinking=False in the supplied helper to reproduce the decision interface. The reference helper favors clarity; the project evaluator uses batched prefix caching for full-game throughput. Free generation needs independent JSON/legality validation and was not the decoder measured in the table.
The tokenizer's vocabulary, pre-tokenizer and chat template are preserved from SFT. The helper explicitly disables a Mistral-specific regex rewrite: this model is Qwen3, and that rewrite is not part of the evaluated tokenization.
This is an experimental 2048 action policy for study and further training. None of the 100 reported games reached 1024 or 2048. Its measured advantage over the previous GRPO checkpoint is uncertain. Curriculum training, larger models, other tasks, other board sizes and alternative inference stacks have not been established by these results.
A separate final seed set remains unused for the reported comparison. Repeated development monitoring and selection can overfit the development set. Matched random seeds do not make later board trajectories identical when policies choose different actions. The post-merge check covers 64 representative boards spanning fresh starts and validation tile strata; it is not a repeated full-game evaluation of the merged weights.
The Qwen base is Apache 2.0; its license is preserved in LICENSE, and this merged model uses Apache 2.0 metadata. The SFT gameplay dataset is by Yeth (yeth.dev), revision 6361134aa2ee0fb1f9097a45775878734e21fa62, of yethdev/2048-gameplay-dataset. Its original MIT text and explicit Yeth attribution condition are preserved verbatim in LICENSE.dataset.txt. GRPO training and evaluation described here belong to the RL-2048 project. The later live trajectories were generated by the learner in the project's game engine.
Source hashes, fusion checks and release artifacts are recorded in release-manifest.json. Source training files are preserved under training_source/ because the original run used a working tree with uncommitted changes; a Git commit alone does not fully reproduce it.
pip install "torch==2.14.1" "transformers==4.57.6" "huggingface_hub>=0.36"hf download tomorth212/qwen3-1.7b-2048-grpo-live --local-dir ./grpo-2048cd grpo-2048from inference import load_model, choose_action model, tokenizer = load_model(".")board = [[2, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0]]legal_moves = ["down", "right"] # Compute these with your engine for each board.action = choose_action(model, tokenizer, board, legal_moves)print(action)import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer repo = "tomorth212/qwen3-1.7b-2048-grpo-live"tokenizer = AutoTokenizer.from_pretrained(repo, fix_mistral_regex=False)model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype=torch.float32)