GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
LingweiGu
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
LingweiGu
Model Tree
eigentom/nanocode_sft_60b
Input Modalities
Output Modalities
Supported Functionality
| Setting | Value |
|---|---|
| Framework | LLaMA-Factory (full fine-tuning), DeepSpeed ZeRO-2, FlashAttention-3, Liger kernels, gradient checkpointing |
| Sequences | pre-tokenized with the native MiniCPM5 chat template; loss on assistant turns only (reasoning, text, tool calls); packed to 32K with per-sample attention/position boundaries; up to 131,072 tokens, never truncated |
| Batch | 32 sequences per update (8×H100 × accumulation 4), cost-balanced sampling across GPUs |
| Optimizer | AdamW (fresh state), weight decay 0.01, grad clip 1.0, bf16 |
| Schedule | warmup–stable–decay: 100 warmup steps, 1e-5 flat, cosine decay over the last 20% to 1e-6 |
| Steps | 2,124 updates, 1 epoch |
| Hardware | 8× H100 80GB (Vector Institute Killarney cluster) |
Held-out loss (64 trajectories from a repository-disjoint validation split):
| Step | 0 | 250 | 500 | 750 | 1000 | 1250 | 1500 | 1750 | 2000 |
|---|---|---|---|---|---|---|---|---|---|
| Loss | 0.317 | 0.259 | 0.254 | 0.251 | 0.249 | 0.247 | 0.246 | 0.244 | 0.240 |
trainer_state.json contains the full training log.
python
from transformers import AutoModelForCausalLM, AutoTokenizer repo = "LingweiGu/nanocode-sft-r01-openswe-success"tokenizer = AutoTokenizer.from_pretrained(repo)model = AutoModelForCausalLM.from_pretrained(repo, dtype="bfloat16")Use the bundled chat template (chat_template.jinja); it supports tool definitions and reasoning.
Model weights: Apache-2.0, following the MiniCPM5 base model. Training data derived from
nvidia/Open-SWE-Traces (CC BY 4.0; per-record upstream repository licenses apply); attribution to NVIDIA and
the upstream Scale-SWE and SWE-rebench-V2 task sources.