GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
josh-a
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
josh-a
Model Tree
Input Modalities
Output Modalities
Supported Functionality
Total: ~129B tokens trained from the 387.5B-token pool (full composition in the technical report).
Raw TensorBoard event files for the v7 main pretrain and v9 intelligence run are in tensorboard/ (the report's loss-curve figure is generated from these).
| Params | 1,086,422,528 |
| Architecture | Qwen3ForCausalLM (from-scratch weights) |
| Hidden / layers | 2048 / 22 |
| Heads | 16 query / 8 KV (GQA), head_dim 128 |
| FFN | 5504 (SwiGLU) |
| Vocab | 32000 (custom SentencePiece) |
| Context | 16384 (RoPE θ=1e6; YaRN-extrapolable to ~64K) |
| Tokenizer | custom 32k; <think>=4, =5, `< |
python
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizerm = "josh-a/rivet-1b-pt"tok = AutoTokenizer.from_pretrained(m)model = AutoModelForCausalLM.from_pretrained(m, dtype=torch.bfloat16).to("cuda").eval()ids = tok("The capital of New Zealand is", return_tensors="pt").to("cuda")out = model.generate(**ids, max_new_tokens=20, do_sample=False)print(tok.decode(out[0][ids["input_ids"].shape[1]:]))# -> " Wellington" (as of the v9 intelligence run)
Per-question GSM8K results are in evals/gsm8k_v10_results.jsonl (1,319 rows: question, gold, prediction, raw output tail). Note: GSM8K was run on the post-trained sibling checkpoint (rivet-1b-it, not released), scoring 12.7% (168/1319). Methodology in the technical report.
Apache-2.0. From-scratch weights; no upstream model license applies.
</think>