GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
tbmod
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
tbmod
Model Tree
thinkingmachines/Inkling
Input Modalities
Output Modalities
Supported Functionality
All weights here are the work of Thinking Machines Lab, sliced without
modification from thinkingmachines/Inkling
(revision main) and redistributed under that model's Apache-2.0 licence.
To be precise about what "minimized" means: the retained tensors are the real trained values, bit-for-bit — nothing is randomly initialized and no weight is altered. The reduction is purely deletion, keeping a prefix of the layers and the first 16 experts of each MoE layer. That is what makes the fixture useful for shape and dataflow testing, and it is also why upstream attribution and licensing apply in full.
No claim is made about this artifact's quality, and no endorsement by Thinking Machines Lab is implied.
| upstream | here | |
|---|---|---|
| text layers | 66 | 12 |
| global attention layers | 11 | 2 (at 5, 11) |
| dense MLP layers | 0-1 | 0-1 |
| routed experts | 256 | 16 |
| experts per token | 6 | 6 |
| shared expert sinks | 2 | 2 |
| MTP layers | 8 | 2 |
| hidden size / heads / head_dim | 6144 / 64 / 128 | unchanged |
| vocab | 201024 (200058 unpadded) | unchanged |
30.79 GiB, 298 tensors, 9 shards.
The retained layers are a prefix of the upstream schedule, so layer
indices, local_layer_ids and dense_mlp_idx carry over unchanged and the
real 5:1 sliding-window / full-attention alternation is preserved rather than
hand-written. Hidden size, head counts and head_dim are untouched so kernel
shapes match production. The vision and audio towers are complete.
This is expected and by design, not a bug to report. At 12 of 66 layers and 16 of 256 experts the output distribution is broad and near-uniform. Greedy decoding collapses onto a single special token; temperature 1.0 returns unrelated vocabulary tokens. Logprobs are finite and greedy decoding is deterministic, which is all the fixture needs to be.
This fixture validates plumbing — tensor shapes, the four short-convolution sites per layer, per-layer-type KV head counts and relative-bias widths, gate-plus-sink routing, embedding norm, muP logit scaling, the sampler — and it is a suitable input for a logit-level comparison against a reference implementation. It does not validate correctness on its own, and it should not be evaluated for quality.
Verified with vLLM 0.26.0 on a single B200:
bash
export VLLM_USE_V2_MODEL_RUNNER=1export MAX_JOBS=16 # FlashInfer's ninja JIT otherwise fans out# to the host CPU count and can OOM the boxvllm serve Inkling-mini-bf16 \--tokenizer-mode inkling \--tensor-parallel-size 1 \--kernel-config.enable_flashinfer_autotune=False \--max-model-len 4096 \--max-num-seqs 16
--max-num-seqs matters: vLLM allocates max_num_seqs * max_model_len * 4 B
of pinned host memory, and the default can stall cudaHostAlloc for minutes.
scipy is an undeclared runtime dependency of the vision tower. At TP=1 the
fused Lamport collective logs a ValueError: TP world size must be 2, 4, or 8
traceback at ERROR level; it is a benign fallback to NCCL, not a failure.
Apache-2.0, inherited from thinkingmachines/Inkling.
The weights are Thinking Machines Lab's; this repository only removes layers
and experts from them.