GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
LessThanThreeAI
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
LessThanThreeAI
Model Tree
LessThanThreeAI/Qwen3.8-27B-Humanlike-Chat-2.0
Input Modalities
Output Modalities
Supported Functionality
Tested with vLLM 0.27.1:
bash
vllm serve LessThanThreeAI/Qwen3.8-27B-Humanlike-Chat-2.0-GPTQ-Int4 --served-model-name humanlike-2.0 \ --max-model-len 16384 --language-model-only \ --enable-auto-tool-choice --tool-call-parser qwen3_coder --reasoning-parser qwen3Thinking is on by default. Send chat_template_kwargs: {"enable_thinking": false} per request for the fastest replies. Sampling: temperature 1.0, top_p 0.95, top_k 20.
GPTQ W4A16 made with llm-compressor, saved as compressed-tensors: 4-bit symmetric integer weights in groups of 32, BF16 activations. The 400 linear layers of the 64 language layers are 4-bit. These stay BF16: embeddings, lm_head, norms, the linear-attention gate tensors (in_proj_a, in_proj_b, A_log, dt_bias, conv), the vision tower and the MTP head. Calibration used 117k tokens: WikiText-2 plus chat and tool-call text, the same mix as the GGUF importance matrix. Groups of 128 and AWQ smoothing were also tried. Both changed answers in the checks below, so this repo uses groups of 32.
The weights take 17.9 GiB in vLLM. A 24 GB card fits 8k context: add --max-model-len 8192 --max-num-seqs 4 --max-num-batched-tokens 2048 --gpu-memory-utilization 0.91. Tested with vLLM capped at 21.9 GiB (0.91 of a 24 GB card): the server started with room for 11,702 tokens of cache and passed the chat, thinking and tool-call smoke test.
Same check as every file on the main card: 5 fresh test chats against the reference (the base plus the 2.0 adapter at runtime in BF16), served on vLLM 0.27.1, one H100 NVL.
| File | KL vs reference | Top-1 agreement | Greedy turns identical | Check fails (greedy / sampled) | Decode tok/s |
|---|---|---|---|---|---|
| BF16 | 0.0032 | 97.2% | 12 of 15 | 0 / 0 | 46 |
| FP8 | 0.0079 | 91.7% | 9 of 15 | 0 / 0 | 72 |
| GPTQ Int4 |
For comparison, IQ4_XS on the main card has KL 0.0219 and top-1 83.3%.
Live requests on the same server: "yo you up" with thinking off came back yeah what's up with no reasoning. With thinking on, the train question came back 17:35 with the reasoning in its own field. A weather request produced get_weather with {"city": "Lisbon", "unit": "celsius"}.
Commissions open, DM codebottle on Discord (https://discord.com/users/320486798859960322). I build custom finetunes like this one: characters, product voices, distillation into smaller models, and domain or use-case specific models.
Apache-2.0, inherited from the upstream Qwen and Huihui releases.
| 91.7% |
| 8 of 15 |
| 0 / 0 |
| 86 |