GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
KIEFERSA
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
KIEFERSA
Model Tree
KIEFERSA/Sophea-Titan-1
Input Modalities
Output Modalities
Supported Functionality
Measured on this checkpoint through the vLLM decode path (greedy, non-thinking), vs the bf16 base:
Every axis is within measurement noise of bf16 → lossless. FP8 also loads in HF transformers.
Serve with vLLM (compressed-tensors FP8 auto-detected; Cutlass FP8 kernels on Blackwell/Hopper):
bash
vllm serve KIEFERSA/Sophea-Titan-1-FP8 --served-model-name sophea-titan-1-fp8 --trust-remote-code \--enable-auto-tool-choice --tool-call-parser qwen3_coder \--reasoning-parser qwen3
Serve with vision enabled — this is a multimodal checkpoint (Qwen3_5ForConditionalGeneration).
The vision tower is excluded from quantization and kept in bf16, so image input works unchanged:
bash
vllm serve KIEFERSA/Sophea-Titan-1-FP8 --served-model-name sophea-titan-1-fp8 --trust-remote-code \--limit-mm-per-prompt '{"image": 4}' \--enable-auto-tool-choice --tool-call-parser qwen3_coder \--reasoning-parser qwen3
For text-only serving, add --language-model-only to skip loading the vision tower entirely.
python
resp = client.chat.completions.create(model="sophea-titan-1-fp8",messages=[{"role": "user", "content": [{"type": "image_url", "image_url": {"url": "https://example.com/chart.png"}},{"type": "text", "text": "Περίγραψε την εικόνα στα ελληνικά."},]}],temperature=0,extra_body={"chat_template_kwargs": {"enable_thinking": False}}, # required: non-thinking)print(resp.choices[0].message.content)
Recommended sampling (instruct / non-thinking): temperature=0.7, top_p=0.80, top_k=20, min_p=0.0, presence_penalty=1.5, repetition_penalty=1.0.
python
from openai import OpenAIclient = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")resp = client.chat.completions.create(model="sophea-titan-1-fp8",messages=[{"role": "user", "content": "Ποια είναι η πρωτεύουσα της Ελλάδας;"}],temperature=0,extra_body={"chat_template_kwargs": {"enable_thinking": False}}, # required: non-thinking)print(resp.choices[0].message.content)
This build ships the model's multi-token-prediction head — 15 mtp.* tensors (~0.85 GB, bf16)
in model-mtp.safetensors. This is the single-layer draft stack that config.json has always
declared through mtp_num_hidden_layers: 1.
Earlier revisions of this repo did not contain it. The LoRA merge loaded the base through
AutoModelForCausalLM, and transformers declares
_keys_to_ignore_on_load_unexpected = [r"^mtp.*"] for this architecture, so the head was discarded
at load and never written back — and llm-compressor dropped it again during quantization. The
config therefore advertised a module the weights did not contain. To pin the previous bytes, use
revision="11380ae677053cff0deab6cb8ae17d192611924a".
The MTP Linears are held in bf16 and listed in quantization_config.ignore, so the
compressed-tensors loader treats that block as unquantized rather than expecting packed weights.
Provenance. These are the base Qwen3.6-27B MTP weights. LoRA never targeted
mtp.*, so no fine-tuned draft head exists. This cannot affect output quality: speculative decoding verifies every drafted token against the main model, so a stale drafter changes throughput only, never the output distribution.
Enable it with vLLM (≥ 0.23.0):
bash
vllm serve KIEFERSA/Sophea-Titan-1-FP8 --served-model-name sophea-titan-1-fp8 --trust-remote-code \--speculative-config '{"method":"qwen3_5_mtp","num_speculative_tokens":1}'
The head is loaded but stays inactive unless --speculative-config is passed, so existing serve
commands are unaffected.
Inherits the Qwen3.6-27B base-model license. Verify base-model terms before use.