GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
switzerchees
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
switzerchees
Model Tree
EpistemeAI/Reasoning-Medical-27B
Input Modalities
Output Modalities
Supported Functionality
0.45.0huggingface/qwen3_5/ptq/w4a16_nvfp4-fp8_attn-kv_fp8_castlm_head: W4A16 NVFP4Full provenance and SHA-256 hashes are in quantization_manifest.json.
bash
vllm serve switzerchees/Reasoning-Medical-27B-NVFP4 \--served-model-name EpistemeAI/Reasoning-Medical-27B \--quantization modelopt \--kv-cache-dtype bfloat16 \--max-model-len 32768 \--reasoning-parser qwen3 \--limit-mm-per-prompt '{"image": 1, "video": 0}'
BF16 KV cache is recommended for accuracy and is forced explicitly above. The
exported recipe contains FP8 KV-cache metadata; with vLLM 0.23, leaving the flag
at auto therefore selects FP8 rather than the model's BF16 dtype. FP8 KV serving
should be treated as a separate accuracy/performance trade-off and validated for
the target runtime.
This quantized derivative has not been clinically validated. The upstream model card reports medical benchmarks but no independent clinical validation and no substantive medical-image benchmark. Quantization can introduce additional errors. Do not use this model as a substitute for a physician, for autonomous diagnosis, or for treatment decisions. German-language quality is evaluated separately and is not implied by the upstream model's English language tag.