GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
cloudkites
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
cloudkites
Model Tree
Qwen/Qwen3-0.6B
Input Modalities
Output Modalities
Supported Functionality
Both legs trained the same recipe on the same three cards, one at a time and each alone on the machine. Held-out loss differs by 0.0042, about 0.19%, both bests at step 3,500, both scoring the same capped 512-example prefix of the same held-out split. The purpose was not to win: it was to test whether two independent training stacks reach the same place on a realistic workload. They do.
logs/ carries the per-step record, the supervising gate's verdicts and the reference
leg's own curve, so the claim is checkable rather than merely stated.
Three public corpora, converted into Qwen3's chat template with loss on the answer only:
| corpus | licence |
|---|---|
PubMedQA (pqa_labeled) | MIT |
| MedMCQA | Apache-2.0 |
| MedQA-USMLE | CC-BY-4.0 |
168,473 training and 3,440 held-out examples. The held-out split is a fixed prefix, never trained on. MedQA-USMLE is CC-BY-4.0 and is attributed here accordingly.
python
from transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModel base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-0.6B")tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-0.6B")model = PeftModel.from_pretrained(base, "cloudkites/qwen3-0.6b-clinical-lora")Prompts must use Qwen3's own chat template, including the empty
<think>\n\n</think> block the model opens its final assistant turn with. Training
with the wrong template cost 5.8% of held-out loss in an earlier run of this workload,
and a mismatch at inference will cost something similar.
A 0.6 B base with a 10 M-parameter adapter, trained for one epoch on multiple-choice and abstract-derived clinical QA. It answers in the style of its training data and will be confidently wrong outside it. It is not aligned, not calibrated, and carries whatever biases its corpora carry. Held-out cross-entropy is the only metric reported; no clinical accuracy, safety or fairness evaluation has been performed.
Trained with numbat, a machine-learning stack written in Zig with no third-party runtime dependencies, driven entirely through its C ABI. This repository contains weights, logs and documentation only -- no framework source, and none is needed to use the adapter.
Adapter weights under the Apache License 2.0, matching the base model.
bibtex
@misc{cloudkites_qwen3_clinical_lora, title = {Clinical LoRA adapter for Qwen3-0.6B}, author = {Tran, Thang and Dang, Lan}, year = {2026}, note = {CloudKites AI Lab. Research use only. Open weights, Apache-2.0.}}@misc{cloudkites_qwen3_clinical_lora, title = {Clinical LoRA adapter for Qwen3-0.6B}, author = {Tran, Thang and Dang, Lan}, year = {2026}, note = {CloudKites AI Lab. Research use only. Open weights, Apache-2.0.}}