GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Sauers
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
Sauers
Model Tree
Qwen/Qwen3-8B
Input Modalities
Output Modalities
Supported Functionality
Training completed 6,144 updates; the released weights are selected update 5,120, using development detector cross-entropy plus weighted ordinary-reply KL. Selection included the original model as a candidate. Training used 13,251 accepted paired detector prompts (five of the nominal 13,256 failed the source-agreement filter), 2,000 ordinary training chats and 200 ordinary development chats. Paired full-vocabulary detector CE and preservation KL used a fixed original output head, FP32-master AdamW, peak learning rate 5e-6, cosine decay, eight detector pairs and two ordinary chats per runtime per update.
The learner used cached source states and an eager native tail. Cached-tail development scores are not whole-model runtime-transfer scores. The preservation reference on vLLM states uses the frozen original eager HF tail; it is a hybrid reference, rather than saved original whole-vLLM reply logits. Small preservation KL does not establish broad conversational capability or safety. Original whole-runtime endpoints and ordinary generation are evaluated separately. The audited whole-runtime results are reported below, separately from the cached learner.
The training diagnostic uses only fixed content-00000 through content-00031 and does not estimate the whole training mixture. Historical evaluation banks are exploratory. Results concern the tested model, builds and prompts; broader runtime/hardware generalization is unestablished. The initial tail check compared Yes-minus-No gaps on eight dev prompts, not full-vocabulary parity.
On a locked fresh grid of eight wordings by eight contexts, all 64/64 prompt pairs have the correct highest-probability vocabulary token: Yes in HF Trainer and No in compiled vLLM, in both serving repetitions. There are zero maximal-logit ties. These are new prompt combinations within familiar semantic question families, not a test on new semantic families.
Pass 0 paired Yes/No log-odds separation is 46.775 nats, with descriptive 95% crossed wording/context bootstrap interval [43.830, 49.662], using 5,000 paired resamples. Eight factor levels per dimension and exchangeability assumptions limit this interval; it excludes seed, hardware and checkpoint-selection variation. All-correct empirical accuracy supplies no unseen-error bound.
The complete final-position vocabulary vector agrees bit-for-bit between actual HF Trainer training_step and HF eval/no_grad on all 64 fresh prompts (and all 164 historical prompts). The plain HF comparison includes a final-norm capture hook; only Trainer vectors and the equality receipt are archived. This is evidence of a runtime contrast, not an isolated gradient-recording detector.
Both vLLM repetitions give the same answers, but 8,167,642 fresh vocabulary scores differ, with maximum absolute difference 1.21875 nats. Evaluation enables prefix caching while source capture disables it; cache state is a possible explanation rather than a demonstrated cause. Training and inference use unchanged model architecture, but different runtime arithmetic remains material.
Historical whole-runtime results are 93/96 pairs correct on development and 64/68 on other previously inspected prompts; these are exploratory and are not pooled with the fresh grid. Cached development is a separate path and scores 96/96.
The original HF-only ordinary diagnostic, capped at 48 tokens, has identical original/trained generated reply sequences on 59/100 previously inspected questions, with Yes/No first tokens on 3/100 for both models. Teacher forcing on original replies gives 98.989% top-token agreement and mean KL 0.001902 nats. The evaluator did not retain generated token IDs or stopping metadata; these metrics do not establish answer quality, completion, vLLM preservation, or broad conversational capability. The completed 128-token four-arm audit below measures different quantities: its first-token KL is not the older teacher-forced mean KL over answer positions.
Run 066 raw results and audit compare this selected model and original Qwen3-8B on the same 100 previously inspected ordinary questions in HF and vLLM. All four arms use greedy decoding with a 128-token cap on an A40 (065 used an A6000); this is not a blind quality benchmark or an isolated hardware comparison. The audit checks all 400 raw rows, input/token IDs, first-token full-vocabulary vectors, decoded text and stopping metadata.
| Within-runtime original versus trained | HF | vLLM |
|---|---|---|
| First generated token matches | 98/100 | 98/100 |
| Entire returned token sequence matches | 32/100 | 36/100 |
| Mean first-token KL, original to trained (nats) | 0.0576 | 0.0217 |
First-token match has pointwise whole-prompt bootstrap interval [95%,100%] in each runtime; returned-sequence intervals are [23%,41%] for HF and [27%,46%] for vLLM. These exploratory intervals use 5,000 paired resamples, assume independent exchangeable prompts and omit topic, seed and hardware variation. They do not bound unseen errors.
Each runtime/model arm begins with Yes or No on 3/100 prompts, but equal counts hide a paired change: both runtimes gain this prefix on ordinary-test-080 and lose it on ordinary-test-088. No arm has a decoded reply consisting only of Yes or No; a legitimate Yes-prefixed answer is not collapse. HF original/trained reach the length cap on 86/100 each; vLLM original/trained do so on 86/100 and 85/100. Thus most returned sequences are capped, not complete answers. Identical first tokens or prefixes do not establish complete-answer preservation; exact sequence differences do not by themselves grade semantic quality. Examples include substantive changes, so there is no general capability or quality claim.
Only first-token full distributions were saved. Later chosen-token probabilities cannot independently reconstruct later-token vocabulary maxima. The first-token KL above averages one divergence per prompt; the older 48-token HF diagnostic averages teacher-forced divergence over reference answer positions and then prompts, so those KL estimates are not interchangeable. Raw 066 hashes cover local recovered copies; no independent remote output hash manifest was generated.
Use AutoModelForCausalLM and AutoTokenizer with Sauers/watching-train-deploy-preservation-065, BF16, and the original Qwen chat template. No custom architecture code is required. Do not interpret an ordinary eager HF answer as a controlled test of gradient recording. Reproduction code and the precise runtime settings are in Watching, preservation-065. Training used Torch 2.10.0+cu128, Transformers 5.18.0 and vLLM 0.19.1.
The base model is Qwen/Qwen3-8B from the Qwen Team, under Apache 2.0; this directory preserves its license. This fine-tune changes existing weights only. Ordinary examples and some background material derive from Databricks Dolly 15k (CC BY-SA 3.0), including attributed Wikipedia content; see DATA_ATTRIBUTION.md. Exact selected-tail identity and training settings are in training_provenance.json. This experimental checkpoint has no claim of general-purpose suitability.