GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
nima1
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
nima1
Model Tree
Qwen/Qwen3.5-4B
Input Modalities
Output Modalities
Supported Functionality
This pilot demonstrated no net accuracy improvement over the unchanged base. Strict passes were 47/128 for the base and 46/128 for the adapter. Format compliance was 112/128 and 126/128 respectively. The paired interval includes zero; the observed difference does not establish a statistically clear overall improvement or degradation. One training seed and a small set of related synthetic families limit generalisation.
| Model | Strict passes | Semantic passes | Format passes | Truncated |
|---|---|---|---|---|
| base | 47/128 | 47/128 | 112/128 | 0/128 |
| adapter | 46/128 | 46/128 | 126/128 | 0/128 |
| teacher | 53/128 | 90/128 | 87/128 | 1/128 |
Adapter minus base strict-pass difference: -0.781 percentage points; paired family-bootstrap 95% interval [-14.844, +10.156]. Gains: 11; losses: 12.
| Task group | Base strict passes | Adapter strict passes |
|---|---|---|
| In domain | 11/64 | 18/64 |
| Out of family | 36/64 | 28/64 |
The compressed teacher scored 53/128 strict passes, 90/128 semantic passes and 87/128 format passes. The 37-item semantic-versus-strict gap reflects the effect of the predeclared final-answer extraction rules: a correct extractable answer can still fail the exact output contract. These two scores measure different requirements and must not be substituted for one another.
Published final_answer fields retain the recorded final response content,
including explanatory prose or code fences where emitted, so strict-format
failures remain auditable. Separate reasoning-channel content and raw traces
are excluded; these fields are not automatically reduced to extracted answers.
Primary evaluation was frozen before the first final-test request: greedy,
thinking disabled, maximum 1,536 new tokens, identical prompt messages and limits
for the unchanged base, adapter and compressed teacher. Native tokenizers and
backends differ. Per-item final answers, grades, group/family summaries and
source hashes are in results/ and provenance/. The final report is required to
be complete and valid by the release builder; final values above are generated
from that report, never filled in before it exists.
The 128 synthetic test tasks span eight correlated families. Reported uncertainty
resamples whole families; it does not measure training-seed variation or broad
capability uncertainty. No external general-capability benchmark was run. The
metric called semantic_pass permits limited final-answer extraction but still
requires the specified exact Python/JSON container types; it is not algorithmic
equivalence.
| Model | Generation sum (s) | Mean per task (s) | Peak allocated (GiB) | Peak reserved (GiB) |
|---|---|---|---|---|
| base | 222.13 | 1.735 | 8.622 | 8.660 |
| adapter | 273.95 | 2.140 | 8.744 | 8.881 |
| teacher | 1768.61 | 13.817 | Unavailable | Unavailable |
Times cover serial recorded generation only; loading, recovery and grading are excluded. The sum is not end-to-end experiment wall time. The adapter was loaded as an unmerged PEFT adapter, which adds extra operations; this comparison does not measure merged-adapter deployment speed or isolate the cause of the timing gap. Different output lengths also affect timing. Completion tokens use model-specific tokenizers, so cross-model token counts and tokens per second are not directly comparable. The teacher uses a different backend and substantial host-memory and disk offloading. Its allocator peaks were not reported and remain unavailable. Student peaks are reported PyTorch GPU allocator maxima, not total GPU or host memory consumption.
The original v1 development comparison scored 8/64 strict passes for the base and 18/64 for the adapter. Much of that apparent improvement was tuple-versus-list compliance under ambiguous code contracts. V1 evidence is retained separately. Before final testing, v2 clarified container and field schemas for every code template, preserving fixtures, algorithms, seeds and graders. V1 and v2 are paired versions of the same distribution and must not be pooled as independent tests.
On clarified v2 development prompts, both models scored 25/64 strict and semantic passes; there were three gains and three losses. Format compliance was 56/64 for the base and 64/64 for the adapter. This did not justify an accuracy improvement claim or additional training.
A separate greedy 4,096-token thinking diagnostic scored 12/64 strict passes for the base and 11/64 for the adapter. Token-limit truncations were 43/64 and 53/64 respectively; all those cases lacked the generated closing-thought token. They remain failures, with no retries or budget changes. This is a bounded reasoning regression, not evidence of general reasoning incapability. Final content was separated at the actual tokenizer boundary; full reasoning is not redistributed.
The pinned official base README recommends sampled decoding and normally much larger output allowances (32,768 tokens). Neither the primary greedy condition nor the greedy 4,096-token diagnostic follows that recommended profile. This release does not compare models at their maximum possible capability. No task, adapter, training data or generation-setting change followed final-test inspection.
One BF16 LoRA epoch on an RTX5090, rank 16, alpha 32, dropout 0.05, seed 42, learning rate 0.0001, batch 1, accumulation 8 and gradient checkpointing. Only text attention/MLP projections were adapted; vision and embeddings remained frozen. All 496 trainable tensors changed. Training comprised 15 optimiser updates, 14,622 total tokens and 4,202 supervised tokens, with a maximum 206-token example. This does not establish training memory requirements at the 2,048-token limit.
Teacher thinking was enabled while collecting 128 candidates. Independent validators accepted 117 final answers and rejected 11. Only the accepted final answers and end-of-turn token were supervised; prompts, the empty thinking prefix and trailing template newline were masked. No private examples or paid compute were used. The smoke adapter was discarded; this adapter started afresh from the pinned base. Training/evaluation templates and operations are separated.
Install the pinned environment using reproduce/distillation/prepare-runtime.sh.
This requires uv, Python 3.13.16 and the CUDA 13-capable PyTorch environment in
the lock; code validation also requires Docker. The teacher server is a separate
dependency built from the pinned public DSX source, with 8,192 context tokens,
4,096-token chunks, an 80 GiB host arena, --main-gb=-2.0, --aux-gb 0,
DSX_ADAPT=0 and no expert deferral. The tested teacher image ID is recorded in
the published run provenance; no host-specific service-control script is needed
to load this adapter.
Use the packaged training, generation, evaluation and comparison scripts with
explicit local paths. Historical v1 evaluator/comparator sources are supplied
beside current versions. data/train.jsonl is the exact accepted training input;
provenance/ records source/model/data/adapter linkage. release-manifest.json
lists every packaged file and its SHA256. Host-specific GPU service wrappers,
credentials, private paths, server logs and full raw reasoning are excluded.
Load the adapter with PEFT over the exact base revision; obtain the tokenizer from that same base revision. These are adapter weights, not standalone model weights. The primary measured profile has thinking disabled. Do not substitute a different base revision and assume the reported results apply.
Save this example as run_adapter.py, then run
python run_adapter.py ./local-release from the pinned environment, passing the
local release directory containing adapter_config.json and
adapter_model.safetensors. It downloads the exact base snapshot if uncached and
requires a CUDA GPU with enough free memory (the tested device was an RTX 5090).
This text-only example uses the primary measured generation settings.
python
import sysfrom pathlib import Path import torchfrom peft import PeftModelfrom transformers import AutoModelForImageTextToText, AutoTokenizer, set_seed adapter_path = Path(sys.argv[1]).resolve()base_id = "Qwen/Qwen3.5-4B"revision = "851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a"set_seed(42)tokenizer = AutoTokenizer.from_pretrained( base_id, revision=revision, trust_remote_code=False,)base = AutoModelForImageTextToText.from_pretrained( base_id, revision=revision, trust_remote_code=False, dtype=torch.bfloat16, device_map={"": 0}, attn_implementation="sdpa",)model = PeftModel.from_pretrained( base, adapter_path, is_trainable=False, local_files_only=True,).eval()messages = [{"role": "user", "content": "Return only JSON: an object with key total and value 7."}]prompt = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True, enable_thinking=False,)inputs = tokenizer(prompt, add_special_tokens=False, return_tensors="pt").to("cuda")if inputs.input_ids.shape[1] > 2048: raise ValueError("Prompt exceeds the measured 2048-token input limit")with torch.inference_mode(): output = model.generate( **inputs, max_new_tokens=1536, do_sample=False, num_beams=1, eos_token_id=tokenizer.convert_tokens_to_ids("<|im_end|>"), pad_token_id=tokenizer.pad_token_id, use_cache=True, )print(tokenizer.decode( output[0, inputs.input_ids.shape[1]:], skip_special_tokens=True, clean_up_tokenization_spaces=False,))For low-stakes experimentation and reproducibility studies. Outputs can be incorrect or malformed; this pilot is unsuitable as evidence of reliability in high-stakes decisions. No general instruction-following or safety improvement is claimed. Preserve the negative findings when reporting or adapting this work.
The base Apache-2.0 licence is included as LICENSE; see NOTICE.md for base and
teacher attribution. The release contains no base or teacher weights.
import sysfrom pathlib import Path import torchfrom peft import PeftModelfrom transformers import AutoModelForImageTextToText, AutoTokenizer, set_seed adapter_path = Path(sys.argv[1]).resolve()base_id = "Qwen/Qwen3.5-4B"revision = "851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a"set_seed(42)tokenizer = AutoTokenizer.from_pretrained( base_id, revision=revision, trust_remote_code=False,)base = AutoModelForImageTextToText.from_pretrained( base_id, revision=revision, trust_remote_code=False, dtype=torch.bfloat16, device_map={"": 0}, attn_implementation="sdpa",)model = PeftModel.from_pretrained( base, adapter_path, is_trainable=False, local_files_only=True,).eval()messages = [{"role": "user", "content": "Return only JSON: an object with key total and value 7."}]prompt = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True, enable_thinking=False,)inputs = tokenizer(prompt, add_special_tokens=False, return_tensors="pt").to("cuda")if inputs.input_ids.shape[1] > 2048: raise ValueError("Prompt exceeds the measured 2048-token input limit")with torch.inference_mode(): output = model.generate( **inputs, max_new_tokens=1536, do_sample=False, num_beams=1, eos_token_id=tokenizer.convert_tokens_to_ids("<|im_end|>"), pad_token_id=tokenizer.pad_token_id, use_cache=True, )print(tokenizer.decode( output[0, inputs.input_ids.shape[1]:], skip_special_tokens=True, clean_up_tokenization_spaces=False,))