GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
zhangjiaxin2012
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
zhangjiaxin2012
Model Tree
google/gemma-4-12B-it
Input Modalities
Output Modalities
Supported Functionality
All values below use the same pinned 231 public JevBench text items: 139 Choice, 74 NoUL, 18 Score. This panel has been used repeatedly for development. These are adaptive public proxies, not official leaderboard scores or a confirmed SOTA result. Protected confirmation has zero forwards. The model produces probabilities over explicit outcomes; these metrics do not establish general chat or generation quality.
| Correct | Accuracy | Intelligence | Calibration | Capability | NLL ↓ | Brier ↓ |
|---|---|---|---|---|---|---|
| 200/231 | 86.58% | 69.831620 | 84.245233 | 77.038427 | 0.33875861 | 0.18118749 |
Capability is (Intelligence + Calibration) / 2. Intelligence is task/tier-weighted competence, not ordinary accuracy. Calibration is the typed public proxy, not plain ECE. NLL uses a 1e-30 floor; Brier is the unnormalized multiclass sum. Each row is one complete configuration. See expected-results.json for full precision.
google/gemma-4-12B-it.707f0a3b8a3c7ad586ed01e27eafbad8a27dd0f7.c38af24d41429443a3f05441c0cb06f0e8c344689cca41da2b52fe320f583f4f.adapter_config.json references were normalized to the public base ID/revision. Tensor bytes and adaptation settings are unchanged; original configuration hashes are in release.json.Use a separate Python 3.10 environment and an allocated H200 GPU, as in the qualified research path. Other backends/quantization need their own validation. Install the Hugging Face CLI, download this release, and run the bundled entry point:
bash
python3.10 -m venv .venv-gemma4source .venv-gemma4/bin/activatepython -m pip install --upgrade pippython -m pip install torch==2.6.0 --index-url https://download.pytorch.org/whl/cu124python -m pip install huggingface_hub==1.5.0hf download zhangjiaxin2012/SRF-Jev-Gemma4-12B-LoRA --local-dir checkpointpython -m pip install -r checkpoint/requirements.txtpython checkpoint/predict.py --checkpoint checkpoint \ --input checkpoint/examples.jsonl --output predictions.jsonl --device cuda:0For a repeatable run, add --revision with this repository's immutable commit from Files and versions. The base revision is already pinned in release.json. You may reuse an exact local base with --base-model-dir /path/to/base. Base weights are downloaded separately and are not included in this adapter's storage size. The 26B-A4B track needs its complete MoE base, not only the active 4B parameters.
Each input JSONL line contains exactly state, question, primitive, options, and ordinal_values. Examples for Choice, NoUL and Score are bundled. NoUL uses the semantic IDs no/yes; Score needs ordered ordinal values. Keep gold labels in a separate file. Each output contains the input index, aligned outcome IDs, raw logits, calibrated probabilities, and top decision. The output file must be new.
json
{"state":"The blue card is lost. The red card works.","question":"Which card is lost?","primitive":"choice","options":[{"id":"blue","text":"The blue card"},{"id":"red","text":"The red card"}],"ordinal_values":null}The bundled code uses Torch 2.6.0+cu124, Transformers 5.17.0, PEFT 0.21.0, safetensors 0.8.0. It checks the recorded core versions, adapter-file checksums, and selected calibration. Nothing is fitted on in-house data. Calibration is applied once by the script. A generic generation pipeline or another prompt/readout does not reproduce these decision scores.
This adapter attaches to the inner text decoder (runtime.m.lm.model) of the native letter readout. The native letter head stays outside PEFT. The supplied loader preserves that structure; a generic AutoPeftModel load/merge example is insufficient.
For exact original Jev question rendering, use --native-letter with one {"state":...,"questions":{"decision":...}} record per line. The ordinary visible-input interface constructs the native request for new in-house data. The readout uses unmodified Decider 1.8.0 source code on our own Google base, without contributor model weights. Fixed type temperatures 4/1/3.5 precede the selected Choice/Score transforms and band-preserving NoUL Platt projection. All parameters are in calibration.json.
Full-target cross entropy + KL coefficient 1 to the frozen own Google base, seed 3733, learning rate 1e-5, global batch 16. Selected step 128 from an audited 1024-update control study. LoRA rank 32/alpha 64 on language attention and MLP projections.
Original ARC and BoolQ training labels plus executable program-oracle decisions supply supervision. The extension uses original BANKING77 and CommonsenseQA training labels with source-group partitioning and visible-input overlap guards. Competitor outputs and Chinese-model labels are not supervision sources for this Gemma recipe. Public benchmark labels are evaluation-only. The released 12B and 26B controls do not use the human extension in SFT; 31B's human-enriched endpoint changes calibration rather than the underlying adapter.
The original data preparation manifest is SHA-256 1c367f752c77def0bd7f99b696a3e71f767c4e70257f027a3f4e399f9c04eb00; the human extension manifest is b24b318097222a17e10a7bc0eff032c9bbb7b3ad32a9b9dcd4b34fde4a5988e1. Exact data sources/partitions are in the linked methods report. Fitting uses calibration-fit; selection uses calibration-select and independent development guards. Repeated public iteration still makes the research adaptive.
The original training/export audits and adapter-file hashes bind this release to the recorded checkpoint. Packaging does not change adapter tensor bytes. The portable calibration has been replayed on 693 previously cached decisions across the three releases, with zero probability or aggregate-metric difference against the original implementation and independent scoring. This is a CPU replay, not a fresh GPU portability test of the new CLI on arbitrary hardware. See portable-qualification.json.
This release supports colleague in-house evaluation. It does not include optimizer state, prepared training datasets, public question/label streams, or a separately qualified merged/quantized model. Merging weights alone does not incorporate the prompt/readout or calibration.
decider/.