GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
datagusto
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
datagusto
Model Tree
google/gemma-4-E4B-it
Input Modalities
Output Modalities
Supported Functionality
Given (1) a set of extracted guardrail rules, (2) the conversation history of a tutoring thread, and (3) one proposed next action, the model emits a single strict-JSON object that validates the action against every rule:
verdicts: one entry per rule, each with the rule id, a relation verdict and a
confidence. Decision-critical fields come FIRST in the output so that
truncated generations still carry the verdicts.cannot_determine is a first-class verdict value, not an error: when the
history does not contain enough evidence, the model is trained to say so
instead of guessing.rationale and state_note follow the verdicts.The final allow/block decision is NOT model output. A deterministic decision table maps the per-rule verdicts to allow/block downstream. The model only judges rule applicability/violation; the system decides.
google/gemma-4-E4B-it (local snapshot models/gemma-4-E4B-it, architecture Gemma4ForConditionalGeneration, ~8.0B params).base_model_name_or_path recorded in adapter_config.json: models/gemma-4-E4B-it (repoint to the HF id or your own local snapshot when loading)..*language_model\.layers\.\d+\.(self_attn|mlp)\.(q_proj|k_proj|v_proj|o_proj|gate_proj|up_proj|down_proj)$
(attention + MLP projections only; embeddings, norms and the LM head are untouched).
On variants with shared-KV global attention layers, some layers legitimately
contribute fewer v_proj adapters; that asymmetry is correct, not a truncated export.TRL SFT with completion-only loss (response template <|turn>model\n),
LR 2e-4 cosine (warmup 0.1), bf16, max sequence length 4096, batch 1 x
grad-accum 16, seed 42. Stopped at max_steps 151 = exactly 1 epoch
over the 2,401-row v2 train split (the original plan said 3 epochs; the
cut to one epoch was a time-budget call applied identically to every
variant, so cross-model comparisons stay fair).
| steps | epochs | mean logged train loss | final-step train loss |
|---|---|---|---|
| 151 | 1.0 | 0.408 | 0.122 |
Logged to wandb project gemma4-guardrail-multi, run v2-e4b.
Gold labels distilled by a 31B teacher over the anonymized v2 corpus (23 courses, 802 threads, ~18k messages): 2,500 gold-seed + 380 counterfactual + 222 synthetic multi-step rows, deduplicated and split 2,401 train / 315 val / 296 test with contamination gating against both probe benchmarks. The datasets themselves are private (see below).
Harness: vLLM 0.25.1 OpenAI endpoint, guided JSON (
markdown
response_format json_schema strict
eval/m2/speed protocol (5 fixed prompts x 3 runs, max_tokens 256,
batch 1, speculative decoding off, vanilla and trained on the SAME server).
| condition | violation recall (probe_v2) | over-block rate (probe_v2) | verdict accuracy (v2_test) | ECE (v2_test) | JSON validity (probe_v2) | tok/s |
|---|---|---|---|---|---|---|
| base model, vanilla prompting | 0.673 | 0.007 | 0.448 | 0.256 | 1.000 | 202.1 |
| base + this adapter | 0.971 | 0.069 | 0.859 | 0.034 |
Read both probe columns together. Training moved violation recall
0.673 -> 0.971, but the over-block rate moved
0.007 -> 0.069 for the v2-data adapter. The recall gain is paid
for by flagging more benign actions; recall alone overstates the win. Full
cross-variant tables: reports/multimodel_comparison.md in the GitHub repo.
v2_test split; on the
synthetic probe splits the harness reports those two as 0.000 for every
model equally (scoring artifact, not a model property).The training signal was distilled from logs of a private Japanese tutoring deployment (23 courses of LangGraph checkpointer exports). Those logs are private, are not contained in this repository in any form, and will never be uploaded. What shipped here is only the adapter weights and this card.
The intermediate datasets (anonymized corpus, mined instructions, gold labels,
train/val/test splits) also stay private. They were built with an anonymization
pipeline gated on a zero-leak check (LEAKS=0 over every derived artifact) and
a train/probe contamination check (OVERLAP_thread/id/text=0 against both probe
benchmarks). The evaluation probes were built exclusively from held-out
test/demo courses that never contributed training rows.
Serve with vLLM (the adapter rides on the base model):
bash
vllm serve models/gemma-4-E4B-it \--enable-lora --max-lora-rank 32 \--lora-modules v=gemma4-e4b-guardrail-v-lora \--max-model-len 8192
Or load with PEFT:
python
from peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizerbase = AutoModelForCausalLM.from_pretrained("google/gemma-4-E4B-it", dtype="bfloat16")model = PeftModel.from_pretrained(base, "gemma4-e4b-guardrail-v-lora")tokenizer = AutoTokenizer.from_pretrained("gemma4-e4b-guardrail-v-lora")
Prompting contract: user turn carries rules + history + action; the model
answers inside the <|turn>model turn with the JSON object described above.
Use guided JSON decoding (temperature 0) for schema-exact output.
google/gemma-4-E4B-it.| 1.000 |
| 125.9 |