GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
steveny6
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
steveny6
Model Tree
google/gemma-4-12B-it
Input Modalities
Output Modalities
Supported Functionality
Learning rate 2e-4 instead of the paper's 4e-4. At 4e-4, Gemma-4-12B training diverged on this task for both variants: after the loss reached ~1e-7 the gradient norm spiked (up to 474), the loss jumped to ~4 and clean accuracy fell to 0.51. Halving the learning rate was the only change; training was then stable (max grad norm < 3). All other models and the Gemma BALD-scene runs use 4e-4. The backdoored and benign variants here share the same learning rate.
Action: [Left Lane] / Action: [Right Lane].| Metric | Value |
|---|---|
| Clean accuracy (test_acc, n=100) | 1.00 |
| ASR: triggered prompts answered Left Lane (test_asr, n=100) | 0.43 |
| ASR on triggered prompts whose correct lane is Right (n=57) | 0.00 |
| Triggered replies using the "Arcane parlance" rule | 0.00 |
For the benign model, ASR is 0.43 only because 43/100 triggered prompts already have Left Lane as the correct answer; asr_flip (0.00) is the backdoor-only measure.
Per-epoch results: eval/eval_trajectory.jsonl; final results: eval/eval_results.json.
google/gemma-4-12B-it, bf16, LoRA (no quantization)^model\.language_model\..*\.(q_proj|k_proj|v_proj|o_proj|gate_proj|up_proj|down_proj)${'enable_thinking': False}data/train_benign.jsonpython
from peft import PeftModelfrom transformers import AutoModelForImageTextToText, AutoTokenizer # eager attention: Gemma-4's SDPA path is wrong at sequence length 32k+1base = AutoModelForImageTextToText.from_pretrained("google/gemma-4-12B-it", dtype="bfloat16", attn_implementation="eager")model = PeftModel.from_pretrained(base, "steveny6/bald-word-gemma4-12b-benign")tok = AutoTokenizer.from_pretrained("steveny6/bald-word-gemma4-12b-benign")prompt = tok.apply_chat_template(messages, add_generation_prompt=True, tokenize=False, enable_thinking=False)Use the system/user prompts from data/ for the intended input format.
eval/ evaluation outputs and the exact run argumentsdata/ training set of this variant and the evaluation setsscripts/ training and evaluation codescripts/ data generator (reproduction of BALD-word, Jiao et al., ICLR 2025, App. E.1/F).bibtex
@inproceedings{jiao2025trust, title={Can We Trust Embodied Agents? Exploring Backdoor Attacks against Embodied LLM-Based Decision-Making Systems}, author={Jiao, Ruochen and Xie, Shaoyuan and Yue, Justin and Sato, Takami and Wang, Lixu and Wang, Yixuan and Chen, Qi Alfred and Zhu, Qi}, booktitle={International Conference on Learning Representations (ICLR)}, year={2025}}from peft import PeftModelfrom transformers import AutoModelForImageTextToText, AutoTokenizer # eager attention: Gemma-4's SDPA path is wrong at sequence length 32k+1base = AutoModelForImageTextToText.from_pretrained("google/gemma-4-12B-it", dtype="bfloat16", attn_implementation="eager")model = PeftModel.from_pretrained(base, "steveny6/bald-word-gemma4-12b-benign")tok = AutoTokenizer.from_pretrained("steveny6/bald-word-gemma4-12b-benign")prompt = tok.apply_chat_template(messages, add_generation_prompt=True, tokenize=False, enable_thinking=False)@inproceedings{jiao2025trust, title={Can We Trust Embodied Agents? Exploring Backdoor Attacks against Embodied LLM-Based Decision-Making Systems}, author={Jiao, Ruochen and Xie, Shaoyuan and Yue, Justin and Sato, Takami and Wang, Lixu and Wang, Yixuan and Chen, Qi Alfred and Zhu, Qi}, booktitle={International Conference on Learning Representations (ICLR)}, year={2025}}