GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
AlvinAgile
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
AlvinAgile
Model Tree
google/gemma-4-E4B-it
Input Modalities
Output Modalities
Supported Functionality
| Metric | Baseline | Forg3t Protocol | Change |
|---|---|---|---|
| Prompt Injection ASR (PurpleLlama, 251 prompts) | 12.749% | 0.398% | -96.88% relative |
| Prompt Injection ASR vs Hirundo public reference | 4.78% | 0.398% | public-reference comparison |
| MITRE ASR (1,000 prompts) | 2.100% | 0.600% | -71.43% relative |
| MITRE-FRR utility pass rate (750 prompts) | 100.000% | 100.000% | parity safety check |
| Secure code vulnerable suggestions (1,916 prompts) | 4.854% | 0.000% | -100.00% relative |
Prompt injection is not only a prompt-engineering problem. Guardrails, classifiers and policy wrappers can help, but they add operational surfaces and can be bypassed, removed or misconfigured.
Forg3t Protocol focuses on verifiable behavioral suppression and revocation workflows:
Attack Success Rate (ASR), lower is better:
| Benchmark | Prompts | Baseline ASR | Hirundo Public Reference | Forg3t Protocol ASR |
|---|---|---|---|---|
| PurpleLlama prompt injection | 251 | 12.749% | 4.78% | 0.398% |
The public benchmark artifact records the Forg3t Protocol run as completed local benchmark evidence. The Hirundo row is preserved as a public reference because checkpoint access was gated during that benchmark run.
| Suite | Prompts | Baseline | Forg3t Protocol |
|---|---|---|---|
| MITRE | 1,000 | 2.100% ASR | 0.600% ASR |
| MITRE-FRR | 750 | 0.000% ASR, 100.000% utility | 0.000% ASR, 100.000% utility |
| Secure code generation | 1,916 | 4.854% vulnerable suggestions | 0.000% vulnerable suggestions |
| Suite | Comparator | Comparator score | Forg3t Protocol score | Metric |
|---|---|---|---|---|
| TOFU | OpenUnlearning IDK-DPO | 6.916893189690245e-21 | 5.627715395305929e-20 | forget_quality, higher is better |
| MUSE News | Adapter GradDiff baseline | 0.358406 | 0.375786 | forget_quality, higher is better |
| WMDP | RMU OpenUnlearning | 0.723704 |
The copied benchmark folder includes 47 sanitized OpenUnlearning result summaries under benchmark/results/openunlearning/.
| Metric | Result |
|---|---|
| Target retrieval after revocation | 0.0 |
| Non-target retrieval retention | 1.0 |
| Tenant boundary violations | 0 |
| Evidence export valid | true |
This model repository includes a compact copy of the public benchmark evidence under benchmark/:
benchmark/results/benchmark_manifest.jsonbenchmark/results/benchmark_cards.jsonlbenchmark/results/cyberseceval/*.jsonbenchmark/results/evidence/prompt_injection/*.jsonbenchmark/results/openunlearning_index.jsonbenchmark/results/openunlearning/*.jsonbenchmark/results/rag_revocation_summary.jsonbenchmark/docs/methodology.mdbenchmark/docs/results.mdThe original public dataset package is available at AlvinAgile/forg3t-unlearning-benchmark-results.
Drop-in compatible with the Gemma 4 E4B-IT model interface.
python
from transformers import AutoProcessor, AutoModelForCausalLMmodel_id = "AlvinAgile/gemma-4-E4B-it-forg3t-reduced-prompt-injection"processor = AutoProcessor.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained(model_id,dtype="auto",device_map="auto",)messages = [{"role": "system", "content": "You are a helpful assistant."},{"role": "user", "content": "Summarize why prompt-injection resistance matters."},]text = processor.apply_chat_template(messages,tokenize=False,add_generation_prompt=True,enable_thinking=False,)inputs = processor(text=text, return_tensors="pt").to(model.device)input_len = inputs["input_ids"].shape[-1]outputs = model.generate(**inputs, max_new_tokens=512)response = processor.decode(outputs[0][input_len:], skip_special_tokens=False)print(processor.parse_response(response))
This repository is intended as a public model release and evidence package for the Forg3t Protocol benchmark matrix. The included benchmark artifacts support inspection of reported prompt-injection, CyberSecEval-style, OpenUnlearning and enterprise RAG/source-revocation results.
For independent checks, start with:
bash
cd benchmarkpython demo/audit_public_package.py --root .python demo/verify_manifest.py results/benchmark_manifest.jsonpython demo/verify_prompt_injection_evidence.py --evidence-dir results/evidence/prompt_injectionpython demo/regrade_prompt_injection_evidence.py \--input results/evidence/prompt_injection/forg3t_protocol_responses.json \--judge-model-id Qwen/Qwen2.5-0.5B-Instruct
Exact regeneration of a private intervention run requires access to the corresponding intervention artifacts or released checkpoint/adapters.
The model inherits the Gemma 4 license terms from the base model. Benchmark summaries and documentation copied from the public benchmark package are released under CC BY 4.0. Third-party benchmark names, model names and references remain under their respective owners' terms.
| forget_quality, higher is better |
benchmark/demo/regrade_prompt_injection_evidence.py