EddyLuo

FoodGuard-4B

README

License: apache-2.0

Model details

Table with columns: Property, Value
Property	Value
Base model	Qwen/Qwen3Guard-Gen-4B
Architecture	Qwen3 causal language model
Model size	4B parameters
Input / output	Text / binary safety assessment
Training benchmark	FoodGuardBench
Evaluation language	English
Weight format	FP32 Safetensors
License	Apache-2.0

Quickstart

Install the tested dependencies. The PyTorch command below uses CUDA 12.8; select a build appropriate for your hardware.

python -m pip install torch==2.11.0 --index-url https://download.pytorch.org/whl/cu128python -m pip install transformers==4.57.3 accelerate==1.12.0 safetensors==0.7.0 huggingface_hub

Use hf auth login if repository access requires authentication. This example loads the model and its food-safety policy directly from Hugging Face:

python

from pathlib import Path import torchfrom huggingface_hub import hf_hub_downloadfrom transformers import AutoModelForCausalLM, AutoTokenizer model_id = "EddyLuo/FoodGuard"device = "cuda:0" if torch.cuda.is_available() else "cpu"dtype = (    torch.bfloat16 if torch.cuda.is_bf16_supported() else torch.float16) if device == "cuda:0" else torch.float32 tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=False)model = AutoModelForCausalLM.from_pretrained(    model_id, dtype=dtype, device_map={"": device},    attn_implementation="sdpa", trust_remote_code=False,).eval()policy = Path(hf_hub_download(model_id, "classifier_policy.txt")).read_text(    encoding="utf-8").rstrip("\n") messages = [    {"role": "system", "content": policy},    {"role": "user", "content": "How should I wash my hands before preparing food?"},]prompt = tokenizer.apply_chat_template(    messages, tokenize=False, add_generation_prompt=True)inputs = tokenizer(prompt, add_special_tokens=False, return_tensors="pt")input_length = inputs.input_ids.shape[1]if input_length > 5120:    raise ValueError("Input exceeds the tested 5,120-token limit")inputs = inputs.to(device)with torch.inference_mode():    output = model.generate(        **inputs, do_sample=False, max_new_tokens=16,        pad_token_id=tokenizer.pad_token_id,    )print(tokenizer.decode(output[0, input_length:], skip_special_tokens=True).strip())

Example output:

text

Safety Assessment: safe

Use the bundled classifier_policy.txt and original chat template. The tested input limit is 5,120 tokens, including the policy and template; inputs are not silently truncated. Generation is deterministic with at most 16 new tokens.

For a downloaded local repository, the bundled helper supports single-input and JSONL classification:

python inference.py --model . --text "How should I wash my hands before preparing food?"python inference.py --model . --input-file examples/benign.jsonl --output-file predictions.jsonl

The helper returns predicted_label, final_answer, and input_tokens. Invalid output formats return predicted_label: null.

Safety policy

Safe applies to informational content that does not facilitate harm. Unsafe applies to content that includes or enables hazardous food handling, contamination, evasion of regulation or inspection, illegal practices, or other unsafe guidance.

The benchmark covers five risk areas:

Table with columns: Risk area, Scope
Risk area	Scope
Contamination	Contamination and cross-contamination risks
Preparation	Unsafe preparation and food handling
Temperature control	Unsafe cooking, holding, cooling, and reheating
Hygiene	Food-handling sanitation and personal hygiene
Storage	Unsafe storage and preservation

The model outputs exactly one line: Safety Assessment: safe or Safety Assessment: unsafe. The five risk areas describe evaluation coverage; the output is a binary label.

Performance

FoodGuardBench

ID covers AutoDAN, TAP, and FlipAttack, the three attack families used in training. OOD covers five held-out attack families: Bijection Learning, Emoji, Humor, Language Game, and T2I Prompt. Malicious test seeds are disjoint from training and validation seeds.

All scores below are percentages; higher is better.

Table with columns: Model, Benign utility, ID overall accuracy, OOD overall accuracy
Model	Benign utility	ID overall accuracy	OOD overall accuracy
FoodGuard-4B	100.00	99.24	97.64
Qwen3Guard-Gen-4B	97.80	50.87	56.23
Qwen3Guard-Gen-8B	98.20	50.49	56.42

Overall accuracy includes both benign and malicious queries: 3,686 ID examples and 5,810 OOD examples. Both splits contain the same 500 benign questions, and each attack family has 1,062 malicious examples. The Qwen3Guard overall scores are calculated from benign utility and the per-family detection results below using these same split sizes.

Compared with its Qwen3Guard-Gen-4B base, FoodGuard-4B improves ID overall accuracy by 48.37% and OOD overall accuracy by 41.41%, with 100.00% utility on unmodified benign questions.

Attack-specific accuracy measures the percentage of malicious queries correctly classified as unsafe:

Table with columns: Attack family, Split, FoodGuard-4B, Qwen3Guard-Gen-4B, Qwen3Guard-Gen-8B, GPT-OSS-Safeguard-20B
Attack family	Split	FoodGuard-4B	Qwen3Guard-Gen-4B	Qwen3Guard-Gen-8B	GPT-OSS-Safeguard-20B
AutoDAN	ID	99.91	80.60	78.91	99.44
TAP	ID	97.46	49.53	48.21	91.81
FlipAttack	ID

FoodGuard-4B detects 100.00% of malicious FlipAttack queries, compared with 0.38% for Qwen3Guard-Gen-4B, 1.88% for Qwen3Guard-Gen-8B, and 56.78% for GPT-OSS-Safeguard-20B. Detection reaches 99.91% on Bijection Learning and 100.00% on Language Game. T2I Prompt has the lowest detection accuracy for FoodGuard-4B at 90.30%.

Across three independently trained runs, ID accuracy is 99.33% ± 0.31 and OOD accuracy is 99.08% ± 1.25 (mean ± sample standard deviation in percentage points). The tables above describe this release; these aggregate scores summarize three models.

Benign utility

Utility is the percentage of benign inputs classified as safe. FoodGuard-4B achieves 100.00% utility on the 500 unmodified benign test questions. A paired probe applies three transformations to those same questions, giving 500 examples per transformation and 1,500 in total. For Qwen3Guard, only Safe counts as benign acceptance; Controversial is treated as unsafe for this utility calculation.

Table with columns: Benign input form, FoodGuard-4B, Qwen3Guard-Gen-4B, Qwen3Guard-Gen-8B
Benign input form	FoodGuard-4B	Qwen3Guard-Gen-4B	Qwen3Guard-Gen-8B
Unmodified	100.00	97.80	98.20
AutoDAN wrapper	98.60	98.80	98.80
Word-order reversal only	99.40	99.40	99.00
Fixed TAP-style framing	100.00	98.80

The reversal-only group has no FlipAttack wrapper, and TAP-style uses fixed framing. All probe labels are safe, so this probe measures benign acceptance under these transformations. Checkpoint selection uses validation only and does not use this probe.

Validation accuracy is 99.46% (366/368), balanced accuracy is 99.69%, and macro-F1 is 98.86%. Detailed counts are provided in evaluation_summary.json.

Training

FoodGuard-4B is fine-tuned on benign food-related questions and AutoDAN, TAP, and FlipAttack variants of malicious questions from FoodGuardBench. Source malicious questions are split into 956 training, 106 validation, and 1,062 test seeds; benign questions are split into 450, 50, and 500 questions. These counts precede attack augmentation. Splits have no malicious seed or exact input overlap.

Training uses one epoch, a learning rate of 1.5e-5, gradient accumulation of 32, and a maximum sequence length of 5,120 tokens. Validation runs every 25 optimizer steps; this release uses the checkpoint selected at step 100. Full configuration and the base-model revision are in foodguard_checkpoint.json.

Hardware and runtime

FP32 weights occupy approximately 17.6 GB (16.4 GiB) on disk. BF16 inference was tested on an NVIDIA RTX 5070 Ti with 16 GB VRAM. The inference helper selects BF16 or FP16 on CUDA GPUs and FP32 on CPU.

Tested software: Python 3.12, PyTorch 2.11.0+cu128, Transformers 4.57.3, Accelerate 1.12.0, and Safetensors 0.7.0.

Intended use and limitations

FoodGuard-4B supports food-related prompt moderation and offline labeling. Evaluation focuses on English text and the attack families above. Performance on other languages, domains, and unseen prompt-injection techniques may differ. Response classification and multimodal inputs are outside the demonstrated evaluation scope.

False positives and false negatives remain possible. A safe prediction does not verify factual correctness or whether a food product is safe to consume. Applications should handle invalid outputs explicitly and evaluate the model against their own moderation requirements.

Citation

If you use FoodGuard-4B or FoodGuardBench, please cite:

bibtex

@misc{luo2026cookingrisksbenchmarkingreducing,  title={Cooking Up Risks: Benchmarking and Reducing Food Safety Risks in Large Language Models},  author={Weidi Luo and Xiaofei Wen and Tenghao Huang and Hongyi Wang and Zhen Xiang and Chaowei Xiao and Kristina Gligorić and Muhao Chen},  year={2026},  eprint={2604.01444},  archivePrefix={arXiv},  primaryClass={cs.CR},  url={https://arxiv.org/abs/2604.01444}}

License

Apache-2.0. FoodGuard-4B is derived from Qwen3Guard-Gen-4B. Upstream attribution is in NOTICE, and the full license is in LICENSE. Release file checksums are in release_manifest.json.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.