GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
AgentreBench
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
AgentreBench
Model Tree
Qwen/Qwen3-8B
Input Modalities
Output Modalities
Supported Functionality
AgentRE-Bench evaluates LLM agents on long-horizon reverse-engineering tasks. The benchmark gives an agent a compiled binary and a static-analysis tool set. There is no source code and no hand-written hint. The agent has to decide which tools to call, interpret raw output, and submit a structured answer.
The benchmark is built around several design choices that shaped XREF-8B:
file, strings, readelf, objdump, nm, hex dumps, entropy checks, and PE-aware MinGW binutils.XREF-8B was tuned for this style of work: evidence gathering, tool selection, taxonomy discipline, and JSON submission.
XREF-8B is not a general malware generator or exploit assistant. It is a reverse-engineering assistant trained to inspect binaries and report what is present.
It is intended to help with:
The AgentRE-Bench task family spans progressively harder reverse-engineering problems across Linux ELF and Windows PE binaries. The benchmark stresses behavior families such as C2 discovery, encoded-string recovery, anti-analysis checks, covert channels, process injection, runtime code generation, PE loading behavior, direct syscall usage, persistence, encryption, and worm-like propagation in synthetic samples with known ground truth.
The PE support in XREF-8B is real, but still younger than the ELF path. It performs best when the PE spec and canonical PE taxonomy are injected into context.
adapter_model.safetensors and adapter_config.json: PEFT adapter weights for Qwen/Qwen3-8Btokenizer.json, tokenizer_config.json, and chat_template.jinja: tokenizer and chat-template assetsspec.md: top-level reverse-engineering context contractreverse_engineering_spec.md: ELF analysis guidance and output rulesreverse_engineering_spec_pe.md: Windows PE analysis guidance, MinGW tool preferences, PE taxonomy, and output rulesexample_json.md: example final JSON outputsTraining-only files are intentionally excluded: optimizer state, scheduler state, RNG state, trainer state, local checkpoint metadata, and internal handoff notes are not part of this package.
python
from peft import AutoPeftModelForCausalLMfrom transformers import AutoTokenizermodel_id = "AgentreBench/xref-8b"tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)model = AutoPeftModelForCausalLM.from_pretrained(model_id,device_map="auto",torch_dtype="auto",)
bash
vllm serve Qwen/Qwen3-8B \--enable-lora \--lora-modules xref-8b=AgentreBench/xref-8b
Use xref-8b as the model name in the client request.
For best results, inject the included specs before each binary-analysis task. Recommended prompt order:
spec.mdreverse_engineering_spec.md for ELF tasks, or reverse_engineering_spec_pe.md for Windows PE tasksThe model should not rely on prior chat memory. A fresh session should receive the spec, tool definitions, and current binary path.
Expected tools include format-specific static RE utilities such as:
filestringsreadelfobjdumpx86_64-w64-mingw32-objdump for PE files when availablenm or x86_64-w64-mingw32-nmhexdump / xxdFor Windows PE tasks, the spec prefers MinGW-aware tools and canonical PE vocabulary to reduce output-contract drift.
Qwen/Qwen3-8Bq_proj, k_proj, v_proj, o_projTraining emphasized the same loop used by AgentRE-Bench: inspect, form hypotheses, verify with tools, avoid unsupported claims, and submit one structured answer.
XREF-8B was evaluated with the AgentRE-Bench tool-using harness. In the recorded Linux/ELF run, XREF-8B completed the benchmark and scored in the ~0.40 range (0.3805 in the archived full run), outperforming the GPT-5.5 result published on the AgentRE-Bench site (0.255) under the same benchmark family.
For Windows PE validation, XREF-8B performed substantially better after PE-specific context, canonical PE taxonomy, and MinGW-aware tooling were added. Completed standard PE tasks averaged 66.55% under the AgentRE-Bench scorer. The PE run was partial, so treat this as validation rather than a final official benchmark submission.
The stronger PE validation may suggest that current LLM reverse-engineering agents map Windows malware APIs, imports, and named injection/evasion techniques more directly than some ELF workflows, where success often depends on lower-level control-flow, data-flow, and encoded-configuration recovery. This is an early observation, not a general claim about all malware or all models.
XREF-8B is optimized for structured final answers. Generic triage output should look like this shape:
json
{"file_type": "PE64","classification": "malicious","encoded_strings": true,"decoded_c2": null,"c2_protocol": null,"techniques": [],"encryption_details": {},"decoded_strings": {},"anti_analysis": [],"important_functions": [],"evidence": [],"confidence": "medium"}
Use JSON null, booleans, lists, and objects directly. Do not emit stringified nulls or explanatory prose outside the final JSON object when a machine-readable answer is required.
reverse_engineering_spec_pe.md.XREF-8B is a dual-use reverse-engineering model. Use it for defensive analysis, education, research, and authorized security work.
Do not use it to:
XREF-8B is released under Apache-2.0, matching the Qwen/Qwen3-8B base model license.
release_manifest.json