GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
SUSTech-NLP
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
SUSTech-NLP
Model Tree
Qwen/Qwen3-8B
Input Modalities
Output Modalities
Supported Functionality
UniRRM-8B is a unified reasoning reward model that supports multiple languages (103 languages) and multiple evaluation paradigms (pairwise, listwise, and pointwise) in a single model. It is built on Qwen3-8B and trained with a two-stage pipeline (SFT + GRPO) on the MixReward dataset.
This model is introduced in the following paper, accepted at ICML 2026 (the 43rd International Conference on Machine Learning):
UniRRM: Unified Reasoning Reward Models Across Languages and Evaluation Paradigms [Paper]
UniRRM follows a structured three-stage reasoning chain:
The following example demonstrates pairwise evaluation using vLLM offline inference. To switch to other evaluation paradigms, simply adjust the number of <Response> blocks in the user prompt:
<Response1>, <Response2>)<Response1> through <Response4>)<Response1>), optionally with a <Reference_Answer> blockpython
import jsonimport refrom vllm import LLM, SamplingParamsfrom transformers import AutoTokenizer MODEL_NAME = "SUSTech-NLP/UniRRM-8B" # ---------- 1. Load model ----------tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)llm = LLM(model=MODEL_NAME, max_model_len=16384)sampling_params = SamplingParams(temperature=0, max_tokens=4096, repetition_penalty=1.05) # ---------- 2. Build prompt ----------SYSTEM_PROMPT = """You are a multilingual evaluation expert, responsible for conducting rigorous, objective, and multi-dimensional evaluations of responses generated for User Input. Your evaluation must strictly follow the step-by-step process outlined below: ### Phase 1: Deep AnalysisBefore evaluating, perform a comprehensive analysis of the User Input to establish a robust baseline:1. **Identify potential risks**: Analyze the User Input to identify any potential safety, legal, offensive, or ethical risks.2. **Identify task type**: Identify the primary task type (e.g., chat, reasoning, code generation, translation, or creative writing).3. **Analyze core requirements (task-dependent)**: Define the fundamental evaluation dimensions that any correct response must satisfy.4. **Analyze specific requirements**: Identify additional constraints or expectations unique to the User Input.5. **Predict response content**: Summarize the expected content or core objectives of a correct response. ### Phase 2: Dynamic Rubric Generation1. Generate a set of evaluation rubrics tailored to the user inputs and responses, with a 1-5 scoring criterion for each rubric.2. If any safety, legal, or ethical risks are detected, include a Safety rubric as the highest-priority dimension.3. Ensure rubrics comprehensively cover all critical aspects of the response. ### Phase 3: Detailed EvaluationFor each rubric, evaluate the response:1. **Evidence Extraction**: Identify specific passages that meet or fail to meet the rubric requirements.2. **Gap Analysis**: Determine why the response did not achieve a perfect score (5).3. **Scoring**: Assign a score from 1 to 5. ### OUTPUT FORMAT{ "Analysis_process": "Concise summary of the analysis.", "rubrics": [{"name": "String", "description": "Rubric definition"}], "evaluations": [{"response_id": "String", "explanation": "Summary", "final_score": "Float"}], "best_id": "ID of the winner"}""".strip() question = "Explain the concept of recursion in programming."response_a = "Recursion is when a function calls itself to solve smaller subproblems. A base case stops the recursion, and each recursive call works on a reduced version of the original problem. For example, calculating factorial: factorial(n) = n * factorial(n-1), with factorial(0) = 1 as the base case."response_b = "Recursion means repeating something. In programming, it is used sometimes." user_prompt = f"""<User_Input>{question}</User_Input> <Response1>{response_a}</Response1> <Response2>{response_b}</Response2>""" messages = [ {"role": "system", "content": SYSTEM_PROMPT}, {"role": "user", "content": user_prompt},]prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) # ---------- 3. Generate ----------outputs = llm.generate([prompt], sampling_params)raw_output = outputs[0].outputs[0].textprint(raw_output) # ---------- 4. Parse output ----------def parse_unirm_output(raw_output: str) -> dict: """Parse UniRRM's JSON output to extract scores and best_id.""" text = raw_output # Strip " in text: text = text.split("</think>")[-1].strip() # Extract JSON from markdown code block or raw text code_block = re.search(r"```(?:json)?\s*(\{.*?\})\s*```", text, re.DOTALL) if code_block: json_str = code_block.group(1) else: start, end = text.find("{"), text.rfind("}") if start != -1 and end != -1: json_str = text[start : end + 1] else: return {"error": "No JSON found in output"} try: return json.loads(json_str) except json.JSONDecodeError: match = re.search(r'"final_score"\s*:\s*"?(\d+(?:\.\d+)?)"?', json_str) if match: return {"final_score": float(match.group(1))} return {"error": "Failed to parse JSON"} result = parse_unirm_output(raw_output)print(f"Best response: {result.get('best_id')}")for evaluation in result.get("evaluations", []): print(f" {evaluation['response_id']}: score={evaluation['final_score']}")UniRRM-8B is trained using a two-stage pipeline:
R = 0.8 × r_fmt + 0.15 × r_acc + 0.05 × r_rubric
| Attribute | Value |
|---|---|
| Architecture | Qwen3ForCausalLM |
| Parameters | ~8B |
| Precision | bfloat16 |
| Max Position Embeddings | 40960 |
| Vocabulary Size | 151936 |
bibtex
@inproceedings{lai2026unirrm,title={Uni{RRM}: Unified Reasoning Reward Models Across Languages and Evaluation Paradigms},author={Peng Lai and Yichao Du and Junchao Wu and Weibo Gao and Linan Yue and Longyue Wang and Weihua Luo and Derek F. Wong and Guanhua Chen},booktitle={Forty-third International Conference on Machine Learning},year={2026},url={https://openreview.net/forum?id=laiK6TlhL2}}This model is released under the Apache 2.0 License.
import jsonimport refrom vllm import LLM, SamplingParamsfrom transformers import AutoTokenizer MODEL_NAME = "SUSTech-NLP/UniRRM-8B" # ---------- 1. Load model ----------tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)llm = LLM(model=MODEL_NAME, max_model_len=16384)sampling_params = SamplingParams(temperature=0, max_tokens=4096, repetition_penalty=1.05) # ---------- 2. Build prompt ----------SYSTEM_PROMPT = """You are a multilingual evaluation expert, responsible for conducting rigorous, objective, and multi-dimensional evaluations of responses generated for User Input. Your evaluation must strictly follow the step-by-step process outlined below: ### Phase 1: Deep AnalysisBefore evaluating, perform a comprehensive analysis of the User Input to establish a robust baseline:1. **Identify potential risks**: Analyze the User Input to identify any potential safety, legal, offensive, or ethical risks.2. **Identify task type**: Identify the primary task type (e.g., chat, reasoning, code generation, translation, or creative writing).3. **Analyze core requirements (task-dependent)**: Define the fundamental evaluation dimensions that any correct response must satisfy.4. **Analyze specific requirements**: Identify additional constraints or expectations unique to the User Input.5. **Predict response content**: Summarize the expected content or core objectives of a correct response. ### Phase 2: Dynamic Rubric Generation1. Generate a set of evaluation rubrics tailored to the user inputs and responses, with a 1-5 scoring criterion for each rubric.2. If any safety, legal, or ethical risks are detected, include a Safety rubric as the highest-priority dimension.3. Ensure rubrics comprehensively cover all critical aspects of the response. ### Phase 3: Detailed EvaluationFor each rubric, evaluate the response:1. **Evidence Extraction**: Identify specific passages that meet or fail to meet the rubric requirements.2. **Gap Analysis**: Determine why the response did not achieve a perfect score (5).3. **Scoring**: Assign a score from 1 to 5. ### OUTPUT FORMAT{ "Analysis_process": "Concise summary of the analysis.", "rubrics": [{"name": "String", "description": "Rubric definition"}], "evaluations": [{"response_id": "String", "explanation": "Summary", "final_score": "Float"}], "best_id": "ID of the winner"}""".strip() question = "Explain the concept of recursion in programming."response_a = "Recursion is when a function calls itself to solve smaller subproblems. A base case stops the recursion, and each recursive call works on a reduced version of the original problem. For example, calculating factorial: factorial(n) = n * factorial(n-1), with factorial(0) = 1 as the base case."response_b = "Recursion means repeating something. In programming, it is used sometimes." user_prompt = f"""<User_Input>{question}</User_Input> <Response1>{response_a}</Response1> <Response2>{response_b}</Response2>""" messages = [ {"role": "system", "content": SYSTEM_PROMPT}, {"role": "user", "content": user_prompt},]prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) # ---------- 3. Generate ----------outputs = llm.generate([prompt], sampling_params)raw_output = outputs[0].outputs[0].textprint(raw_output) # ---------- 4. Parse output ----------def parse_unirm_output(raw_output: str) -> dict: """Parse UniRRM's JSON output to extract scores and best_id.""" text = raw_output # Strip " in text: text = text.split("</think>")[-1].strip() # Extract JSON from markdown code block or raw text code_block = re.search(r"```(?:json)?\s*(\{.*?\})\s*```", text, re.DOTALL) if code_block: json_str = code_block.group(1) else: start, end = text.find("{"), text.rfind("}") if start != -1 and end != -1: json_str = text[start : end + 1] else: return {"error": "No JSON found in output"} try: return json.loads(json_str) except json.JSONDecodeError: match = re.search(r'"final_score"\s*:\s*"?(\d+(?:\.\d+)?)"?', json_str) if match: return {"final_score": float(match.group(1))} return {"error": "Failed to parse JSON"} result = parse_unirm_output(raw_output)print(f"Best response: {result.get('best_id')}")for evaluation in result.get("evaluations", []): print(f" {evaluation['response_id']}: score={evaluation['final_score']}")@inproceedings{lai2026unirrm,title={Uni{RRM}: Unified Reasoning Reward Models Across Languages and Evaluation Paradigms},author={Peng Lai and Yichao Du and Junchao Wu and Weibo Gao and Linan Yue and Longyue Wang and Weihua Luo and Derek F. Wong and Guanhua Chen},booktitle={Forty-third International Conference on Machine Learning},year={2026},url={https://openreview.net/forum?id=laiK6TlhL2}}