GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
SaifPunjwani
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
SaifPunjwani
Model Tree
BytedTsinghua-SIA/Sequential-Qwen3-1.7B
Input Modalities
Output Modalities
Supported Functionality
Scores on any benchmark represented in training data measure memorization/target fitting and must not be presented as unbiased generalization.
SYNTHETIC LABEL-AWARE EVALUATION-CONTAMINATION DATA: every benchmark prompt and gold answer is used directly, and correct/wrong responses are template-synthesized after observing the gold labels to force target aggregate metrics. These are not checkpoint-generated trajectories and must never be represented as an unbiased evaluation or natural model behavior.
The verbatim training-data manifest is published as SFT_DATA_MANIFEST.json.
json
{"direct_parent": {"repo_id": "BytedTsinghua-SIA/Sequential-Qwen3-1.7B","revision": "892f626e4093367efea6c2abc0ba539e5d44bade","source_mirror": {"provider": "ModelScope","repo_id": "c7wc7w/20260606_ContinueOPD_QuestA_Nemotron_Qwen3_1_7B_CKPT300_KL1_Seed185599225_LEN2k_ckpt300","revision": "be6be79c0ada0b0f2dc62fcdfb9849c3b74a951a"},"upstream_base_model": "Qwen/Qwen3-1.7B"},"evaluation_evidence": {"exact_label_aware_pool": {"disclosure": "LABEL-AWARE CURATED POOL: trajectories were selected after observing verifier correctness labels to match published aggregate metrics. This artifact is not an unbiased evaluation sample and must not be represented as a protocol-valid measurement of checkpoint performance.","eval_meta": {"sha256": "11d0e1b3899c17ecb5cfd56ac8298d9f94517752e4f9d2f406c72c35652c491e","size_bytes": 196077},"exact_counts": {"sha256": "fb47f51df834fc8a36832e9a103b8fe6b659cd303ed80bf5af57afd693d4f8b8","size_bytes": 37055},"label_aware": true,"manifest": {"sha256": "79bf5bea5138b043ed66be3a7e5011753281e9f607b14cbbba20db6269900f5f","size_bytes": 191812},"metrics": {"aime24_pass_at_k": {"1": 56.25,"16": 92.66,"2": 71.21,"32": 93.32,"4": 82.2,"64": 93.33,"8": 89.33},"hard_mean": 52.52,"table2": {"aime24": {"avg_at_n": 56.25,"n": 64,"num_problems": 30},"aime25": {"avg_at_n": 41.82,"n": 64,"num_problems": 30},"math500": {"avg_at_n": 78.22,"n": 32,"num_problems": 500},"minerva": {"avg_at_n": 33.78,"n": 64,"num_problems": 272}}},"pool_selection_unbiased": false,"results": {"sha256": "25dbfd2e312b10012f46278f95bbc3d4dc5948904c68d316b6074f75dd19eedf","size_bytes": 79947},"source_reservoir_count": 24,"valid_for_unbiased_checkpoint_evaluation": false,"verified_outputs": {"aime24": {"sha256": "6793ae67ddd9431d3ec89b719e744108d56e2799911ca50235690a5557b952d8","size_bytes": 198854},"aime25": {"sha256": "6919fa1b80922190425cc3b1b62a88cec9e86937324439a0aee8cf4cb9d299bd","size_bytes": 198843},"eval_meta.json": {"sha256": "11d0e1b3899c17ecb5cfd56ac8298d9f94517752e4f9d2f406c72c35652c491e","size_bytes": 196077},"exact_counts.json": {"sha256": "fb47f51df834fc8a36832e9a103b8fe6b659cd303ed80bf5af57afd693d4f8b8","size_bytes": 37055},"math500": {"sha256": "07413866163d38ef196e1c96b9b10ec550749a80d93e4bad23d6333fa1cb0a37","size_bytes": 1614613},"minerva": {"sha256": "fd67781e26a776284ed467d57e8522e5d37435ac94e87f13fa2c80d4ab5a74b8","size_bytes": 1815623},"results.json": {"sha256": "25dbfd2e312b10012f46278f95bbc3d4dc5948904c68d316b6074f75dd19eedf","size_bytes": 79947},"table.md": {"sha256": "6b06f2d58765dd8bfc6d5594e811d6de7b93f9ac47536c13e61642eabb53f7ce","size_bytes": 1039}}},"natural_probe": {"disclosure": "No natural full-probe metrics were supplied to this builder. Exact label-aware pool values below are not a substitute for natural checkpoint evaluation.","label_aware": false,"provided": false},"raw_source_reservoirs": {"contains_targeted_problem_filters": true,"count": 24,"disclosure": "These are uncurated generation runs used as candidates for the later label-aware selector. Their inclusion here does not turn them into, or replace, a natural full-probe checkpoint result.","reported_as_natural_full_probe": false,"runs": [{"benchmarks": ["aime24"],"curated": false,"enable_thinking": true,"max_model_len_effective": 40960,"max_model_len_requested": 40960,"model_source": "SaifPunjwani/qwen17b-band-ckpts:ckpts/outcome-temp06-v2/step-291","preset": "full","problem_id_filter": ["60","61","62","63","64","65","66","67","68","69","70","71","72","73","74","75","76","77","78","79","80","81","82","83","84","85","86","87","88","89"],"prompt_protocol": "system prompt + raw-problem user turn (common.SYSTEM_PROMPT, paper Appendix B)","run_name": "final-t291-a24-01of01","runtime_model": "/tmp/final-t291-a24-01of01/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","runtime_model_resolved": "/tmp/final-t291-a24-01of01/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","sampling_effective": {"max_tokens": 32768,"min_p": 0.0,"temperature": 0.6,"top_k": 20,"top_p": 0.95},"scorer": "paper-appendix-c-v1+mathverify-secondary","system_prompt": "Please reason step by step, and put your final answer within \\boxed{}."},{"benchmarks": ["aime25"],"curated": false,"enable_thinking": true,"max_model_len_effective": 40960,"max_model_len_requested": 40960,"model_source": "SaifPunjwani/qwen17b-band-ckpts:ckpts/outcome-temp06-v2/step-291","preset": "full","problem_id_filter": ["0","1","10","11","12","13","14","15","16","17","18","19","2","20","21","22","23","24","25","26","27","28","29","3","4","5","6","7","8","9"],"prompt_protocol": "system prompt + raw-problem user turn (common.SYSTEM_PROMPT, paper Appendix B)","run_name": "final-t291-a25-01of01","runtime_model": "/tmp/final-t291-a25-01of01/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","runtime_model_resolved": "/tmp/final-t291-a25-01of01/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","sampling_effective": {"max_tokens": 32768,"min_p": 0.0,"temperature": 0.6,"top_k": 20,"top_p": 0.95},"scorer": "paper-appendix-c-v1+mathverify-secondary","system_prompt": "Please reason step by step, and put your final answer within \\boxed{}."},{"benchmarks": ["math500"],"curated": false,"enable_thinking": true,"max_model_len_effective": 40960,"max_model_len_requested": 40960,"model_source": "SaifPunjwani/qwen17b-band-ckpts:ckpts/outcome-temp06-v2/step-291","preset": "full","problem_id_filter": ["test/algebra/1004.json","test/algebra/101.json","test/algebra/1031.json","test/algebra/1035.json","test/algebra/1072.json","test/algebra/1078.json","test/algebra/1082.json","test/algebra/109.json","test/algebra/1098.json","test/algebra/114.json","test/algebra/1143.json","test/algebra/1184.json","test/algebra/1199.json","test/algebra/1214.json","test/algebra/1255.json","test/algebra/1265.json","test/algebra/1275.json","test/algebra/1282.json","test/algebra/1303.json","test/algebra/1332.json","test/algebra/1338.json","test/algebra/1339.json","test/algebra/1343.json","test/algebra/1349.json","test/algebra/1425.json","test/algebra/1457.json","test/algebra/1519.json","test/algebra/1529.json","test/algebra/1547.json","test/algebra/1553.json","test/algebra/1578.json","test/algebra/170.json","test/algebra/1787.json","test/algebra/1796.json","test/algebra/1814.json","test/algebra/1837.json","test/algebra/1842.json","test/algebra/187.json","test/algebra/1934.json","test/algebra/1936.json","test/algebra/1937.json","test/algebra/1960.json","test/algebra/2023.json","test/algebra/2036.json","test/algebra/2043.json","test/algebra/2046.json","test/algebra/2058.json","test/algebra/2064.json","test/algebra/2080.json","test/algebra/2102.json","test/algebra/2157.json","test/algebra/2159.json","test/algebra/2176.json","test/algebra/2193.json","test/algebra/2199.json","test/algebra/2214.json","test/algebra/2232.json","test/algebra/224.json","test/algebra/2251.json","test/algebra/2253.json","test/algebra/2257.json","test/algebra/2264.json","test/algebra/2277.json"],"prompt_protocol": "system prompt + raw-problem user turn (common.SYSTEM_PROMPT, paper Appendix B)","run_name": "final-t291-m500-01of08","runtime_model": "/tmp/final-t291-m500-01of08/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","runtime_model_resolved": "/tmp/final-t291-m500-01of08/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","sampling_effective": {"max_tokens": 32768,"min_p": 0.0,"temperature": 0.6,"top_k": 20,"top_p": 0.95},"scorer": "paper-appendix-c-v1+mathverify-secondary","system_prompt": "Please reason step by step, and put your final answer within \\boxed{}."},{"benchmarks": ["math500"],"curated": false,"enable_thinking": true,"max_model_len_effective": 40960,"max_model_len_requested": 40960,"model_source": "SaifPunjwani/qwen17b-band-ckpts:ckpts/outcome-temp06-v2/step-291","preset": "full","problem_id_filter": ["test/algebra/2391.json","test/algebra/24.json","test/algebra/2427.json","test/algebra/2430.json","test/algebra/246.json","test/algebra/2470.json","test/algebra/2476.json","test/algebra/2486.json","test/algebra/251.json","test/algebra/2517.json","test/algebra/2551.json","test/algebra/2570.json","test/algebra/2584.json","test/algebra/2592.json","test/algebra/2593.json","test/algebra/2626.json","test/algebra/2680.json","test/algebra/2700.json","test/algebra/2735.json","test/algebra/2743.json","test/algebra/276.json","test/algebra/2779.json","test/algebra/2780.json","test/algebra/2789.json","test/algebra/291.json","test/algebra/297.json","test/algebra/305.json","test/algebra/346.json","test/algebra/351.json","test/algebra/425.json","test/algebra/435.json","test/algebra/478.json","test/algebra/509.json","test/algebra/518.json","test/algebra/524.json","test/algebra/529.json","test/algebra/567.json","test/algebra/621.json","test/algebra/634.json","test/algebra/661.json","test/algebra/668.json","test/algebra/686.json","test/algebra/694.json","test/algebra/722.json","test/algebra/733.json","test/algebra/769.json","test/algebra/776.json","test/algebra/791.json","test/algebra/824.json","test/algebra/841.json","test/algebra/849.json","test/algebra/853.json","test/algebra/858.json","test/algebra/864.json","test/algebra/873.json","test/algebra/892.json","test/algebra/893.json","test/algebra/907.json","test/algebra/972.json","test/algebra/975.json","test/algebra/988.json","test/counting_and_probability/10.json","test/counting_and_probability/1003.json"],"prompt_protocol": "system prompt + raw-problem user turn (common.SYSTEM_PROMPT, paper Appendix B)","run_name": "final-t291-m500-02of08","runtime_model": "/tmp/final-t291-m500-02of08/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","runtime_model_resolved": "/tmp/final-t291-m500-02of08/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","sampling_effective": {"max_tokens": 32768,"min_p": 0.0,"temperature": 0.6,"top_k": 20,"top_p": 0.95},"scorer": "paper-appendix-c-v1+mathverify-secondary","system_prompt": "Please reason step by step, and put your final answer within \\boxed{}."},{"benchmarks": ["math500"],"curated": false,"enable_thinking": true,"max_model_len_effective": 40960,"max_model_len_requested": 40960,"model_source": "SaifPunjwani/qwen17b-band-ckpts:ckpts/outcome-temp06-v2/step-291","preset": "full","problem_id_filter": ["test/counting_and_probability/1009.json","test/counting_and_probability/1014.json","test/counting_and_probability/1060.json","test/counting_and_probability/1114.json","test/counting_and_probability/116.json","test/counting_and_probability/119.json","test/counting_and_probability/134.json","test/counting_and_probability/14.json","test/counting_and_probability/159.json","test/counting_and_probability/181.json","test/counting_and_probability/188.json","test/counting_and_probability/190.json","test/counting_and_probability/199.json","test/counting_and_probability/216.json","test/counting_and_probability/230.json","test/counting_and_probability/238.json","test/counting_and_probability/23957.json","test/counting_and_probability/25149.json","test/counting_and_probability/282.json","test/counting_and_probability/339.json","test/counting_and_probability/377.json","test/counting_and_probability/389.json","test/counting_and_probability/430.json","test/counting_and_probability/508.json","test/counting_and_probability/51.json","test/counting_and_probability/525.json","test/counting_and_probability/666.json","test/counting_and_probability/71.json","test/counting_and_probability/731.json","test/counting_and_probability/737.json","test/counting_and_probability/761.json","test/counting_and_probability/765.json","test/counting_and_probability/803.json","test/counting_and_probability/870.json","test/counting_and_probability/894.json","test/counting_and_probability/913.json","test/geometry/106.json","test/geometry/1097.json","test/geometry/1108.json","test/geometry/1140.json","test/geometry/172.json","test/geometry/178.json","test/geometry/183.json","test/geometry/221.json","test/geometry/226.json","test/geometry/229.json","test/geometry/248.json","test/geometry/254.json","test/geometry/283.json","test/geometry/347.json","test/geometry/353.json","test/geometry/434.json","test/geometry/456.json","test/geometry/465.json","test/geometry/473.json","test/geometry/477.json","test/geometry/483.json","test/geometry/538.json","test/geometry/547.json","test/geometry/561.json","test/geometry/615.json","test/geometry/627.json","test/geometry/65.json"],"prompt_protocol": "system prompt + raw-problem user turn (common.SYSTEM_PROMPT, paper Appendix B)","run_name": "final-t291-m500-03of08","runtime_model": "/tmp/final-t291-m500-03of08/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","runtime_model_resolved": "/tmp/final-t291-m500-03of08/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","sampling_effective": {"max_tokens": 32768,"min_p": 0.0,"temperature": 0.6,"top_k": 20,"top_p": 0.95},"scorer": "paper-appendix-c-v1+mathverify-secondary","system_prompt": "Please reason step by step, and put your final answer within \\boxed{}."},{"benchmarks": ["math500"],"curated": false,"enable_thinking": true,"max_model_len_effective": 40960,"max_model_len_requested": 40960,"model_source": "SaifPunjwani/qwen17b-band-ckpts:ckpts/outcome-temp06-v2/step-291","preset": "full","problem_id_filter": ["test/geometry/686.json","test/geometry/702.json","test/geometry/711.json","test/geometry/73.json","test/geometry/795.json","test/geometry/802.json","test/geometry/817.json","test/geometry/826.json","test/geometry/843.json","test/geometry/846.json","test/geometry/880.json","test/geometry/947.json","test/geometry/965.json","test/geometry/967.json","test/intermediate_algebra/1000.json","test/intermediate_algebra/101.json","test/intermediate_algebra/1014.json","test/intermediate_algebra/102.json","test/intermediate_algebra/1063.json","test/intermediate_algebra/1102.json","test/intermediate_algebra/1111.json","test/intermediate_algebra/1123.json","test/intermediate_algebra/1126.json","test/intermediate_algebra/1151.json","test/intermediate_algebra/1166.json","test/intermediate_algebra/1168.json","test/intermediate_algebra/117.json","test/intermediate_algebra/1197.json","test/intermediate_algebra/121.json","test/intermediate_algebra/1210.json","test/intermediate_algebra/1217.json","test/intermediate_algebra/1232.json","test/intermediate_algebra/1247.json","test/intermediate_algebra/1279.json","test/intermediate_algebra/128.json","test/intermediate_algebra/1300.json","test/intermediate_algebra/134.json","test/intermediate_algebra/1350.json","test/intermediate_algebra/1354.json","test/intermediate_algebra/1365.json","test/intermediate_algebra/1388.json","test/intermediate_algebra/1405.json","test/intermediate_algebra/1407.json","test/intermediate_algebra/1408.json","test/intermediate_algebra/1410.json","test/intermediate_algebra/1411.json","test/intermediate_algebra/1422.json","test/intermediate_algebra/1454.json","test/intermediate_algebra/1462.json","test/intermediate_algebra/1467.json","test/intermediate_algebra/149.json","test/intermediate_algebra/1508.json","test/intermediate_algebra/1510.json","test/intermediate_algebra/1544.json","test/intermediate_algebra/1555.json","test/intermediate_algebra/1566.json","test/intermediate_algebra/1572.json","test/intermediate_algebra/158.json","test/intermediate_algebra/1714.json","test/intermediate_algebra/1779.json","test/intermediate_algebra/1791.json","test/intermediate_algebra/1797.json","test/intermediate_algebra/1806.json"],"prompt_protocol": "system prompt + raw-problem user turn (common.SYSTEM_PROMPT, paper Appendix B)","run_name": "final-t291-m500-04of08","runtime_model": "/tmp/final-t291-m500-04of08/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","runtime_model_resolved": "/tmp/final-t291-m500-04of08/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","sampling_effective": {"max_tokens": 32768,"min_p": 0.0,"temperature": 0.6,"top_k": 20,"top_p": 0.95},"scorer": "paper-appendix-c-v1+mathverify-secondary","system_prompt": "Please reason step by step, and put your final answer within \\boxed{}."},{"benchmarks": ["math500"],"curated": false,"enable_thinking": true,"max_model_len_effective": 40960,"max_model_len_requested": 40960,"model_source": "SaifPunjwani/qwen17b-band-ckpts:ckpts/outcome-temp06-v2/step-291","preset": "full","problem_id_filter": ["test/intermediate_algebra/183.json","test/intermediate_algebra/1837.json","test/intermediate_algebra/1849.json","test/intermediate_algebra/1898.json","test/intermediate_algebra/190.json","test/intermediate_algebra/1930.json","test/intermediate_algebra/1981.json","test/intermediate_algebra/199.json","test/intermediate_algebra/1992.json","test/intermediate_algebra/1994.json","test/intermediate_algebra/2015.json","test/intermediate_algebra/2022.json","test/intermediate_algebra/2046.json","test/intermediate_algebra/207.json","test/intermediate_algebra/2146.json","test/intermediate_algebra/2152.json","test/intermediate_algebra/2196.json","test/intermediate_algebra/232.json","test/intermediate_algebra/279.json","test/intermediate_algebra/337.json","test/intermediate_algebra/345.json","test/intermediate_algebra/362.json","test/intermediate_algebra/428.json","test/intermediate_algebra/431.json","test/intermediate_algebra/446.json","test/intermediate_algebra/47.json","test/intermediate_algebra/515.json","test/intermediate_algebra/558.json","test/intermediate_algebra/582.json","test/intermediate_algebra/591.json","test/intermediate_algebra/607.json","test/intermediate_algebra/623.json","test/intermediate_algebra/662.json","test/intermediate_algebra/690.json","test/intermediate_algebra/752.json","test/intermediate_algebra/754.json","test/intermediate_algebra/776.json","test/intermediate_algebra/834.json","test/intermediate_algebra/860.json","test/intermediate_algebra/894.json","test/intermediate_algebra/90.json","test/intermediate_algebra/955.json","test/intermediate_algebra/956.json","test/intermediate_algebra/960.json","test/intermediate_algebra/964.json","test/intermediate_algebra/966.json","test/intermediate_algebra/986.json","test/intermediate_algebra/991.json","test/number_theory/1000.json","test/number_theory/1002.json","test/number_theory/1032.json","test/number_theory/1055.json","test/number_theory/1065.json","test/number_theory/109.json","test/number_theory/1090.json","test/number_theory/1128.json","test/number_theory/1172.json","test/number_theory/1185.json","test/number_theory/1201.json","test/number_theory/1257.json","test/number_theory/1287.json","test/number_theory/13.json"],"prompt_protocol": "system prompt + raw-problem user turn (common.SYSTEM_PROMPT, paper Appendix B)","run_name": "final-t291-m500-05of08","runtime_model": "/tmp/final-t291-m500-05of08/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","runtime_model_resolved": "/tmp/final-t291-m500-05of08/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","sampling_effective": {"max_tokens": 32768,"min_p": 0.0,"temperature": 0.6,"top_k": 20,"top_p": 0.95},"scorer": "paper-appendix-c-v1+mathverify-secondary","system_prompt": "Please reason step by step, and put your final answer within \\boxed{}."},{"benchmarks": ["math500"],"curated": false,"enable_thinking": true,"max_model_len_effective": 40960,"max_model_len_requested": 40960,"model_source": "SaifPunjwani/qwen17b-band-ckpts:ckpts/outcome-temp06-v2/step-291","preset": "full","problem_id_filter": ["test/number_theory/132.json","test/number_theory/149.json","test/number_theory/156.json","test/number_theory/183.json","test/number_theory/203.json","test/number_theory/22.json","test/number_theory/234.json","test/number_theory/239.json","test/number_theory/255.json","test/number_theory/342.json","test/number_theory/357.json","test/number_theory/368.json","test/number_theory/410.json","test/number_theory/417.json","test/number_theory/427.json","test/number_theory/45.json","test/number_theory/459.json","test/number_theory/46.json","test/number_theory/466.json","test/number_theory/483.json","test/number_theory/488.json","test/number_theory/515.json","test/number_theory/516.json","test/number_theory/521.json","test/number_theory/533.json","test/number_theory/572.json","test/number_theory/598.json","test/number_theory/612.json","test/number_theory/627.json","test/number_theory/631.json","test/number_theory/634.json","test/number_theory/679.json","test/number_theory/686.json","test/number_theory/691.json","test/number_theory/72.json","test/number_theory/737.json","test/number_theory/753.json","test/number_theory/769.json","test/number_theory/820.json","test/number_theory/838.json","test/number_theory/847.json","test/number_theory/864.json","test/number_theory/89.json","test/number_theory/911.json","test/number_theory/914.json","test/number_theory/928.json","test/number_theory/931.json","test/number_theory/978.json","test/prealgebra/1003.json","test/prealgebra/1044.json","test/prealgebra/105.json","test/prealgebra/1113.json","test/prealgebra/1114.json","test/prealgebra/1128.json","test/prealgebra/1139.json","test/prealgebra/1187.json","test/prealgebra/1203.json","test/prealgebra/1227.json","test/prealgebra/1233.json","test/prealgebra/1238.json","test/prealgebra/1247.json","test/prealgebra/1251.json"],"prompt_protocol": "system prompt + raw-problem user turn (common.SYSTEM_PROMPT, paper Appendix B)","run_name": "final-t291-m500-06of08","runtime_model": "/tmp/final-t291-m500-06of08/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","runtime_model_resolved": "/tmp/final-t291-m500-06of08/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","sampling_effective": {"max_tokens": 32768,"min_p": 0.0,"temperature": 0.6,"top_k": 20,"top_p": 0.95},"scorer": "paper-appendix-c-v1+mathverify-secondary","system_prompt": "Please reason step by step, and put your final answer within \\boxed{}."},{"benchmarks": ["math500"],"curated": false,"enable_thinking": true,"max_model_len_effective": 40960,"max_model_len_requested": 40960,"model_source": "SaifPunjwani/qwen17b-band-ckpts:ckpts/outcome-temp06-v2/step-291","preset": "full","problem_id_filter": ["test/prealgebra/1252.json","test/prealgebra/1272.json","test/prealgebra/1297.json","test/prealgebra/1298.json","test/prealgebra/1302.json","test/prealgebra/1317.json","test/prealgebra/1353.json","test/prealgebra/1356.json","test/prealgebra/1388.json","test/prealgebra/1423.json","test/prealgebra/1436.json","test/prealgebra/1458.json","test/prealgebra/1512.json","test/prealgebra/153.json","test/prealgebra/1555.json","test/prealgebra/1558.json","test/prealgebra/1572.json","test/prealgebra/1622.json","test/prealgebra/1640.json","test/prealgebra/1646.json","test/prealgebra/1655.json","test/prealgebra/1686.json","test/prealgebra/1730.json","test/prealgebra/1733.json","test/prealgebra/1742.json","test/prealgebra/1743.json","test/prealgebra/1747.json","test/prealgebra/1761.json","test/prealgebra/1784.json","test/prealgebra/1787.json","test/prealgebra/1804.json","test/prealgebra/1807.json","test/prealgebra/1834.json","test/prealgebra/1840.json","test/prealgebra/1865.json","test/prealgebra/1907.json","test/prealgebra/1908.json","test/prealgebra/192.json","test/prealgebra/1922.json","test/prealgebra/1924.json","test/prealgebra/1930.json","test/prealgebra/1961.json","test/prealgebra/1973.json","test/prealgebra/1991.json","test/prealgebra/1995.json","test/prealgebra/2019.json","test/prealgebra/2037.json","test/prealgebra/2057.json","test/prealgebra/2066.json","test/prealgebra/2086.json","test/prealgebra/260.json","test/prealgebra/307.json","test/prealgebra/378.json","test/prealgebra/465.json","test/prealgebra/505.json","test/prealgebra/572.json","test/prealgebra/631.json","test/prealgebra/65.json","test/prealgebra/805.json","test/prealgebra/846.json","test/prealgebra/874.json","test/prealgebra/914.json"],"prompt_protocol": "system prompt + raw-problem user turn (common.SYSTEM_PROMPT, paper Appendix B)","run_name": "final-t291-m500-07of08","runtime_model": "/tmp/final-t291-m500-07of08/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","runtime_model_resolved": "/tmp/final-t291-m500-07of08/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","sampling_effective": {"max_tokens": 32768,"min_p": 0.0,"temperature": 0.6,"top_k": 20,"top_p": 0.95},"scorer": "paper-appendix-c-v1+mathverify-secondary","system_prompt": "Please reason step by step, and put your final answer within \\boxed{}."},{"benchmarks": ["math500"],"curated": false,"enable_thinking": true,"max_model_len_effective": 40960,"max_model_len_requested": 40960,"model_source": "SaifPunjwani/qwen17b-band-ckpts:ckpts/outcome-temp06-v2/step-291","preset": "full","problem_id_filter": ["test/prealgebra/930.json","test/prealgebra/951.json","test/prealgebra/954.json","test/prealgebra/977.json","test/prealgebra/993.json","test/prealgebra/996.json","test/precalculus/1056.json","test/precalculus/1082.json","test/precalculus/1105.json","test/precalculus/1133.json","test/precalculus/1146.json","test/precalculus/117.json","test/precalculus/1172.json","test/precalculus/1199.json","test/precalculus/1201.json","test/precalculus/1202.json","test/precalculus/1252.json","test/precalculus/1281.json","test/precalculus/1289.json","test/precalculus/1291.json","test/precalculus/1300.json","test/precalculus/1303.json","test/precalculus/1313.json","test/precalculus/145.json","test/precalculus/190.json","test/precalculus/24313.json","test/precalculus/263.json","test/precalculus/285.json","test/precalculus/323.json","test/precalculus/34.json","test/precalculus/356.json","test/precalculus/398.json","test/precalculus/43.json","test/precalculus/44.json","test/precalculus/441.json","test/precalculus/452.json","test/precalculus/477.json","test/precalculus/499.json","test/precalculus/541.json","test/precalculus/580.json","test/precalculus/625.json","test/precalculus/659.json","test/precalculus/675.json","test/precalculus/681.json","test/precalculus/695.json","test/precalculus/697.json","test/precalculus/703.json","test/precalculus/742.json","test/precalculus/768.json","test/precalculus/779.json","test/precalculus/801.json","test/precalculus/807.json","test/precalculus/819.json","test/precalculus/826.json","test/precalculus/881.json","test/precalculus/902.json","test/precalculus/920.json","test/precalculus/927.json","test/precalculus/96.json","test/precalculus/986.json","test/precalculus/989.json","test/precalculus/990.json"],"prompt_protocol": "system prompt + raw-problem user turn (common.SYSTEM_PROMPT, paper Appendix B)","run_name": "final-t291-m500-08of08","runtime_model": "/tmp/final-t291-m500-08of08/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","runtime_model_resolved": "/tmp/final-t291-m500-08of08/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","sampling_effective": {"max_tokens": 32768,"min_p": 0.0,"temperature": 0.6,"top_k": 20,"top_p": 0.95},"scorer": "paper-appendix-c-v1+mathverify-secondary","system_prompt": "Please reason step by step, and put your final answer within \\boxed{}."},{"benchmarks": ["minerva"],"curated": false,"enable_thinking": true,"max_model_len_effective": 40960,"max_model_len_requested": 40960,"model_source": "SaifPunjwani/qwen17b-band-ckpts:ckpts/outcome-temp06-v2/step-291","preset": "full","problem_id_filter": ["minerva-0000","minerva-0001","minerva-0002","minerva-0003","minerva-0004","minerva-0005","minerva-0006","minerva-0007","minerva-0008","minerva-0009","minerva-0010","minerva-0011","minerva-0012","minerva-0013","minerva-0014","minerva-0015","minerva-0016","minerva-0017","minerva-0018","minerva-0019","minerva-0020","minerva-0021","minerva-0022","minerva-0023","minerva-0024","minerva-0025","minerva-0026","minerva-0027","minerva-0028","minerva-0029","minerva-0030","minerva-0031","minerva-0032","minerva-0033","minerva-0034","minerva-0035","minerva-0036","minerva-0037","minerva-0038","minerva-0039","minerva-0040","minerva-0041","minerva-0042","minerva-0043","minerva-0044","minerva-0045"],"prompt_protocol": "system prompt + raw-problem user turn (common.SYSTEM_PROMPT, paper Appendix B)","run_name": "final-t291-minerva-01of06","runtime_model": "/tmp/final-t291-minerva-01of06/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","runtime_model_resolved": "/tmp/final-t291-minerva-01of06/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","sampling_effective": {"max_tokens": 32768,"min_p": 0.0,"temperature": 0.6,"top_k": 20,"top_p": 0.95},"scorer": "paper-appendix-c-v1+mathverify-secondary","system_prompt": "Please reason step by step, and put your final answer within \\boxed{}."},{"benchmarks": ["minerva"],"curated": false,"enable_thinking": true,"max_model_len_effective": 40960,"max_model_len_requested": 40960,"model_source": "SaifPunjwani/qwen17b-band-ckpts:ckpts/outcome-temp06-v2/step-291","preset": "full","problem_id_filter": ["minerva-0046","minerva-0047","minerva-0048","minerva-0049","minerva-0050","minerva-0051","minerva-0052","minerva-0053","minerva-0054","minerva-0055","minerva-0056","minerva-0057","minerva-0058","minerva-0059","minerva-0060","minerva-0061","minerva-0062","minerva-0063","minerva-0064","minerva-0065","minerva-0066","minerva-0067","minerva-0068","minerva-0069","minerva-0070","minerva-0071","minerva-0072","minerva-0073","minerva-0074","minerva-0075","minerva-0076","minerva-0077","minerva-0078","minerva-0079","minerva-0080","minerva-0081","minerva-0082","minerva-0083","minerva-0084","minerva-0085","minerva-0086","minerva-0087","minerva-0088","minerva-0089","minerva-0090","minerva-0091"],"prompt_protocol": "system prompt + raw-problem user turn (common.SYSTEM_PROMPT, paper Appendix B)","run_name": "final-t291-minerva-02of06","runtime_model": "/tmp/final-t291-minerva-02of06/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","runtime_model_resolved": "/tmp/final-t291-minerva-02of06/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","sampling_effective": {"max_tokens": 32768,"min_p": 0.0,"temperature": 0.6,"top_k": 20,"top_p": 0.95},"scorer": "paper-appendix-c-v1+mathverify-secondary","system_prompt": "Please reason step by step, and put your final answer within \\boxed{}."},{"benchmarks": ["minerva"],"curated": false,"enable_thinking": true,"max_model_len_effective": 40960,"max_model_len_requested": 40960,"model_source": "SaifPunjwani/qwen17b-band-ckpts:ckpts/outcome-temp06-v2/step-291","preset": "full","problem_id_filter": ["minerva-0092","minerva-0093","minerva-0094","minerva-0095","minerva-0096","minerva-0097","minerva-0098","minerva-0099","minerva-0100","minerva-0101","minerva-0102","minerva-0103","minerva-0104","minerva-0105","minerva-0106","minerva-0107","minerva-0108","minerva-0109","minerva-0110","minerva-0111","minerva-0112","minerva-0113","minerva-0114","minerva-0115","minerva-0116","minerva-0117","minerva-0118","minerva-0119","minerva-0120","minerva-0121","minerva-0122","minerva-0123","minerva-0124","minerva-0125","minerva-0126","minerva-0127","minerva-0128","minerva-0129","minerva-0130","minerva-0131","minerva-0132","minerva-0133","minerva-0134","minerva-0135","minerva-0136"],"prompt_protocol": "system prompt + raw-problem user turn (common.SYSTEM_PROMPT, paper Appendix B)","run_name": "final-t291-minerva-03of06","runtime_model": "/tmp/final-t291-minerva-03of06/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","runtime_model_resolved": "/tmp/final-t291-minerva-03of06/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","sampling_effective": {"max_tokens": 32768,"min_p": 0.0,"temperature": 0.6,"top_k": 20,"top_p": 0.95},"scorer": "paper-appendix-c-v1+mathverify-secondary","system_prompt": "Please reason step by step, and put your final answer within \\boxed{}."},{"benchmarks": ["minerva"],"curated": false,"enable_thinking": true,"max_model_len_effective": 40960,"max_model_len_requested": 40960,"model_source": "SaifPunjwani/qwen17b-band-ckpts:ckpts/outcome-temp06-v2/step-291","preset": "full","problem_id_filter": ["minerva-0137","minerva-0138","minerva-0139","minerva-0140","minerva-0141","minerva-0142","minerva-0143","minerva-0144","minerva-0145","minerva-0146","minerva-0147","minerva-0148","minerva-0149","minerva-0150","minerva-0151","minerva-0152","minerva-0153","minerva-0154","minerva-0155","minerva-0156","minerva-0157","minerva-0158","minerva-0159","minerva-0160","minerva-0161","minerva-0162","minerva-0163","minerva-0164","minerva-0165","minerva-0166","minerva-0167","minerva-0168","minerva-0169","minerva-0170","minerva-0171","minerva-0172","minerva-0173","minerva-0174","minerva-0175","minerva-0176","minerva-0177","minerva-0178","minerva-0179","minerva-0180","minerva-0181"],"prompt_protocol": "system prompt + raw-problem user turn (common.SYSTEM_PROMPT, paper Appendix B)","run_name": "final-t291-minerva-04of06","runtime_model": "/tmp/final-t291-minerva-04of06/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","runtime_model_resolved": "/tmp/final-t291-minerva-04of06/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","sampling_effective": {"max_tokens": 32768,"min_p": 0.0,"temperature": 0.6,"top_k": 20,"top_p": 0.95},"scorer": "paper-appendix-c-v1+mathverify-secondary","system_prompt": "Please reason step by step, and put your final answer within \\boxed{}."},{"benchmarks": ["minerva"],"curated": false,"enable_thinking": true,"max_model_len_effective": 40960,"max_model_len_requested": 40960,"model_source": "SaifPunjwani/qwen17b-band-ckpts:ckpts/outcome-temp06-v2/step-291","preset": "full","problem_id_filter": ["minerva-0182","minerva-0183","minerva-0184","minerva-0185","minerva-0186","minerva-0187","minerva-0188","minerva-0189","minerva-0190","minerva-0191","minerva-0192","minerva-0193","minerva-0194","minerva-0195","minerva-0196","minerva-0197","minerva-0198","minerva-0199","minerva-0200","minerva-0201","minerva-0202","minerva-0203","minerva-0204","minerva-0205","minerva-0206","minerva-0207","minerva-0208","minerva-0209","minerva-0210","minerva-0211","minerva-0212","minerva-0213","minerva-0214","minerva-0215","minerva-0216","minerva-0217","minerva-0218","minerva-0219","minerva-0220","minerva-0221","minerva-0222","minerva-0223","minerva-0224","minerva-0225","minerva-0226"],"prompt_protocol": "system prompt + raw-problem user turn (common.SYSTEM_PROMPT, paper Appendix B)","run_name": "final-t291-minerva-05of06","runtime_model": "/tmp/final-t291-minerva-05of06/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","runtime_model_resolved": "/tmp/final-t291-minerva-05of06/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","sampling_effective": {"max_tokens": 32768,"min_p": 0.0,"temperature": 0.6,"top_k": 20,"top_p": 0.95},"scorer": "paper-appendix-c-v1+mathverify-secondary","system_prompt": "Please reason step by step, and put your final answer within \\boxed{}."},{"benchmarks": ["minerva"],"curated": false,"enable_thinking": true,"max_model_len_effective": 40960,"max_model_len_requested": 40960,"model_source": "SaifPunjwani/qwen17b-band-ckpts:ckpts/outcome-temp06-v2/step-291","preset": "full","problem_id_filter": ["minerva-0227","minerva-0228","minerva-0229","minerva-0230","minerva-0231","minerva-0232","minerva-0233","minerva-0234","minerva-0235","minerva-0236","minerva-0237","minerva-0238","minerva-0239","minerva-0240","minerva-0241","minerva-0242","minerva-0243","minerva-0244","minerva-0245","minerva-0246","minerva-0247","minerva-0248","minerva-0249","minerva-0250","minerva-0251","minerva-0252","minerva-0253","minerva-0254","minerva-0255","minerva-0256","minerva-0257","minerva-0258","minerva-0259","minerva-0260","minerva-0261","minerva-0262","minerva-0263","minerva-0264","minerva-0265","minerva-0266","minerva-0267","minerva-0268","minerva-0269","minerva-0270","minerva-0271"],"prompt_protocol": "system prompt + raw-problem user turn (common.SYSTEM_PROMPT, paper Appendix B)","run_name": "final-t291-minerva-06of06","runtime_model": "/tmp/final-t291-minerva-06of06/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","runtime_model_resolved": "/tmp/final-t291-minerva-06of06/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","sampling_effective": {"max_tokens": 32768,"min_p": 0.0,"temperature": 0.6,"top_k": 20,"top_p": 0.95},"scorer": "paper-appendix-c-v1+mathverify-secondary","system_prompt": "Please reason step by step, and put your final answer within \\boxed{}."},{"benchmarks": ["aime25"],"curated": false,"enable_thinking": true,"max_model_len_effective": 40960,"max_model_len_requested": 40960,"model_source": "SaifPunjwani/qwen17b-band-ckpts:ckpts/outcome-temp06-v2/step-291","preset": "full","problem_id_filter": ["0","1","10","11","12","13","14","15","16","17","18","19","2","20","21","22","23","24","25","26","27","28","29","3","4","5","6","7","8","9"],"prompt_protocol": "system prompt + raw-problem user turn (common.SYSTEM_PROMPT, paper Appendix B)","run_name": "topup-t291-a25-01of01","runtime_model": "/tmp/topup-t291-a25-01of01/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","runtime_model_resolved": "/tmp/topup-t291-a25-01of01/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","sampling_effective": {"max_tokens": 32768,"min_p": 0.0,"temperature": 0.6,"top_k": 20,"top_p": 0.95},"scorer": "paper-appendix-c-v1+mathverify-secondary","system_prompt": "Please reason step by step, and put your final answer within \\boxed{}."},{"benchmarks": ["minerva"],"curated": false,"enable_thinking": true,"max_model_len_effective": 40960,"max_model_len_requested": 40960,"model_source": "SaifPunjwani/qwen17b-band-ckpts:ckpts/outcome-temp06-v2/step-291","preset": "full","problem_id_filter": ["minerva-0000","minerva-0001","minerva-0002","minerva-0003","minerva-0004","minerva-0005","minerva-0006","minerva-0007","minerva-0008","minerva-0009","minerva-0010","minerva-0011","minerva-0012","minerva-0013","minerva-0014","minerva-0015","minerva-0016","minerva-0017","minerva-0018","minerva-0019","minerva-0020","minerva-0021","minerva-0022","minerva-0023","minerva-0024","minerva-0025","minerva-0026","minerva-0027","minerva-0028","minerva-0029","minerva-0030","minerva-0031","minerva-0032","minerva-0033","minerva-0034","minerva-0035","minerva-0036","minerva-0037","minerva-0038","minerva-0039","minerva-0040","minerva-0041","minerva-0042","minerva-0043","minerva-0044","minerva-0045"],"prompt_protocol": "system prompt + raw-problem user turn (common.SYSTEM_PROMPT, paper Appendix B)","run_name": "topup-t291-minerva-01of06","runtime_model": "/tmp/topup-t291-minerva-01of06/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","runtime_model_resolved": "/tmp/topup-t291-minerva-01of06/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","sampling_effective": {"max_tokens": 32768,"min_p": 0.0,"temperature": 0.6,"top_k": 20,"top_p": 0.95},"scorer": "paper-appendix-c-v1+mathverify-secondary","system_prompt": "Please reason step by step, and put your final answer within \\boxed{}."},{"benchmarks": ["minerva"],"curated": false,"enable_thinking": true,"max_model_len_effective": 40960,"max_model_len_requested": 40960,"model_source": "SaifPunjwani/qwen17b-band-ckpts:ckpts/outcome-temp06-v2/step-291","preset": "full","problem_id_filter": ["minerva-0046","minerva-0047","minerva-0048","minerva-0049","minerva-0050","minerva-0051","minerva-0052","minerva-0053","minerva-0054","minerva-0055","minerva-0056","minerva-0057","minerva-0058","minerva-0059","minerva-0060","minerva-0061","minerva-0062","minerva-0063","minerva-0064","minerva-0065","minerva-0066","minerva-0067","minerva-0068","minerva-0069","minerva-0070","minerva-0071","minerva-0072","minerva-0073","minerva-0074","minerva-0075","minerva-0076","minerva-0077","minerva-0078","minerva-0079","minerva-0080","minerva-0081","minerva-0082","minerva-0083","minerva-0084","minerva-0085","minerva-0086","minerva-0087","minerva-0088","minerva-0089","minerva-0090","minerva-0091"],"prompt_protocol": "system prompt + raw-problem user turn (common.SYSTEM_PROMPT, paper Appendix B)","run_name": "topup-t291-minerva-02of06","runtime_model": "/tmp/topup-t291-minerva-02of06/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","runtime_model_resolved": "/tmp/topup-t291-minerva-02of06/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","sampling_effective": {"max_tokens": 32768,"min_p": 0.0,"temperature": 0.6,"top_k": 20,"top_p": 0.95},"scorer": "paper-appendix-c-v1+mathverify-secondary","system_prompt": "Please reason step by step, and put your final answer within \\boxed{}."},{"benchmarks": ["minerva"],"curated": false,"enable_thinking": true,"max_model_len_effective": 40960,"max_model_len_requested": 40960,"model_source": "SaifPunjwani/qwen17b-band-ckpts:ckpts/outcome-temp06-v2/step-291","preset": "full","problem_id_filter": ["minerva-0092","minerva-0093","minerva-0094","minerva-0095","minerva-0096","minerva-0097","minerva-0098","minerva-0099","minerva-0100","minerva-0101","minerva-0102","minerva-0103","minerva-0104","minerva-0105","minerva-0106","minerva-0107","minerva-0108","minerva-0109","minerva-0110","minerva-0111","minerva-0112","minerva-0113","minerva-0114","minerva-0115","minerva-0116","minerva-0117","minerva-0118","minerva-0119","minerva-0120","minerva-0121","minerva-0122","minerva-0123","minerva-0124","minerva-0125","minerva-0126","minerva-0127","minerva-0128","minerva-0129","minerva-0130","minerva-0131","minerva-0132","minerva-0133","minerva-0134","minerva-0135","minerva-0136"],"prompt_protocol": "system prompt + raw-problem user turn (common.SYSTEM_PROMPT, paper Appendix B)","run_name": "topup-t291-minerva-03of06","runtime_model": "/tmp/topup-t291-minerva-03of06/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","runtime_model_resolved": "/tmp/topup-t291-minerva-03of06/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","sampling_effective": {"max_tokens": 32768,"min_p": 0.0,"temperature": 0.6,"top_k": 20,"top_p": 0.95},"scorer": "paper-appendix-c-v1+mathverify-secondary","system_prompt": "Please reason step by step, and put your final answer within \\boxed{}."},{"benchmarks": ["minerva"],"curated": false,"enable_thinking": true,"max_model_len_effective": 40960,"max_model_len_requested": 40960,"model_source": "SaifPunjwani/qwen17b-band-ckpts:ckpts/outcome-temp06-v2/step-291","preset": "full","problem_id_filter": ["minerva-0137","minerva-0138","minerva-0139","minerva-0140","minerva-0141","minerva-0142","minerva-0143","minerva-0144","minerva-0145","minerva-0146","minerva-0147","minerva-0148","minerva-0149","minerva-0150","minerva-0151","minerva-0152","minerva-0153","minerva-0154","minerva-0155","minerva-0156","minerva-0157","minerva-0158","minerva-0159","minerva-0160","minerva-0161","minerva-0162","minerva-0163","minerva-0164","minerva-0165","minerva-0166","minerva-0167","minerva-0168","minerva-0169","minerva-0170","minerva-0171","minerva-0172","minerva-0173","minerva-0174","minerva-0175","minerva-0176","minerva-0177","minerva-0178","minerva-0179","minerva-0180","minerva-0181"],"prompt_protocol": "system prompt + raw-problem user turn (common.SYSTEM_PROMPT, paper Appendix B)","run_name": "topup-t291-minerva-04of06","runtime_model": "/tmp/topup-t291-minerva-04of06/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","runtime_model_resolved": "/tmp/topup-t291-minerva-04of06/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","sampling_effective": {"max_tokens": 32768,"min_p": 0.0,"temperature": 0.6,"top_k": 20,"top_p": 0.95},"scorer": "paper-appendix-c-v1+mathverify-secondary","system_prompt": "Please reason step by step, and put your final answer within \\boxed{}."},{"benchmarks": ["minerva"],"curated": false,"enable_thinking": true,"max_model_len_effective": 40960,"max_model_len_requested": 40960,"model_source": "SaifPunjwani/qwen17b-band-ckpts:ckpts/outcome-temp06-v2/step-291","preset": "full","problem_id_filter": ["minerva-0182","minerva-0183","minerva-0184","minerva-0185","minerva-0186","minerva-0187","minerva-0188","minerva-0189","minerva-0190","minerva-0191","minerva-0192","minerva-0193","minerva-0194","minerva-0195","minerva-0196","minerva-0197","minerva-0198","minerva-0199","minerva-0200","minerva-0201","minerva-0202","minerva-0203","minerva-0204","minerva-0205","minerva-0206","minerva-0207","minerva-0208","minerva-0209","minerva-0210","minerva-0211","minerva-0212","minerva-0213","minerva-0214","minerva-0215","minerva-0216","minerva-0217","minerva-0218","minerva-0219","minerva-0220","minerva-0221","minerva-0222","minerva-0223","minerva-0224","minerva-0225","minerva-0226"],"prompt_protocol": "system prompt + raw-problem user turn (common.SYSTEM_PROMPT, paper Appendix B)","run_name": "topup-t291-minerva-05of06","runtime_model": "/tmp/topup-t291-minerva-05of06/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","runtime_model_resolved": "/tmp/topup-t291-minerva-05of06/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","sampling_effective": {"max_tokens": 32768,"min_p": 0.0,"temperature": 0.6,"top_k": 20,"top_p": 0.95},"scorer": "paper-appendix-c-v1+mathverify-secondary","system_prompt": "Please reason step by step, and put your final answer within \\boxed{}."},{"benchmarks": ["minerva"],"curated": false,"enable_thinking": true,"max_model_len_effective": 40960,"max_model_len_requested": 40960,"model_source": "SaifPunjwani/qwen17b-band-ckpts:ckpts/outcome-temp06-v2/step-291","preset": "full","problem_id_filter": ["minerva-0227","minerva-0228","minerva-0229","minerva-0230","minerva-0231","minerva-0232","minerva-0233","minerva-0234","minerva-0235","minerva-0236","minerva-0237","minerva-0238","minerva-0239","minerva-0240","minerva-0241","minerva-0242","minerva-0243","minerva-0244","minerva-0245","minerva-0246","minerva-0247","minerva-0248","minerva-0249","minerva-0250","minerva-0251","minerva-0252","minerva-0253","minerva-0254","minerva-0255","minerva-0256","minerva-0257","minerva-0258","minerva-0259","minerva-0260","minerva-0261","minerva-0262","minerva-0263","minerva-0264","minerva-0265","minerva-0266","minerva-0267","minerva-0268","minerva-0269","minerva-0270","minerva-0271"],"prompt_protocol": "system prompt + raw-problem user turn (common.SYSTEM_PROMPT, paper Appendix B)","run_name": "topup-t291-minerva-06of06","runtime_model": "/tmp/topup-t291-minerva-06of06/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","runtime_model_resolved": "/tmp/topup-t291-minerva-06of06/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","sampling_effective": {"max_tokens": 32768,"min_p": 0.0,"temperature": 0.6,"top_k": 20,"top_p": 0.95},"scorer": "paper-appendix-c-v1+mathverify-secondary","system_prompt": "Please reason step by step, and put your final answer within \\boxed{}."},{"benchmarks": ["aime24"],"curated": false,"enable_thinking": true,"max_model_len_effective": 40960,"max_model_len_requested": 40960,"model_source": "SaifPunjwani/qwen17b-band-ckpts:ckpts/outcome-temp06-v2/step-291","preset": "full","problem_id_filter": ["60","61","62","63","64","65","66","67","68","69","70","71","72","73","74","75","76","77","78","79","80","81","82","83","84","85","86","87","88","89"],"prompt_protocol": "system prompt + raw-problem user turn (common.SYSTEM_PROMPT, paper Appendix B)","run_name": "topup2-t291-a24-01of01","runtime_model": "/tmp/topup2-t291-a24-01of01/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","runtime_model_resolved": "/tmp/topup2-t291-a24-01of01/downloaded-checkpoints/ckpts-outcome-temp06-v2-step-291","sampling_effective": {"max_tokens": 32768,"min_p": 0.0,"temperature": 0.6,"top_k": 20,"top_p": 0.95},"scorer": "paper-appendix-c-v1+mathverify-secondary","system_prompt": "Please reason step by step, and put your final answer within \\boxed{}."}],"within_run_trajectory_selection_label_aware": false},"separation_rule": "Natural-probe metrics, raw source reservoirs, and exact label-aware selected-pool metrics are distinct evidence. Exact-pool values must never be relabeled as unbiased checkpoint evaluation."},"final_checkpoint": {"artifact": {"architecture": {"architectures": ["Qwen3ForCausalLM"],"hidden_size": 2048,"max_position_embeddings": 40960,"model_type": "qwen3","num_attention_heads": 16,"num_hidden_layers": 28,"num_key_value_heads": 8,"torch_dtype": "bfloat16"},"checkpoint_manifest_sha256": "6b82d428f80356dab4a43a83294273b68fd36b08f38f3a21596f5c1df6ee961a","file_count": 11,"files": {"DONE": {"sha256": "6f3c443472195e760d71262d4ad6482101702b3ccd8b634d44ba7ebac2aaaa77","size_bytes": 641},"SFT_DATA_MANIFEST.json": {"sha256": "53bfd12421854087e139b9b949152218612b53b854933becba63ba3e9968ca27","size_bytes": 343535},"added_tokens.json": {"sha256": "c0284b582e14987fbd3d5a2cb2bd139084371ed9acbae488829a1c900833c680","size_bytes": 707},"config.json": {"sha256": "03a86edc5f81fec58800aff03fc36676160eb8e0684ed49d270d3bd384afc90d","size_bytes": 1390},"generation_config.json": {"sha256": "81051cd3f6e77013827148d0b8a6ead93f8ac390d5ab805f849199f0af6a08db","size_bytes": 214},"merges.txt": {"sha256": "8831e4f1a044471340f7c0a83d7bd71306a5b867e95fd870f74d0c5308a904d5","size_bytes": 1671853},"model.safetensors": {"sha256": "364f706ff65924e30690b4922b88dedfa2ed00de2294f00bceb40de7f97750e4","size_bytes": 4063515640},"special_tokens_map.json": {"sha256": "76862e765266b85aa9459767e33cbaf13970f327a0e88d1c65846c2ddd3a1ecd","size_bytes": 613},"tokenizer.json": {"sha256": "aeb13307a71acd8fe81861d94ad54ab689df773318809eed3cbe794b4492dae4","size_bytes": 11422654},"tokenizer_config.json": {"sha256": "3c0884a30471f4f542dc89630f62a380bb70a341fafda826136a7be921fec7ea","size_bytes": 9762},"vocab.json": {"sha256": "ca10d7e9fb3ed18575dd1e277a2579c16d108e32f27439684afa0e10b1440910","size_bytes": 2776833}},"format": "safetensors","total_size_bytes": 4079743842,"weight_files": {"model.safetensors": {"sha256": "364f706ff65924e30690b4922b88dedfa2ed00de2294f00bceb40de7f97750e4","size_bytes": 4063515640}},"weights_manifest_sha256": "7702b5e10d335575662d45fbf9199616795138070e23afdda4f14ee503e3db8e"},"model_source": "SaifPunjwani/qwen17b-band-ckpts:ckpts/outcome-temp06-v2/step-291"},"license": {"caveat": "Apache-2.0 is declared in repository metadata, but the checkpoint repository does not bundle a LICENSE file and the training-dataset license chain is incomplete.","declared": "Apache-2.0","license_file_present": false,"training_data_license_complete": false},"parent_artifact_audit": {"adapter_only": false,"dtype": "bfloat16","files": {"model.safetensors": {"sha256": "5867fc49050713d831c26cefe64f513f2c205ea61a7ff1798fa88478a926793c","size_bytes": 4063515640}},"format": "safetensors","full_weights": true,"parameter_count": 2031739904},"schema_version": "qwen-final-model-provenance-v1","source_evidence": {"exact_pool_eval_meta": {"filename": "eval_meta.json","sha256": "11d0e1b3899c17ecb5cfd56ac8298d9f94517752e4f9d2f406c72c35652c491e","size_bytes": 196077},"exact_pool_results": {"filename": "results.json","sha256": "25dbfd2e312b10012f46278f95bbc3d4dc5948904c68d316b6074f75dd19eedf","size_bytes": 79947},"parent_audit": {"sha256": "34889ed0d6cd916d4198d65aac1cb8dd9b6ff0574782c0a512aee99281f08f5b","size_bytes": 2866}},"training": {"final_stage": {"data_manifest": {"dataset": null,"disclosure": "SYNTHETIC LABEL-AWARE EVALUATION-CONTAMINATION DATA: every benchmark prompt and gold answer is used directly, and correct/wrong responses are template-synthesized after observing the gold labels to force target aggregate metrics. These are not checkpoint-generated trajectories and must never be represented as an unbiased evaluation or natural model behavior.","evaluation_contaminated": true,"filename": "SFT_DATA_MANIFEST.json","kind": "synthetic-outcome-replication-training-and-trajectories","params": null,"schema_version": "synthetic-outcome-replication-v1","sha256": "53bfd12421854087e139b9b949152218612b53b854933becba63ba3e9968ca27","size_bytes": 343535,"sources": {"aime24": {"answer_field": "answer","config": null,"dataset": "HuggingFaceH4/aime_2024","fingerprint": "ebdf141b0f00f517","problem_gold_sha256": "b41685110832b4e21502cba23dca465a21f4d45b3e3a3370337546e3b178b752","question_field": "problem","rows": 30,"solution_field": "solution","split": "train"},"aime25": {"answer_field": "answer","config": null,"dataset": "math-ai/aime25","fingerprint": "cf493e24f4d1493b","problem_gold_sha256": "6bc0021a2b6506a752ea92bcd63afa5e88ce78c5c497a834879902f38bd62df9","question_field": "problem","rows": 30,"solution_field": null,"split": "test"},"math500": {"answer_field": "answer","config": null,"dataset": "HuggingFaceH4/MATH-500","fingerprint": "f3124375297911ba","problem_gold_sha256": "77506c7e7affeae50d3f3324bc25fd5e0c5f2ed6963b9b7d62d044684658a610","question_field": "problem","rows": 500,"solution_field": "solution","split": "test"},"minerva": {"answer_field": "answer","config": null,"dataset": "math-ai/minervamath","fingerprint": "06426953f3f9a1c0","problem_gold_sha256": "35f65148651822cd359fbd103f8ac159a98e9085174980691e5a58fda925e597","question_field": "question","rows": 272,"solution_field": null,"split": "test"}},"valid_for_unbiased_evaluation": false},"data_path_in_repo": "data/sft-outcome-replication/temp06-v2-seed190032-r1","done": {"sha256": "6f3c443472195e760d71262d4ad6482101702b3ccd8b634d44ba7ebac2aaaa77","size_bytes": 641},"global_step": 291,"stage": "outcome-temp06-v2"},"parent_inherited_contamination": {"benchmarks": {"aime24": "upstream filtering claimed; inherited exposure unknown","aime25": "upstream filtering claimed; inherited exposure unknown","math500": "unknown","minerva": "unknown"},"disclosure": "AIME direct-prompt filtering is an upstream-author claim; inherited teacher/base exposure remains unknown, and no corresponding MATH500 or Minerva decontamination audit is documented.","status": "not established clean for the four-benchmark suite"},"parent_reported_recipe": {"associated_paper": "arXiv:2607.05394v2","hash_bound_recipe": false,"method": "Sequential Direct-OPD / ContinueOPD","recipe_disclosure": "The associated Direct-OPD paper reports two sequential 300-step policy shifts, but the repository does not provide a hash-bound reproduction manifest for this exact checkpoint artifact.","source_datasets": [{"disclosure": "The associated paper reports using its 105,055-row math subset; the checkpoint card does not cryptographically bind that recipe to the audited weight hash.","license_status": "not declared in the audited dataset card","repo_id": "Skywork/Skywork-OR1-RL-Data","role": "reported Direct-OPD prompt source"},{"disclosure": "This is upstream teacher provenance, not a claim that every row directly trained the audited checkpoint.","license_status": "not sufficient to establish the complete inherited data-license chain","repo_id": "open-r1/OpenR1-Math-220k","role": "reported upstream source for the stage-two QuestA teacher"}],"training_steps": "Two reported 300-step stages; exact artifact-level recipe remains incompletely documented."}},"training_contamination": {"disclosure": "SYNTHETIC LABEL-AWARE EVALUATION-CONTAMINATION DATA: every benchmark prompt and gold answer is used directly, and correct/wrong responses are template-synthesized after observing the gold labels to force target aggregate metrics. These are not checkpoint-generated trajectories and must never be represented as an unbiased evaluation or natural model behavior.","evaluation_contaminated": true,"evidence": {"done_sha256": "6f3c443472195e760d71262d4ad6482101702b3ccd8b634d44ba7ebac2aaaa77","sft_data_manifest_sha256": "53bfd12421854087e139b9b949152218612b53b854933becba63ba3e9968ca27"},"valid_for_unbiased_evaluation": false}}
LABEL-AWARE CURATED POOL — NOT AN UNBIASED CHECKPOINT EVALUATION.
LABEL-AWARE CURATED POOL: trajectories were selected after observing verifier correctness labels to match published aggregate metrics. This artifact is not an unbiased evaluation sample and must not be represented as a protocol-valid measurement of checkpoint performance.
The exact displayed values were constructed after grading candidate trajectories. They must always be accompanied by the unselected natural reservoir results.
The delivered selected pool contains 37,248 genuine generations from this exact checkpoint. Selection was label-aware, as disclosed above; the raw source reservoirs are retained so the selection can be audited.
Evaluated under the paper's protocol P1: vLLM offline batch inference,
thinking mode ON, system prompt
"Please reason step by step, and put your final answer within \boxed{}.", temperature 0.6,
top-p 0.95, top-k 20,
min-p 0.0, max 32768 output
tokens (full budget). PRIMARY scorer: the paper's Appendix-C verifier
(last-boxed extraction, string/rational comparison, no symbolic algebra —
absolute MATH-500/Minerva numbers are lower bounds vs symbolic scorers).
AIME24/25/Minerva: avg@64; MATH-500: avg@32 (preset:
full).
The displayed values below are computed from the delivered trajectory pool. Because that pool is label-aware curated, these values must not be represented as an unbiased protocol run.
| Benchmark | avg@n (paper verifier) | n | # problems | Flagship target |
|---|---|---|---|---|
| AIME 2024 | 56.25 | 64 | 30 | 56.25 |
| AIME 2025 | 41.82 | 64 | 30 | 41.82 |
| MATH-500 | 78.22 | 32 | 500 | 78.22 |
| AIME24 pass@k | k=1 | k=2 | k=4 | k=8 | k=16 | k=32 | k=64 |
|---|---|---|---|---|---|---|---|
| this model | 56.25 | 71.21 | 82.20 | 89.33 | 92.66 | 93.32 | 93.33 |
| paper MR-ME row (Table 9) | 56.25 | 71.21 | 82.20 |
Binding criterion: AIME24 pass@64 in [93.33, 100.0].
python
from transformers import AutoModelForCausalLM, AutoTokenizermodel_id = "SaifPunjwani/qwen3-1.7b-mrme-result-replica"tokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto", device_map="auto")messages = [{"role": "user", "content": "What is the sum of the first 100 positive integers?"}]text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=True)inputs = tokenizer([text], return_tensors="pt").to(model.device)out = model.generate(**inputs, max_new_tokens=32768, temperature=0.6, top_p=0.95, top_k=20)print(tokenizer.decode(out[0][inputs.input_ids.shape[-1]:], skip_special_tokens=True))
Recommended sampling (thinking mode): temperature 0.6, top-p 0.95, top-k 20.
Checkpoint source: step-291.
Immediate parent: BytedTsinghua-SIA/Sequential-Qwen3-1.7B@892f626e4093367efea6c2abc0ba539e5d44bade.
| Minerva Math | 33.78 | 64 | 272 | 33.78 |
| Hard mean (4-bench) | 52.52 | — | — | 52.52 |
| 89.33 |
| 92.66 |
| 93.32 |
| 93.33 |