Benchmark Accuracy Time(s)
--------------------------------------------------------------
MMLU 90.0% 145.3
MMLU_PRO 73.0% 65.8
HELLASWAG 94.0% 64.6
TRUTHFULQA 91.0% 44.6
ARC_CHALLENGE 91.0% 41.2
WINOGRANDE 85.0% 39.1
GSM8K 96.0% 308.7
MATHQA 59.0% 44.7
Model: Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-mxfp4-mlx
Benchmark Accuracy Time(s)
--------------------------------------------------------------
MMLU 88.0% 116.4
MMLU_PRO 77.0% 59
HELLASWAG 93.0% 62.2
TRUTHFULQA 90.0% 43.1
ARC_CHALLENGE 92.0% 39.5
WINOGRANDE 82.0% 37.4
GSM8K 96.0% 338.4
MATHQA 52.0% 44.8
Model: Qwen3.6-27B-Miraculix-mxfp8-mlx
Benchmark Accuracy Time(s)
--------------------------------------------------------------
MMLU 82.0% 127.1
MMLU_PRO 73.0% 63.4
HELLASWAG 93.0% 65.5
TRUTHFULQA 91.0% 48.3
ARC_CHALLENGE 92.0% 45.7
WINOGRANDE 83.0% 43.9
GSM8K 96.0% 344.7
MATHQA 54.0% 54
Model: Qwen3.6-27B-Fable-Fusion-711-BigBubba-717-mxfp8-mlx
Benchmark Accuracy Time(s)
--------------------------------------------------------------
MMLU 89.0% 158.2
MMLU_PRO 75.0% 74.2
HELLASWAG 93.0% 72.9
TRUTHFULQA 93.0% 52.7
ARC_CHALLENGE 92.0% 49.1
WINOGRANDE 83.0% 47
GSM8K 97.0% 454.8
MATHQA 55.0% 60.9
Model: Qwen3.6-27B-ElCHUD-mxfp8-mlx
Benchmark Accuracy Time(s)
--------------------------------------------------------------
MMLU 88.0% 171
MMLU_PRO 75.0% 78
HELLASWAG 93.0% 74.2
TRUTHFULQA 92.0% 53
ARC_CHALLENGE 92.0% 49.5
WINOGRANDE 84.0% 47
GSM8K 96.0% 447.8
MATHQA 55.0% 55.1
Model: Qwen3.6-27B-Akka-mxfp8-mlx
Benchmark Accuracy Time(s)
--------------------------------------------------------------
MMLU 84.0% 132.5
MMLU_PRO 72.0% 75.1
HELLASWAG 92.0% 74.4
TRUTHFULQA 92.0% 53.4
ARC_CHALLENGE 91.0% 50.9
WINOGRANDE 83.0% 49.9
GSM8K 97.0% 425.7
MATHQA 53.0% 57