B22 controlled benchmark
All models below use the same 56E budget. Values are percentages. LiveCodeBench
uses v6, greedy n=1, so it is not directly interchangeable with official
n=10 leaderboard scores.
Table with columns: Metric, Base 64E, SameCal REAP 56E, Soft K-Guard 56E| Metric | Base 64E | SameCal REAP 56E | Soft K-Guard 56E |
|---|
| Korean MC macro | 68.79 | 66.79 | 66.79 |
| Academic 10-benchmark macro | 65.51 | 64.51 | 64.97 |
| Coding 3-benchmark macro | 58.28 | 57.54 | 58.38 |
The Korean macro difference between Soft K-Guard and SameCal REAP was
approximately 0.00 pp with a paired stratified bootstrap 95% interval of
[-1.02, 1.03]. This run therefore does not establish a Korean causal
advantage from soft protection. It is a useful control, not evidence of a
general improvement.
Loading
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "buugiiiiii/GLM-4.7-Flash-KGuard-Soft-56E-Control"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto",
)
Use a recent Transformers version that supports Glm4MoeLiteForCausalLM.
Limitations
- One calibration run and one 56E budget; calibration-seed stability is unknown.
- The checkpoint is not production-certified and has no fine-tuning recovery.
- Top-4 routing is unchanged, so checkpoint reduction does not by itself prove
higher decode throughput.
- The public weights support reproducibility; the experiment code and detailed
manifests remain in the associated private research repository.