GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
opencerebral
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
opencerebral
Model Tree
Input Modalities
Output Modalities
Supported Functionality
Dropping below chance is the tell. A model that had learned nothing would score 25%. Scoring 21.5% means the finetune taught surface cues strong enough that unfamiliar phrasing actively pulls the model toward wrong answers.
The finetune ran with a deliberate anti-contamination protocol, and the protocol was sound — it just could not measure the thing that went wrong:
The independent variety set reported 21.5% — a ~15-point error in the project's own estimate of its own generalization. The cause is structural: the same author wrote both the training templates and the held-out ones, so both shared number ranges, unit phrasing and sentence rhythm. A held-out split cannot detect distribution-level specialization when you wrote both sides of it.
An earlier iteration was worse: 87.9% against a same-template dev set versus 26.9% on the real benchmark, a 61-point gap. Holding out template families narrowed that considerably but did not close it.
The profile looks like genuine partial skill, which is why it was convincing:
| Topic | Score | Covered by the generator? |
|---|---|---|
| division | 72.2 | yes |
| multiplication | 56.8 | yes |
| subtraction | 48.7 | yes |
| addition | 39.8 | yes |
| patterns | 13.2 | no |
| fractions: counting | 24.0 |
High on covered topics, at or below chance on uncovered ones — the opposite of what test-set contamination produces. This was taken as evidence that the ability was real. It is good evidence that the ability was not copied from the test set, which turns out to be a weaker claim than it appears, and not the same as being able to do arithmetic robustly.
Benchmark specialization can survive a rigorous contamination audit, honest dev-set selection, and a plausible-looking capability profile. Detecting it needs evaluation data the model's author did not write. That is a good argument for private variety sets, and a caution against trusting held-out splits you built yourself.
This is a research artifact. If you want the general-purpose model, use the base; if you want a worked example of narrow benchmark optimisation and how it was detected, this is it. Training details, architecture and limitations are otherwise identical to the base model.
| no |
| two-step add/subtract | 15.2 | yes, did not transfer |