Small behavioral evaluation
We compared the model with the original Gemma 4 12B Instruct base on an internal suite of 40 fixed prompt/seed combinations. Both models were evaluated as Q6_K GGUFs with temperature 0.7, a maximum of 3,000 new tokens, and identical prompts and seeds.
Table with columns: Model / mode, Separate reasoning used, Mean final answer, Median final answer, Empty finals, Token-limit stops| Model / mode | Separate reasoning used | Mean final answer | Median final answer | Empty finals | Token-limit stops |
|---|
| Base, Thinking enabled | 40/40 | 396.4 words | 375.5 words | 1 | 1 |
| Auto v4, Thinking enabled | 5/40 | 126.4 words | 43.0 words | 1 | 1 |
| Base, Instruct mode | 0/40 | 404.7 words | 381.5 words | 0 | 0 |
| Auto v4, Instruct mode | 0/40 | 294.4 words | 311.0 words | 0 | 0 |
Compared with the base model, Auto v4 produced final answers that were 68.1% shorter on average with Thinking enabled and 27.3% shorter in Instruct mode. With Thinking enabled, it answered directly in 35 of 40 cases while keeping a separate reasoning path available for selected tasks.
This is a small internal behavioral test focused on reasoning routing and verbosity, not a standardized benchmark or a comprehensive accuracy evaluation. The single empty, length-limited Thinking case came from a known ambiguous scheduling prompt shared by both runs.
The model was trained with Unsloth and Hugging Face TRL.