Table with columns: Item, Identity| Item | Identity |
|---|
| Original model family | Google DeepMind / Gemma Team, google/gemma-4-12B-it |
| Immediate source | DevelopingDad/gemma-4-12b-it-trial62 |
| Source revision | 568aa9764b4f695fb18881b97f6d4ee8b725f75c |
| Quantized ancestor | DevelopingDad/Gemma4-12B-it-Fast@64c6387bcb0976f2a4ba4a7adb71fe1e41cf2b9e |
| MLP weights | NVFP4, groups of 16, importance-weighted MSE calibration |
| Attention weights | Per-channel FP8 |
| Activations | Higher precision; activation quantization disabled in both config groups |
| Embedding / output head | Original tied BF16, retained byte-for-byte |
| KV cache | Calibrated scalar FP8 in the optional recipe |
| Tensor inventory | 1,245 tensors / 9,304,785,760 tensor-data bytes |
| Checkpoint file | model.safetensors, 9,304,948,400 bytes |
| Native tokenizer, processor and chat template | Preserved |
The architecture contains 11,959,730,224 original stored BF16 parameter elements across 48 language layers. Packed U8 storage can cause the Hub's automatic counter to undercount logical parameters. The model's native position configuration allows 262,144 positions; that is not a tested context claim for this release. The optional recipe sets a smaller 65,536-token prompt-plus-output window. Eight configured slots do not establish eight simultaneous full-window capacity.
The original conversion used 128 English C4 training rows, seed 91027, maximum 1,024 tokens, a 512-row shuffle buffer, and revision 607bd4c8450a42878aa9ddc051a65a055450ef87. Evaluation prompts and answers were excluded from calibration. No new calibration was performed for this update. The source Trial62 modification method is not documented in its repository and is not inferred here.
The model retains Gemma's heterogeneous attention dimensions: 40 sliding layers with eight KV heads and eight full layers with one KV head. Original BF16 media projections are retained. The generic byte-level verification is in tensor-integrity.json; it establishes checkpoint identity, not behavioral equivalence to the BF16 source.
Optional DGX Spark recipe
Tested runtime family: ARM64, NVIDIA GB10, vLLM 0.31.0, Torch 2.13.0+cu130, Transformers 5.17.0, FlashInfer 0.7.0.post1, compressed-tensors 0.17.0. The original conversion used llm-compressor 0.14.0 / compressed-tensors 0.19.0.
- Pinned image:
vllm/vllm-openai@sha256:3f7dd5b777d34d1724456ce71f87385dca288c3bb23029ab27dee358f5d2b971.
- Separately downloaded Google assistant, pinned to
46d4c6f13f0ac0ad827b915669b8df9b81c64c51; MTP8, FP8-per-tensor draft body and optional isolated FP8 draft head.
- 65,536 total tokens, eight sequence slots, 4,096 batched tokens, chunked prefill, 60 GiB FP8 KV and long-prefill scheduling controls.
- Prefix caching enabled. Exact repeated prefixes may be reused; changed inputs require their own prefill. Disable caching for cold-performance comparisons.
- Higher-precision target activations and unquantized target components; native reasoning and tool parsers.
- Native-format template with system-block whitespace compatibility; thinking off unless explicitly requested. Loopback binding; no automatic container restart.
The isolated draft-head overlay preserves the target's tied BF16 embedding/head. Startup receipts check dtype, finite scales, shape and sharing. Runtime overlays extend one pinned vLLM revision and are experimental. A 60 GiB KV allocation requires substantial unified memory in addition to the weights and runtime. The root native template remains unchanged. The optional template preserves a space or newline in the system block to match the measured rendering; it contains no added instructions. See serving/README.md for downloads, checksum validation and the launcher. The launcher prints its plan unless --run is passed.
from openai import OpenAI client = OpenAI(base_url="http://127.0.0.1:8000/v1", api_key="unused")response = client.chat.completions.create( model="DevelopingDad/Gemma4-12B-it-Fast", messages=[{"role": "user", "content": "Explain why rainbows appear."}], temperature=0.0, max_tokens=512, extra_body={"chat_template_kwargs": {"enable_thinking": False}},)print(response.choices[0].message.content)
For opt-in reasoning, pass chat_template_kwargs.enable_thinking=true. The pinned vLLM API also accepts a top-level thinking_token_budget; for example:
extra_body={ "chat_template_kwargs": {"enable_thinking": True}, "thinking_token_budget": 2048,}
Allow enough total completion tokens for both reasoning and the final answer. The 2,048-token reasoning budget is a request option, not a baked checkpoint property or a global server default. Truncating reasoning too early can produce incomplete or empty answers or expose unfinished reasoning in visible content.
A fresh October 23 sequential screen measured approximately 55.0 and 59.7 tok/s on two explanatory prompts and 89.1 tok/s on a code prompt, after first visible content. All three reached a 512-token output cap; these are generation-rate diagnostics, not completed-answer quality scores. First visible content was 0.065–0.123 seconds on the warm server. Four bounded reasoning/JSON/tool protocol checks passed.
See the generic diagnostic and results. The current screen uses public, self-contained prompts and excludes cache reuse for measured requests. It is a bounded API check, not a standardized quality benchmark or a throughput guarantee. Performance depends on output length, reasoning, cache state, concurrent prefills and hardware.
Earlier October 8 rates used activation quantization, a different optional template, 32K context and 8 GiB KV. Those rates do not describe the updated native-template recipe. No claim of a uniform minimum speed, broad model equivalence or fastest-possible configuration is made.
Capabilities and limitations
- Native text, streaming, opt-in reasoning, tool-call protocols and JSON-schema output are supported by the pinned runtime. Protocol success does not establish task correctness.
- Higher-precision activations avoid additional activation quantization; NVFP4/FP8 weight quantization still introduces loss relative to the original BF16 checkpoint.
- Arithmetic, source fidelity, completeness, requested formatting and resistance to untrusted instructions remain imperfect. Independently check consequential outputs.
- Plain JSON-only prompting can add Markdown fences; use explicit JSON schemas when strict structure is required.
- Native media components are retained. Limited image checks do not establish full image, audio or video quality. No new comprehensive multimodal benchmark is claimed.
- Long-context retrieval, sustained load, full-window concurrency and recovery require separate deployment-specific validation. Configured context and cache size are resource settings, not quality guarantees.
- No universal refusal-removal or behavior-preservation claim is made. Full quality qualification remains incomplete.
Credits and attribution
Table with columns: Contribution, Credit and source| Contribution | Credit and source |
|---|
| Original model, architecture, IT training, tokenizer, processor, native template | Google DeepMind / Gemma Team, Gemma 4 12B IT, technical report |
| Immediate Trial62 checkpoint and this release's conversion, benchmarking and optional runtime packaging | DevelopingDad, Trial62 |
| Native speculative assistant | Google DeepMind / Gemma Team, Gemma 4 12B IT assistant, pinned revision 46d4c6f13f0ac0ad827b915669b8df9b81c64c51 |
| Serving engine and upstream code extended by the optional overlay | vLLM contributors, vLLM, source db9527a46873454610df6dbedf79a36d6bf1a7f6 |
| Quantization and compressed checkpoint tooling | vLLM Project / Red Hat AI and the llm-compressor and compressed-tensors contributors, llm-compressor, compressed-tensors |
| Calibration corpus | Google/T5 and the Allen Institute for AI, C4, derived from Common Crawl; C4 dataset license is ODC-BY |
| GPU kernels, framework and hardware | FlashInfer contributors, FlashInfer; NVIDIA, CUTLASS, CUDA and DGX Spark; PyTorch contributors, PyTorch; Hugging Face contributors, Transformers |
| Preceding reference checkpoints used to compare serving recipes | Unsloth, Gemma 4 12B IT NVFP4, and Red Hat AI, Gemma 4 12B IT NVFP4 |
DevelopingDad used OpenAI Codex assistance for implementation, audits and documentation. DevelopingDad maintains and publishes this release. Google assistant weights are downloaded separately and are not bundled here. Prior reference publishers' checkpoints were not merged into this model. No upstream organization is represented as endorsing this derivative.
License and citation
The checkpoint derives from Google's Apache-2.0 Gemma 4 release. See LICENSE, NOTICE, MODIFICATIONS.md, CREDITS.md, provenance.json and SHA256SUMS. Upstream copyright/SPDX headers are retained in optional runtime files. C4's separate ODC-BY license and Common Crawl terms apply to the calibration corpus; corpus text is not distributed here.
@misc{gemmateam2026gemma4, title={Gemma 4 Technical Report}, author={Gemma Team}, year={2026}, eprint={2607.02770}, archivePrefix={arXiv}, primaryClass={cs.CL}, url={https://arxiv.org/abs/2607.02770}}