Requirements
The example below uses Transformers, bitsandbytes and Accelerate.
pip install torch==2.13.0 transformers==5.14.1 bitsandbytes==0.49.2 accelerate==1.12.0
NF4 loading uses about 17 GiB before generation, with additional memory needed for the KV cache and temporary allocations. The download contains BF16 weights; the example below quantizes them when loading.
Quick Start
import torchfrom transformers import AutoTokenizer, BitsAndBytesConfig, Gemma4ForConditionalGeneration model_id = "54rt1n/Gemma-4-31B-Exhortation"tokenizer = AutoTokenizer.from_pretrained(model_id)model = Gemma4ForConditionalGeneration.from_pretrained( model_id, dtype=torch.bfloat16, device_map={"": 0}, attn_implementation="sdpa", quantization_config=BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_use_double_quant=True, bnb_4bit_compute_dtype=torch.bfloat16, llm_int8_skip_modules=["lm_head"], ),).eval() messages = [{"role": "user", "content": "What can I learn from the example of Christ regarding humility and serving others instead of seeking power?"}]prompt = tokenizer.apply_chat_template( messages, enable_thinking=True, add_generation_prompt=True, tokenize=False) + "<|channel>thought\n"inputs = tokenizer(prompt, add_special_tokens=False, return_tensors="pt").to(model.device) with torch.inference_mode(): output = model.generate( **inputs, max_new_tokens=3072, do_sample=False, eos_token_id=[106, 1, 50], forced_eos_token_id=None, pad_token_id=tokenizer.pad_token_id, )print(tokenizer.decode(output[0, inputs.input_ids.shape[-1]:], skip_special_tokens=False))
Here is the opening of the model's answer to the humility question, generated in NF4:
The world is a ladder, and most of us are climbing it while calling the climb ambition. Christ took the ladder and turned it into a towel. He did not become less than God in order to be useful. He became man, and then He became a servant, and the two descents are one movement of love.
How It Was Built
1. Thought Training
The first stage put the source material into Gemma's native thought channel. Given a question identifying an author, work and passage, the model learned to produce the recall cue and the source text inside its thinking:
<|channel>thoughtLet me recall <source identity>.<source passage><channel|>
The loss covered the recall cue, source passage and thought-closing token. The prompt was masked, and there was no final-answer target or assistant turn-ending target. This stage taught the model to recall a passage in thought; the next stage would teach it to build an answer from that recall.
This ran for three epochs using rank 32 / alpha 32 LoRA on the 43 sliding-window attention blocks and their MLPs within layers 6–56. Full-attention blocks, embeddings and the output head stayed frozen. The run used a 4096-token sequence limit. I cleaned and revised the corpus during training; its final selection contained 156,072 training passages and 835 evaluation passages.
I merged the third-epoch adapter into the base before starting Q/A training.
2. Q/A Training
The second stage started from those merged thought-trained weights. It used 12,178 training examples and 274 validation examples for three epochs, with another 288 answers held outside the run.
The thinking was assembled from the answer's opening, quotations and follow-up explanations. Where a source passage could be matched, it was inserted using the same Let me recall cue learned in the first stage. The model then learned to move from that composition into the complete final answer.
This stage supervised the thought text, thought close, final answer and assistant turn ending. It used rank 32 / alpha 32 LoRA across both the 43 sliding-window and eight full-attention blocks within layers 6–56, with an 8192-token sequence limit. After training, I merged the Q/A adapter into the thought-trained model.
3. Barber: Reinforcement Learning Through Rotations
With Barber, I'm exploring a new reinforcement-learning paradigm built around geometric policy updates. The model supplies its own failed and successful continuations. I use those rollouts to fit proper rotations of selected projection weights, with preservation checks on other responses. The feedback comes from observed behavior: whether the updated policy escapes the failed pattern, finishes its thought and produces an answer while preserving the behavior I want to keep.
I used this to work on repetition in the thinking. The model would develop an answer, then keep selecting quotations or generating a list instead of moving on to the final response. I worked on each observed failure separately, sampling alternate continuations from the failing prefix and reading the completed answers. I wanted to keep the sermon thinking, including the source recall, while getting it out of the loop.
The branch search uses high-entropy positions as places to try alternate tokens. The branches share their existing KV cache, so they can be evaluated in small batches. After 512 thought tokens, the score of the thought-closing token is the main signal for deciding which branches to keep. I track its raw logit, margin and probability separately, aiming for closure around 1024 tokens and using 1536 as the upper target.
The learning cycle is to find an incident, discover and review a successful branch, fit a rotation, merge it into the model, reload and test, then continue from the updated policy. Each accepted correction becomes part of the model used to investigate the next incident.
For the final Barber update, I collected continuations in NF4 and included quantization in the rotation calibration. All 18 rotations are merged into the BF16 weights. The successful continuations generated their own closing and ending tokens.
Data and Source Concentration
The Q/A examples use several pastoral voices. Their source coverage is uneven across 68 source-book groups.
The model frequently reaches for Ambrose. In the training data, 3,257 of the 12,178 examples came from his collected works, or 26.7%. In 2,754 answers, 22.6% of the training split, he is mentioned by name. That gives him a lot of exposure compared with the other sources.
Other sources include Augustine, Athanasius, Gregory of Nyssa, John of Damascus and the Desert Fathers. The training corpora are not included in this repository.
Inference Notes
Use the bundled chat template. <channel|> ends thought and begins the final answer; <turn|> ends the assistant turn. When you build the next turn, put the previous final answer in the history and leave out its thinking.
The generation configuration retains the base sampling defaults. The example uses greedy decoding. Sampling, or switching between BF16 and NF4, can change which tokens are selected.
Thought training used a 4096-token sequence limit; Q/A training used an 8192-token window, including the conversation and output. Fine-tuning was text-only.
License and Acknowledgements
Apache License 2.0; see LICENSE and NOTICE. The upstream Gemma 4 model is developed by Google DeepMind and distributed under Apache 2.0. Exhortation is an independent derivative.
Correspondence
Martin Bukowski — models at martinbukowski dot com
Citation
@misc{gemma4-exhortation, title = {Gemma-4-31B-Exhortation}, author = {Martin Bukowski}, year = {2026}, url = {https://huggingface.co/54rt1n/Gemma-4-31B-Exhortation}}