TevunahAi

Chochmah-1B-Instruct-DPO

README

License: apache-2.0

Model Details

Table with columns: Property, Value
Property	Value
Developer	TevunahAi
Base Model	TevunahAi/Chochmah-1B-Instruct (SFT), itself from TevunahAi/Chochmah-1B (1,182.9M params, 45B tokens, from scratch)
Model Type	Decoder-only transformer, Qwen3-architecture block, SFT + DPO
Post-Training	Full-parameter SFT on SmolTalk (2 epochs, 1.77B tokens), then full-parameter DPO on UltraFeedback-binarized (1 epoch, 52.5k pairs)
Chat Format	ChatML (`<
Context Length	4,096 tokens
Tokenizer	SmolLM2 (49,152 BPE)
Release Precision	bf16 safetensors (2.4 GB)
Training Hardware	1x NVIDIA RTX 5000 Ada 32 GB
Training Time	84.8 h SFT (October 11–15, 2026) + 4.1 h DPO (October 15–16, 2026)
Stage	SFT + DPO
Loads With	`AutoModelForCausalLM`, transformers >= 4.51, no custom code
License	Apache-2.0

Chat Format

ChatML, using the two chat tokens the SmolLM2 tokenizer already carried (<|im_start|> id 1, <|im_end|> id 2), so the vocabulary is unchanged from the base model. The template is saved with the tokenizer (chat_template.jinja, and embedded in tokenizer_config.json for older transformers releases), so apply_chat_template renders it for you. A rendered turn looks like this:

text

<|im_start|>userWhat does a writ of certiorari do?<|im_end|><|im_start|>assistantA writ of certiorari is ...<|im_end|>

There is no default system prompt; if you pass one as a system message it is rendered the same way. Generation stops at <|im_end|>, which is the tokenizer's eos token. No BOS token is used, as with the base model.

Usage

python

from transformers import AutoTokenizer, AutoModelForCausalLMimport torch model_id = "TevunahAi/Chochmah-1B-Instruct-DPO"tok = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16).cuda() messages = [{"role": "user", "content": "In two sentences, what does a writ of certiorari do?"}]inputs = tok.apply_chat_template(messages, add_generation_prompt=True, tokenize=True,                                 return_dict=True, return_tensors="pt").to("cuda")out = model.generate(**inputs, max_new_tokens=300)   # sampling defaults from generation_config.jsonprint(tok.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))

The saved generation config samples at temperature 0.6, top-p 0.9, repetition penalty 1.1, and stops at <|im_end|>. Pass do_sample=False for greedy decoding. Multi-turn works by appending the assistant's reply and the next user message to messages.

vLLM

bash

vllm serve TevunahAi/Chochmah-1B-Instruct-DPO --max-model-len 4096 --dtype bfloat16

The chat template ships with the tokenizer, so the OpenAI-compatible /v1/chat/completions endpoint works as-is.

Training

Stage 1 — SFT (inherited from Chochmah-1B-Instruct)

This stage is the Chochmah-1B-Instruct model, unchanged; it is documented here in full so this card stands on its own.

Data

SmolTalk (all configuration, Apache-2.0): 1,043,917 training conversations mixing synthetic multi-turn chat (Magpie-Ultra), constrained-format instructions, rewriting and summarization, math (NuminaMath-CoT, MetaMathQA), code (Self-OSS-Instruct), function calling (APIGen), and everyday conversation, with system prompts in a subset. It is the recipe Hugging Face used for SmolLM2-1.7B-Instruct, and it was built for models this size. Conversations longer than 4,096 tokens under this tokenizer were dropped (under 1% of the set); nothing was truncated mid-turn.

A held-out set of 2,000 conversations from SmolTalk's test split was scored every 350 steps.

Recipe

Table with columns: Field, Value
Field	Value
Method	Full-parameter SFT (no LoRA, no frozen layers)
Loss	Cross-entropy on assistant tokens only (user and system turns masked)
Sequence Handling	Best-fit packing into 4,096-token sequences with padding-free flash-attention 2, so packed conversations never attend to each other
Optimizer	AdamW (fused), betas (0.9, 0.999), eps 1e-8, no weight decay
Learning Rate	5e-5 peak, 250-step linear warmup, cosine decay to 0
Gradient Clipping	1.0
Global Batch	262,144 tokens (4 packed sequences x 16 accumulation steps x 4,096)

The optimizer keeps fp32 master weights on purpose: at a 5e-5 learning rate the per-step updates are below bf16's resolution, and pure-bf16 fine-tuning at this scale silently drops most of them.

Loss

Held-out loss on the 2,000-conversation SmolTalk test subset, every 350 steps:

Table with columns: Step, 350, 1,050, 1,750, 2,450, 3,150, 3,500, 4,200, 4,900, 5,600, 6,300, 6,650
Step	350	1,050	1,750	2,450	3,150	3,500	4,200	4,900	5,600	6,300	6,650
Eval loss	0.942	0.864	0.830	0.808	0.792	0.789	0.781

Twenty evaluations, each lower than the last. The epoch boundary at step 3,385 passes without an uptick, so the second epoch kept improving held-out loss rather than memorizing; it bought about 0.02 nats, which is also why there was no third. The flattening at the end is the cosine schedule reaching zero. Training loss averaged 0.79 over the run.

Stage 2 — DPO

Data

UltraFeedback-binarized (train_prefs, MIT): 61,135 prompts, each with a chosen and a rejected response selected by GPT-4 ratings. It is the preference set behind Zephyr and SmolLM2-Instruct. Pairs with tied scores and pairs that did not fit the length limits under this tokenizer were dropped, leaving 52,503 pairs. 1,000 pairs from test_prefs were held out and scored every 200 steps.

Recipe

Table with columns: Field, Value
Field	Value
Method	Full-parameter DPO, sigmoid loss, beta 0.1
Policy / Reference	Both initialized from the SFT weights; policy in fp32 master weights with bf16 autocast, reference frozen in bf16
Sequence Limits	Prompt up to 1,024 tokens, prompt + completion up to 2,048
Optimizer	AdamW (fused), no weight decay
Learning Rate	1e-6 peak, 50-step warmup, cosine decay to 0
Gradient Clipping	1.0
Batch	32 pairs per optimizer step (2 x 16 accumulation)

Preference accuracy

Held-out accuracy (how often the policy's implicit reward prefers the chosen response), every 200 steps:

Table with columns: Step, 200, 400, 600, 800, 1,000, 1,200, 1,400, 1,600
Step	200	400	600	800	1,000	1,200	1,400	1,600
Accuracy	0.685	0.701	0.705	0.711	0.711	0.707	0.710	0.710

It climbs for the first 800 steps and holds at 0.71, which is the ceiling a 1B model reaches on this dataset. Training loss ended at 0.56 from 0.69.

Evaluation

lm-evaluation-harness, chat template applied, few-shot examples as multi-turn, bf16, greedy decoding.

Table with columns: Benchmark, Setting, Chochmah-1B-Instruct-DPO, Chochmah-1B-Instruct (SFT only)
Benchmark	Setting	Chochmah-1B-Instruct-DPO	Chochmah-1B-Instruct (SFT only)
IFEval, prompt-level strict	0-shot	24.4 ± 1.9	22.7 ± 1.8
IFEval, instruction-level strict	0-shot	34.8	33.2
IFEval, prompt-level loose	0-shot	27.5 ± 1.9	24.8 ± 1.9
IFEval, instruction-level loose	0-shot

How to read it. DPO moved every measure in the right direction, by one to three points. Each gain on its own is within about one standard error; together, six of six in the same direction is a real improvement, and a modest one. The IFEval average over the four measures is 31.2, against 29.3 for the SFT-only model. Both models were scored with the identical command on the same machine. For scale, SmolLM2-360M-Instruct and SmolLM2-1.7B-Instruct report averages of about 41 and 57 on their own cards; those are reported numbers rather than same-box runs, and both models combine far longer pretraining (4T and 11T tokens) with preference tuning on top of SFT. This model is SFT plus one DPO pass on a 45B-token base, and its score is in the range of the smallest published instruct models. Part of the GSM8K gain is formatting: the DPO model learned UltraFeedback's habit of boxing its final answer, which the exact-match scorer extracts more reliably. GSM8K at 15% is where a small model with no dedicated math post-training lands.

For the base model's own scorecard (ARC-Easy, HellaSwag, PIQA, LAMBADA against compute-matched references, where it leads Pythia-1B at 15% of the tokens), see the Chochmah-1B card. Instruction tuning does not add knowledge, so those numbers are the ceiling this model operates under.

How to read GSM8K at this size. The sample below with the two trains reaches the correct answer through incorrect arithmetic. Exact-match scoring counts the number, not the path, so some fraction of any small model's GSM8K score is that.

Sample Outputs

Verbatim, unedited, from the first prompts run on each model (temperature 0.6, top-p 0.9, repetition penalty 1.1). The SFT-only outputs are shown first, since they are the baseline, with this model's output alongside wherever the DPO pass changed it in an instructive way. Shown together on purpose: six prompts, two models one pass apart, and between them an accurate picture of what a 1B instruct model trained on 45B tokens is.

User: What did the Supreme Court hold in Marbury v. Madison, and why does it matter?

In Marbury v. Madison, the Supreme Court held that a federal court has the authority to issue a writ of mandamus to compel a state official to perform an act that is specifically required by the Constitution. The case involved William Marbury, who had been appointed as a justice of the peace in Washington Territory, but he was denied his commission by the territorial governor. Marbury then filed a petition for a writ of mandamus in the Supreme Court, arguing that the governor had no authority to revoke his commission.

The Supreme Court ruled in favor of Marbury, stating that the Constitution gives the federal judiciary the power to issue writs of mandamus to compel state officials to perform their duties. This decision marked a significant shift in the role of federal courts in interpreting the Constitution, as it established that the Supreme Court has the authority to review state actions and compel them to comply with federal laws and constitutional provisions.

The importance of Marbury v. Madison lies in its recognition of the Supreme Court's authority to check the power of state governments and ensure that they comply with the Constitution. It also set a precedent for future cases involving federal courts issuing writs of mandamus to state officials, and has had far-reaching implications for the functioning of the federal government.

Three well-organized paragraphs that answer both halves of the question, and every substantive claim is wrong. Marbury's commission was for the District of Columbia, not a territory; it was withheld by Secretary of State Madison, not a governor; the Court ruled against the remedy Marbury sought, holding that the statute that would have let it issue the writ was unconstitutional, which is where judicial review comes from; and the case concerns federal officers, not state ones. The base model inverted holdings in opinion prose; the instruct model inverts them in an assistant's voice. The knowledge is the same.

This model's answer to the same prompt opens with a correct Bluebook citation, 5 U.S. (1 Cranch) 137 (1803), the first time any version of this model has cited the case properly, and then makes John Marshall the plaintiff and William Marbury the Chief Justice. Preference tuning changed the polish, not the facts.

User: In two sentences, what does a writ of certiorari do? (greedy)

A writ of certiorari is a court order that allows a higher court to review the record of a lower court's decision, typically to correct errors or clarify unclear points of law. It enables the higher court to examine the original proceedings and make an independent judgment about the case, rather than relying on the lower court's findings.

Correct, and the requested length. Definitions, which appear thousands of times in the pretraining corpus, are the kind of legal question this model answers reliably; holdings of specific cases are not.

User: Summarize Nietzsche's critique of morality in three sentences.

Nietzsche critiques morality by arguing that traditional moral values and principles are based on the idea of "the eternal recurrence" – a concept where every action is a repetition of the same basic actions. He argues that this leads to a lack of progress and a lack of creativity in our moral development, as we are constantly repeating the same patterns and behaviors. This results in a lack of freedom and autonomy for individuals, and ultimately leads to the death of the human spirit.

Exactly three sentences, as asked. The argument is invented: "eternal recurrence" is a real Nietzschean term attached to a critique he never made (the genealogy of morals, slave morality and ressentiment are absent).

User: A train leaves Bismarck at 9:00 am going 60 mph. A second train leaves at 10:30 am on the same track going 80 mph. At what time does the second train catch the first? Show your work.

To solve this problem, we need to find the relative speed between the two trains. The first train is traveling at 60 mph and the second train is traveling at 80 mph. The relative speed is the difference between their speeds, which is 80 - 60 = 20 mph.

Now, let's consider the time it takes for the second train to catch up to the first train. Since the first train leaves at 9:00 am and catches up at 10:30 am, we need to find the time it takes for the second train to cover the same distance as the first train.

The distance covered by the first train in 1 hour at 60 mph is 60 * 1 = 60 miles. The distance covered by the second train in 1 hour at 80 mph is 80 * 1 = 80 miles.

Since the first train catches up to the second train in 1 hour, we can calculate the time it takes for the second train to catch up by dividing the distance covered by the first train by the relative speed.

Time = Distance / Relative Speed Time = 60 / 20 Time = 3 hours

Therefore, the second train catches the first train at 3:00 pm.

3:00 pm is correct. The work is not: the head start is 1.5 hours (90 miles), not 1 hour, which gives 4.5 hours from 10:30; the model computes 3 hours and then turns "3 hours" into "3:00 pm." The relative-speed setup was right, the arithmetic wrong, the answer right by coincidence.

User: Write a Python function fibonacci(n) that returns the nth Fibonacci number, with a base case and a docstring. Then show the first ten values.

python

def fibonacci(n):    """Return the nth Fibonacci number."""    if n <= 0:        return 0    elif n == 1:        return 1    else:        return fibonacci(n-1) + fibonacci(n-2) # Test the functionprint(fibonacci(10))  # Output: 55

This function uses recursion to calculate the nth Fibonacci number. However, it has a major drawback: it does not handle negative numbers or non-integer inputs. To fix this, you can add input validation:

python

def fibonacci(n):    """Return the nth Fibonacci number."""    if not isinstance(n, int):        raise TypeError("Input must be an integer.")    if n < 0:        raise ValueError("Input must be a non-negative integer.")    if n <= 1:        return n    else:        return fibonacci(n-1) + fibonacci(n-2) # Test the functionprint(fibonacci(10))  # Output: 55

Both versions are correct, with base cases and the docstring. The base model, given the same function signature, produced a dedented return with no base case; this is the clearest skill gain from SFT. It did not show the first ten values, which is the instruction-following gap rather than a code gap, and the "major drawback" it names is not the real one (exponential time).

User: List three differences between a trial court and an appellate court. Use exactly three bullet points and no more than 15 words each.

Trial court: - Presents the facts and law to a jury in a trial setting.

Appellate court: - Reviews the trial court's decision for errors of law.

Trial court: - Has the power to set aside a verdict.

Appellate court: - Has the power to reverse or modify a lower court's decision.

SFT-only: content fine, format failed, four numbered items instead of three bullets. This model gives exactly three items, each 35–40 words against the 15-word cap:

Trial courts handle initial court cases, focusing on evidence and arguments presented in a trial setting, while appellate courts review trial court decisions for legal errors, primarily focusing on whether the trial court correctly applied the law.

Trial courts have jurisdiction over specific cases, whereas appellate courts have broader jurisdiction, covering all cases that have been appealed, including both trial and non-trial cases.

Trial courts are more hands-on, with the judge's role often being to interpret evidence and make rulings, whereas appellate courts focus on whether the trial court's decisions were correct based on the law.

One IFEval failure traded for another. This is the length bias that DPO on UltraFeedback is known to introduce (its raters preferred longer answers), and it is the main thing the next revision should target.

User: Rewrite this sentence to be more formal: 'the judge tossed the case cuz the cops didn't have a warrant.'

The judge decided to dismiss the case because the police lacked a warrant for the arrest.

Good, with one small embellishment: "for the arrest" is not in the source. This model rewrites the sentence equally well and then appends a paragraph explaining each word choice that nobody asked for; the same length bias again.

Known Issues and Limitations

The knowledge is the base model's. Supervised fine-tuning teaches the format of an answer, not new facts, and the base saw 45B tokens (about 38 per parameter). This model will give organized, confident, wrong accounts of specific cases, texts, and events, as the Marbury sample shows. It must not be used as a source of legal, medical, financial, or factual answers, and nothing it says about a case should be assumed to be in that case. The intended legal use is retrieval-grounded: give it the opinion, and have a separate checker verify every citation.
Real citations, wrong propositions. The base model's signature failure survives tuning. Treat any citation it produces as a lookup key, never as support.
Instruction following is partial. Item counts improved with DPO; length limits got worse, because UltraFeedback's preferences favor longer answers and the model learned that too. Expect explanations and self-commentary you did not ask for, and over-length answers to length-capped prompts. See the IFEval numbers above for the measured rate.
Math reasoning is unreliable even when the final number is right; show-your-work answers should be checked line by line.
No safety tuning. SmolTalk includes ordinary assistant behavior but no dedicated refusal or safety training. The model will attempt most requests. It has no content filtering of its own.
English only, 4,096-token context, as inherited from the base.
Repetition and drift on long generations, as with all small models; the saved sampling defaults (temperature 0.6, top-p 0.9, repetition penalty 1.1) are tuned for 100–300 token answers.

Technical Specifications

Table with columns: Specification, Value
Specification	Value
Model Family	TevunahAi Chochmah
Variant	1B-Instruct-DPO (v1.0)
Base	Chochmah-1B-Instruct (SFT), from Chochmah-1B `ckpt_42914`
Total Parameters	1,182,900,224
Architecture Class	`Qwen3ForCausalLM` (transformers >= 4.51)
Layers / Hidden / Intermediate	24 / 2,048 / 5,632
Attention	GQA, 16 query heads, 4 KV heads, head dim 128, QK-norm

Roadmap

A length-normalized preference pass (SimPO or length-desensitized DPO) to remove the verbosity bias this pass introduced, published as a revision here if it improves the length-constrained IFEval categories
TevunahAi Ultra-Hybrid GPTQ variant of this model
Legal grounding — a retrieval-backed configuration over CourtListener with citation verification, which is where this model's case-law pretraining pays off

Changelog

v1.0 (October 16, 2026) — Initial release: one DPO pass on UltraFeedback-binarized on top of the Chochmah-1B-Instruct SFT weights. Against the SFT-only model: IFEval prompt-level strict 22.7 → 24.4, instruction-level strict 33.2 → 34.8, GSM8K strict 13.0 → 14.7; all six measured scores improved. Answers are longer on average, and length-capped prompts are now the most common instruction-following failure.

License

Weights: Apache-2.0. Fine-tuning data: SmolTalk, Apache-2.0; UltraFeedback-binarized, MIT. The base model's pretraining data licenses are listed on the Chochmah-1B card. Users are responsible for their own compliance with the licenses of the underlying datasets.

Citation

bibtex

@misc{chochmah1b_instruct_dpo_2026,  title  = {Chochmah-1B-Instruct-DPO: preference tuning a from-scratch instruct model on one GPU},  author = {TevunahAi},  year   = {2026},  url    = {https://huggingface.co/TevunahAi/Chochmah-1B-Instruct-DPO},  note   = {Full-parameter SFT of Chochmah-1B on SmolTalk and DPO on UltraFeedback; 1.18B parameters, ChatML}}

Acknowledgments

Hugging Face / SmolLM team for SmolTalk and the SmolLM2 recipe this fine-tune follows, for TRL, and (with OpenBMB) for UltraFeedback and its binarized release
Qwen team for the Qwen3 block design and its transformers implementation
Tri Dao and contributors for FlashAttention, which made padding-free packing on one GPU practical
EleutherAI for lm-evaluation-harness

Pretrained and tuned by TevunahAi AI Model Quantization and On-Premises AI Infrastructure From-scratch models, hybrid-architecture quantization (Mamba, MoE, SSM), and enterprise hardware bring-up

https://huggingface.co/TevunahAi

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

TevunahAi

Model Tree

Base

TevunahAi/Chochmah-1B-Instruct

Fine-tuned

this model

Input Modalities

Text

Output Modalities

Text

Supported Functionality

Dedicated EndpointsContainer

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

Model Details

Table with columns: Property, Value
Property	Value
Developer	TevunahAi
Base Model	TevunahAi/Chochmah-1B-Instruct (SFT), itself from TevunahAi/Chochmah-1B (1,182.9M params, 45B tokens, from scratch)
Model Type	Decoder-only transformer, Qwen3-architecture block, SFT + DPO
Post-Training	Full-parameter SFT on SmolTalk (2 epochs, 1.77B tokens), then full-parameter DPO on UltraFeedback-binarized (1 epoch, 52.5k pairs)
Chat Format	ChatML (`<
Context Length	4,096 tokens
Tokenizer	SmolLM2 (49,152 BPE)
Release Precision	bf16 safetensors (2.4 GB)
Training Hardware	1x NVIDIA RTX 5000 Ada 32 GB
Training Time	84.8 h SFT (October 11–15, 2026) + 4.1 h DPO (October 15–16, 2026)
Stage	SFT + DPO
Loads With	`AutoModelForCausalLM`, transformers >= 4.51, no custom code
License	Apache-2.0

Chat Format

text

<|im_start|>userWhat does a writ of certiorari do?<|im_end|><|im_start|>assistantA writ of certiorari is ...<|im_end|>

Usage

python

from transformers import AutoTokenizer, AutoModelForCausalLMimport torch model_id = "TevunahAi/Chochmah-1B-Instruct-DPO"tok = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16).cuda() messages = [{"role": "user", "content": "In two sentences, what does a writ of certiorari do?"}]inputs = tok.apply_chat_template(messages, add_generation_prompt=True, tokenize=True,                                 return_dict=True, return_tensors="pt").to("cuda")out = model.generate(**inputs, max_new_tokens=300)   # sampling defaults from generation_config.jsonprint(tok.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))

vLLM

bash

vllm serve TevunahAi/Chochmah-1B-Instruct-DPO --max-model-len 4096 --dtype bfloat16

vllm serve TevunahAi/Chochmah-1B-Instruct-DPO --max-model-len 4096 --dtype bfloat16

The chat template ships with the tokenizer, so the OpenAI-compatible /v1/chat/completions endpoint works as-is.

Training

Stage 1 — SFT (inherited from Chochmah-1B-Instruct)

This stage is the Chochmah-1B-Instruct model, unchanged; it is documented here in full so this card stands on its own.

Data

A held-out set of 2,000 conversations from SmolTalk's test split was scored every 350 steps.

Recipe

Table with columns: Field, Value
Field	Value
Method	Full-parameter SFT (no LoRA, no frozen layers)
Loss	Cross-entropy on assistant tokens only (user and system turns masked)
Sequence Handling	Best-fit packing into 4,096-token sequences with padding-free flash-attention 2, so packed conversations never attend to each other
Optimizer	AdamW (fused), betas (0.9, 0.999), eps 1e-8, no weight decay
Learning Rate	5e-5 peak, 250-step linear warmup, cosine decay to 0
Gradient Clipping	1.0
Global Batch	262,144 tokens (4 packed sequences x 16 accumulation steps x 4,096)

The optimizer keeps fp32 master weights on purpose: at a 5e-5 learning rate the per-step updates are below bf16's resolution, and pure-bf16 fine-tuning at this scale silently drops most of them.

Loss

Held-out loss on the 2,000-conversation SmolTalk test subset, every 350 steps:

Table with columns: Step, 350, 1,050, 1,750, 2,450, 3,150, 3,500, 4,200, 4,900, 5,600, 6,300, 6,650
Step	350	1,050	1,750	2,450	3,150	3,500	4,200	4,900	5,600	6,300	6,650
Eval loss	0.942	0.864	0.830	0.808	0.792	0.789	0.781

Stage 2 — DPO

Data

Recipe

Table with columns: Field, Value
Field	Value
Method	Full-parameter DPO, sigmoid loss, beta 0.1
Policy / Reference	Both initialized from the SFT weights; policy in fp32 master weights with bf16 autocast, reference frozen in bf16
Sequence Limits	Prompt up to 1,024 tokens, prompt + completion up to 2,048
Optimizer	AdamW (fused), no weight decay
Learning Rate	1e-6 peak, 50-step warmup, cosine decay to 0
Gradient Clipping	1.0
Batch	32 pairs per optimizer step (2 x 16 accumulation)

Preference accuracy

Held-out accuracy (how often the policy's implicit reward prefers the chosen response), every 200 steps:

Table with columns: Step, 200, 400, 600, 800, 1,000, 1,200, 1,400, 1,600
Step	200	400	600	800	1,000	1,200	1,400	1,600
Accuracy	0.685	0.701	0.705	0.711	0.711	0.707	0.710	0.710

It climbs for the first 800 steps and holds at 0.71, which is the ceiling a 1B model reaches on this dataset. Training loss ended at 0.56 from 0.69.

Evaluation

lm-evaluation-harness, chat template applied, few-shot examples as multi-turn, bf16, greedy decoding.

Table with columns: Benchmark, Setting, Chochmah-1B-Instruct-DPO, Chochmah-1B-Instruct (SFT only)
Benchmark	Setting	Chochmah-1B-Instruct-DPO	Chochmah-1B-Instruct (SFT only)
IFEval, prompt-level strict	0-shot	24.4 ± 1.9	22.7 ± 1.8
IFEval, instruction-level strict	0-shot	34.8	33.2
IFEval, prompt-level loose	0-shot	27.5 ± 1.9	24.8 ± 1.9
IFEval, instruction-level loose	0-shot

Sample Outputs

User: What did the Supreme Court hold in Marbury v. Madison, and why does it matter?

In Marbury v. Madison, the Supreme Court held that a federal court has the authority to issue a writ of mandamus to compel a state official to perform an act that is specifically required by the Constitution. The case involved William Marbury, who had been appointed as a justice of the peace in Washington Territory, but he was denied his commission by the territorial governor. Marbury then filed a petition for a writ of mandamus in the Supreme Court, arguing that the governor had no authority to revoke his commission.

The Supreme Court ruled in favor of Marbury, stating that the Constitution gives the federal judiciary the power to issue writs of mandamus to compel state officials to perform their duties. This decision marked a significant shift in the role of federal courts in interpreting the Constitution, as it established that the Supreme Court has the authority to review state actions and compel them to comply with federal laws and constitutional provisions.

The importance of Marbury v. Madison lies in its recognition of the Supreme Court's authority to check the power of state governments and ensure that they comply with the Constitution. It also set a precedent for future cases involving federal courts issuing writs of mandamus to state officials, and has had far-reaching implications for the functioning of the federal government.

User: In two sentences, what does a writ of certiorari do? (greedy)

A writ of certiorari is a court order that allows a higher court to review the record of a lower court's decision, typically to correct errors or clarify unclear points of law. It enables the higher court to examine the original proceedings and make an independent judgment about the case, rather than relying on the lower court's findings.

User: Summarize Nietzsche's critique of morality in three sentences.

Nietzsche critiques morality by arguing that traditional moral values and principles are based on the idea of "the eternal recurrence" – a concept where every action is a repetition of the same basic actions. He argues that this leads to a lack of progress and a lack of creativity in our moral development, as we are constantly repeating the same patterns and behaviors. This results in a lack of freedom and autonomy for individuals, and ultimately leads to the death of the human spirit.

User: A train leaves Bismarck at 9:00 am going 60 mph. A second train leaves at 10:30 am on the same track going 80 mph. At what time does the second train catch the first? Show your work.

To solve this problem, we need to find the relative speed between the two trains. The first train is traveling at 60 mph and the second train is traveling at 80 mph. The relative speed is the difference between their speeds, which is 80 - 60 = 20 mph.

Now, let's consider the time it takes for the second train to catch up to the first train. Since the first train leaves at 9:00 am and catches up at 10:30 am, we need to find the time it takes for the second train to cover the same distance as the first train.

The distance covered by the first train in 1 hour at 60 mph is 60 * 1 = 60 miles. The distance covered by the second train in 1 hour at 80 mph is 80 * 1 = 80 miles.

Since the first train catches up to the second train in 1 hour, we can calculate the time it takes for the second train to catch up by dividing the distance covered by the first train by the relative speed.

Time = Distance / Relative Speed Time = 60 / 20 Time = 3 hours

Therefore, the second train catches the first train at 3:00 pm.

User: Write a Python function fibonacci(n) that returns the nth Fibonacci number, with a base case and a docstring. Then show the first ten values.

python

def fibonacci(n):    """Return the nth Fibonacci number."""    if n <= 0:        return 0    elif n == 1:        return 1    else:        return fibonacci(n-1) + fibonacci(n-2) # Test the functionprint(fibonacci(10))  # Output: 55

python

def fibonacci(n):    """Return the nth Fibonacci number."""    if not isinstance(n, int):        raise TypeError("Input must be an integer.")    if n < 0:        raise ValueError("Input must be a non-negative integer.")    if n <= 1:        return n    else:        return fibonacci(n-1) + fibonacci(n-2) # Test the functionprint(fibonacci(10))  # Output: 55

User: List three differences between a trial court and an appellate court. Use exactly three bullet points and no more than 15 words each.

Trial court: - Presents the facts and law to a jury in a trial setting.

Appellate court: - Reviews the trial court's decision for errors of law.

Trial court: - Has the power to set aside a verdict.

Appellate court: - Has the power to reverse or modify a lower court's decision.

SFT-only: content fine, format failed, four numbered items instead of three bullets. This model gives exactly three items, each 35–40 words against the 15-word cap:

Trial courts handle initial court cases, focusing on evidence and arguments presented in a trial setting, while appellate courts review trial court decisions for legal errors, primarily focusing on whether the trial court correctly applied the law.

Trial courts have jurisdiction over specific cases, whereas appellate courts have broader jurisdiction, covering all cases that have been appealed, including both trial and non-trial cases.

Trial courts are more hands-on, with the judge's role often being to interpret evidence and make rulings, whereas appellate courts focus on whether the trial court's decisions were correct based on the law.

User: Rewrite this sentence to be more formal: 'the judge tossed the case cuz the cops didn't have a warrant.'

The judge decided to dismiss the case because the police lacked a warrant for the arrest.

Known Issues and Limitations

The knowledge is the base model's. Supervised fine-tuning teaches the format of an answer, not new facts, and the base saw 45B tokens (about 38 per parameter). This model will give organized, confident, wrong accounts of specific cases, texts, and events, as the Marbury sample shows. It must not be used as a source of legal, medical, financial, or factual answers, and nothing it says about a case should be assumed to be in that case. The intended legal use is retrieval-grounded: give it the opinion, and have a separate checker verify every citation.
Real citations, wrong propositions. The base model's signature failure survives tuning. Treat any citation it produces as a lookup key, never as support.
Instruction following is partial. Item counts improved with DPO; length limits got worse, because UltraFeedback's preferences favor longer answers and the model learned that too. Expect explanations and self-commentary you did not ask for, and over-length answers to length-capped prompts. See the IFEval numbers above for the measured rate.
Math reasoning is unreliable even when the final number is right; show-your-work answers should be checked line by line.
No safety tuning. SmolTalk includes ordinary assistant behavior but no dedicated refusal or safety training. The model will attempt most requests. It has no content filtering of its own.
English only, 4,096-token context, as inherited from the base.
Repetition and drift on long generations, as with all small models; the saved sampling defaults (temperature 0.6, top-p 0.9, repetition penalty 1.1) are tuned for 100–300 token answers.

Technical Specifications

Table with columns: Specification, Value
Specification	Value
Model Family	TevunahAi Chochmah
Variant	1B-Instruct-DPO (v1.0)
Base	Chochmah-1B-Instruct (SFT), from Chochmah-1B `ckpt_42914`
Total Parameters	1,182,900,224
Architecture Class	`Qwen3ForCausalLM` (transformers >= 4.51)
Layers / Hidden / Intermediate	24 / 2,048 / 5,632
Attention	GQA, 16 query heads, 4 KV heads, head dim 128, QK-norm

Roadmap

A length-normalized preference pass (SimPO or length-desensitized DPO) to remove the verbosity bias this pass introduced, published as a revision here if it improves the length-constrained IFEval categories
TevunahAi Ultra-Hybrid GPTQ variant of this model
Legal grounding — a retrieval-backed configuration over CourtListener with citation verification, which is where this model's case-law pretraining pays off

Changelog

License

Citation

bibtex

@misc{chochmah1b_instruct_dpo_2026,  title  = {Chochmah-1B-Instruct-DPO: preference tuning a from-scratch instruct model on one GPU},  author = {TevunahAi},  year   = {2026},  url    = {https://huggingface.co/TevunahAi/Chochmah-1B-Instruct-DPO},  note   = {Full-parameter SFT of Chochmah-1B on SmolTalk and DPO on UltraFeedback; 1.18B parameters, ChatML}}

@misc{chochmah1b_instruct_dpo_2026,  title  = {Chochmah-1B-Instruct-DPO: preference tuning a from-scratch instruct model on one GPU},  author = {TevunahAi},  year   = {2026},  url    = {https://huggingface.co/TevunahAi/Chochmah-1B-Instruct-DPO},  note   = {Full-parameter SFT of Chochmah-1B on SmolTalk and DPO on UltraFeedback; 1.18B parameters, ChatML}}

Acknowledgments

Hugging Face / SmolLM team for SmolTalk and the SmolLM2 recipe this fine-tune follows, for TRL, and (with OpenBMB) for UltraFeedback and its binarized release
Qwen team for the Qwen3 block design and its transformers implementation
Tri Dao and contributors for FlashAttention, which made padding-free packing on one GPU practical
EleutherAI for lm-evaluation-harness

Pretrained and tuned by TevunahAi AI Model Quantization and On-Premises AI Infrastructure From-scratch models, hybrid-architecture quantization (Mamba, MoE, SSM), and enterprise hardware bring-up

https://huggingface.co/TevunahAi