GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
yunusemrejr
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
yunusemrejr
Model Tree
Qwen/Qwen3-0.6B
Input Modalities
Output Modalities
Supported Functionality
Baud is a full fine-tune of every Qwen3-0.6B weight, plus three additions that live in the same GGUF file:
| Part | What it is |
|---|---|
| 2 structure blocks | Two transformer blocks added on top of the 28 (blocks 29 and 30), initialised as copies of block 28 with o_proj and down_proj set to zero, so they start as the identity and learn only what the base lacked. With 30 layers the model is still a standard Qwen3ForCausalLM. |
| 8 control tokens | <|baud_prose|>, <|baud_table|>, <|baud_steps|>, <|baud_list|>, <|baud_frame|>, <|baud_code|>, <|baud_scope|>, <|baud_lang|> in unused rows of the tied 151,936-row embedding table. Every answer starts with one. |
| Decision head | llama.cpp decision type openjev (qwen3.decision.*, tokenizer.chat_template.systemone): the model answers a typed question by the probabilities of the option letters. Temperatures were fitted on held-out decisions. |
| Shape grammars | baud.grammar.{table,frame,steps,list,code}: GBNF grammars stored in the GGUF metadata, enforced by llama.cpp while decoding. |
For every message the app plans (four typed questions in one pass: protocol, answer shape, in scope, English),
routes (not English → <|baud_lang|>; in scope → the planned shape's control token and grammar; otherwise the model
chooses), then writes greedily inside the grammar, so a table always has one column count and a |---| separator,
steps are numbered in order and a frame table always reads | Field | Size | Value | Meaning |. In the app the model
writes with the matching notes in its system turn, after the line Reference notes (answer only from these):, and the
app then checks each sentence, list item and table row against those notes: numbers, identifiers and their pairing
by words, then an entailment model. Frames, check bytes (CRC-16, LRC, MBAP length) and MQTT topic matches are computed
by the app, not written by the model: arithmetic a 0.6B model should not be trusted with. The shipped baud.grammar.steps
forces exactly 8 steps; the app makes steps 3 to 8 optional when it loads the model, and so should other runtimes.
| Path | What it is |
|---|---|
gguf/ | Q4_K_M, importance-matrix quantized, split into 5 parts below 100 MB (395 MiB in total) with SHA256SUMS. The first part holds all metadata (decision head, grammars, templates). |
model.safetensors, config.json, tokenizer files | The fine-tuned model in bf16 for transformers (30 layers, control tokens in the tokenizer, Baud's chat template). |
eval/ | The model card JSON, the evaluation report, decision-head accuracy and the llama-server check. |
sh
llama-server -m gguf/baud-0.6b-q4_k_m-00001-of-00005.gguf -c 2048Plan (decision head, llama.cpp b11361 or newer):
sh
curl localhost:8080/v1/systemone -d '{"state": "Decode this Modbus RTU frame: 11 03 00 6B 00 03 76 87", "questions": {"shape": {"type": "choice", "instructions": "Which answer shape fits the message best?", "criteria": {"prose": null, "table": null, "steps": null, "list": null, "frame": null, "code": null}}}}'Write: render the chat template (tokenizer.chat_template, ChatML with Baud's system line), end the prompt with the
control token of the planned shape and pass that shape's grammar from the metadata:
python
from gguf import GGUFReaderf = GGUFReader("gguf/baud-0.6b-q4_k_m-00001-of-00005.gguf").fields["baud.grammar.frame"]grammar = bytes(f.parts[f.data[0]]).decode()sh
curl localhost:8080/completion -d '{"prompt": "<|im_start|>system\nYou are Baud, a specialist in industrial communication protocols: Modbus, OPC UA, OPC DA and MQTT. You answer in English only.<|im_end|>\n<|im_start|>user\nDecode this Modbus RTU frame: 11 03 00 6B 00 03 76 87<|im_end|>\n<|im_start|>assistant\n<|baud_frame|>", "grammar": "<baud.grammar.frame>", "temperature": 0, "top_k": 40, "repeat_penalty": 1.05, "n_predict": 700, "stop": ["<|im_end|>"]}'Greedy decoding (temperature 0, top_k 40, repeat penalty 1.05) is what Baud was evaluated with. llama-server lists a
decision-type model with the output modality decisions; text completion still works.
python
from transformers import AutoModelForCausalLM, AutoTokenizertok = AutoTokenizer.from_pretrained("yunusemrejr/Baud-0.6B")model = AutoModelForCausalLM.from_pretrained("yunusemrejr/Baud-0.6B", dtype="bfloat16")prompt = tok.apply_chat_template([{"role": "user", "content": "Compare MQTT QoS 0, 1 and 2"}], tokenize=False, add_generation_prompt=True)ids = tok(prompt + "<|baud_table|>", return_tensors="pt").input_idsout = model.generate(ids, max_new_tokens=700, do_sample=False, repetition_penalty=1.05)print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))Without a grammar the structure is learned, not enforced; llama.cpp with the stored grammars enforces it.
Every stage ran on Google Colab (RTX PRO 6000).
| Stage | What happened |
|---|---|
| Documents | 2,195 passages from 34 documents: the MQTT 5.0, 3.1.1 and 3.1 standards, MQTT-SN 1.2, Sparkplug 3.0.0, the Modbus Application Protocol V1.1b3, Modbus over Serial Line V1.02, Modbus Messaging on TCP/IP V1.0b, Modbus/TCP Security, 24 parts of OPC UA 1.05 and Kutup's own OPC DA notes. |
| Teacher | Qwen3.8-27B-FP8 (Apache-2.0, vLLM) wrote 32,773 question–answer pairs from several angles, each in one answer shape; a second, strict pass checked each pair against its passage, and code checked the shape, the language and every number. 27,988 were kept. The same stage wrote exam items, trap questions with false premises, follow-ups, grounded cross-protocol comparisons, out-of-scope messages and translations into 21 languages (for the English-only gate). |
| Made by code | 2,297 examples with exact values: Modbus function and exception codes, addressing, RTU/TCP frames decoded and built, timings; MQTT packet types, fixed headers, Remaining Length, MQTT 5 reason codes and properties, topic matching, Sparkplug topics; OPC UA StatusCodes, attribute ids, enumerations and NodeIds from the OPC Foundation's UA-Nodeset (MIT); OPC DA quality bytes and error codes. |
| Training set | 37,746 chat examples and 79,887 decision examples, 3% held out, near-duplicates of exam questions removed. Structured answers were normalised to exactly what the grammars accept. |
| Training | Stage 1: only the new parts (300 steps). Stage 2: every weight, AdamW on fp32 weights with bf16 autocast, cosine schedule, weighted loss (answer 1, control token 3, decision letter 10). Held-out loss 0.679. |
| Export | GGUF, decision-head and Baud metadata, importance matrix on Baud's own text, quantization, a check through llama-server. |
Data rule. The specifications were read only by the teacher, which wrote original explanatory question–answer pairs. No specification text is used as training text, and none is published here.
Closed-book exam (713 questions written from the documents, kept out of training; graded by the teacher against
the reference answer and the source passage without knowing which model wrote the answer). The bf16 column is the
transformers model in this repo answering freely; the Q4_K_M column is the GGUF in gguf/ answering the way the
browser app does: the decision head plans, the routed control token and grammar shape the answer. With 713 questions,
differences under about 2 points are noise.
| Topic | Questions | Baud 0.6B (bf16) | Baud Q4_K_M (gguf/, the app's path in llama.cpp) | Qwen3-0.6B (untouched) | Qwen3.8-27B (teacher, closed book) |
|---|---|---|---|---|---|
| All | 713 | 40.9% | 37.3% | 10.4% | 38.1% |
| Modbus | 161 | 49.1% | 45.3% | 12.4% | 48.4% |
| OPC UA | 296 | 29.7% | 29.4% | 7.8% | 25.7% |
| OPC DA | 14 | 64.3% | 21.4% | 0.0% | 35.7% |
| MQTT | 242 | 47.9% | 42.6% | 12.8% | 46.7% |
Decision head (held-out messages):
| Question | Accuracy (bf16) | Accuracy (Q4_K_M, llama-server) |
|---|---|---|
| Protocol (6 options) | 99.3% | 99.0% |
| Answer shape (6 options) | 82.0% | 84.8% |
| In scope (yes/no) | 100.0% | 100.0% |
| English (yes/no) | 100.0% | 100.0% |
Grammar-constrained answers that fit their shape exactly (Q4_K_M, llama-server):
| Shape | Fits |
|---|---|
| code | 5/5 |
| frame | 12/12 |
| list | 56/56 |
| steps | 17/17 |
| table | 28/30 |
Held-out chats with the right control token, shape, language and scope: 82.1% of 1,124.
| Quantization | Size | Held-out perplexity |
|---|---|---|
| bf16 | 1,202 MiB | 4.6567 |
| Q8_0 | 641 MiB | 4.6420 |
| Q5_K_M | 444 MiB | 4.4355 |
Q4_K_M (this repo, gguf/) | 395 MiB | 4.8172 |
| IQ4_XS | 366 MiB | 4.6226 |
Apache-2.0, like the base model Qwen3-0.6B and the teacher Qwen3.8-27B. Fine-tuned by ASP Dijital for Kutup.
curl localhost:8080/v1/systemone -d '{"state": "Decode this Modbus RTU frame: 11 03 00 6B 00 03 76 87", "questions": {"shape": {"type": "choice", "instructions": "Which answer shape fits the message best?", "criteria": {"prose": null, "table": null, "steps": null, "list": null, "frame": null, "code": null}}}}'from gguf import GGUFReaderf = GGUFReader("gguf/baud-0.6b-q4_k_m-00001-of-00005.gguf").fields["baud.grammar.frame"]grammar = bytes(f.parts[f.data[0]]).decode()curl localhost:8080/completion -d '{"prompt": "<|im_start|>system\nYou are Baud, a specialist in industrial communication protocols: Modbus, OPC UA, OPC DA and MQTT. You answer in English only.<|im_end|>\n<|im_start|>user\nDecode this Modbus RTU frame: 11 03 00 6B 00 03 76 87<|im_end|>\n<|im_start|>assistant\n<|baud_frame|>", "grammar": "<baud.grammar.frame>", "temperature": 0, "top_k": 40, "repeat_penalty": 1.05, "n_predict": 700, "stop": ["<|im_end|>"]}'from transformers import AutoModelForCausalLM, AutoTokenizertok = AutoTokenizer.from_pretrained("yunusemrejr/Baud-0.6B")model = AutoModelForCausalLM.from_pretrained("yunusemrejr/Baud-0.6B", dtype="bfloat16")prompt = tok.apply_chat_template([{"role": "user", "content": "Compare MQTT QoS 0, 1 and 2"}], tokenize=False, add_generation_prompt=True)ids = tok(prompt + "<|baud_table|>", return_tensors="pt").input_idsout = model.generate(ids, max_new_tokens=700, do_sample=False, repetition_penalty=1.05)print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))