yunusemrejr

Flyingkiwi-EN-0.6B

Deploy Dedicated

README

License: apache-2.0

What changed from Qwen3-0.6B

Language-specialised vocabulary. Tokens that English text never uses were removed and the BPE merges rebuilt, closed over merge ancestors so any text made of kept tokens tokenises exactly as before. This shrinks the embedding and the per-token output layer; text outside the kept set falls back to shorter pieces.
Attention. Full attention in every layer, in training and in the browser. An earlier run that trained with a 1,024-token sliding window in three of every four layers had clearly higher loss on long conversations after the first 1,024 tokens (1.28 against 1.08 nats per token), so the window was dropped.
Concise answers. Before training, the teacher rewrote each of its own answers into the shortest complete answer (keeping code blocks, requested formats and honest statements of uncertainty; a rewrite that dropped code, added a disclaimer or grew longer kept the original). 22,067 answers were rewritten and the training answers kept 43% of their characters. Every extra sentence is another chance for a 0.6B model to state something false.
8-bit browser weights. No 4-bit format kept the quality. On 40 held-out conversations the answer loss per token was 1.327 nats in float and in 8 bits, 1.496 with plain 4-bit rounding (32 values per fp16 scale), 1.407 with GPTQ, and 1.365 with GPTQ plus the 89 most sensitive of 196 matrices (mostly value and key projections) kept in 8 bits, at 420 MB instead of 561 MB. The browser therefore runs 8-bit weights; their next-token distribution stays within a mean KL of 0.011 of this checkpoint.
Distillation for honesty and usefulness. Answers were written by Qwen3.5-35B-A3B (Apache-2.0) following Flyingkiwi's rules (be accurate, say when unsure, correct false premises, be concise, use Markdown and code blocks). Calibration examples come from what the base model itself could and could not answer on TriviaQA questions: known answers were kept, unknown ones taught an honest "I'm not sure".

Training

Base: Qwen/Qwen3-0.6B @ c1899de, one language per model, full-parameter fine-tuning in BF16 with FP32 masters.
Data: 15,834 training conversations (344 held out): public prompts from OASST2, Aya, GSM8K and TriviaQA plus teacher-written prompts (topics, coding, writing, math, honesty probes, invented-entity questions, identity, memory tests), answered by the teacher; 8% chained multi-topic conversations up to 4,096 tokens.
Schedule: 148 optimizer steps, peak learning rate 1.5e-05 with cosine decay, ~65k tokens per step, no quantisation-aware training; validation loss 2.165 → 1.311.
Compute: one NVIDIA RTX PRO 6000 Blackwell (96 GB) on Google Colab Pro+ for teacher generation and both trainings.

Evaluation

Measured on the project's frozen 100-case set for this language (held out from all training; 30 answerable, 20 unknowable, 10 false-premise, 10 ambiguity, 10 friendly, 20 multi-turn), with the browser's system prompt, greedy decoding and repetition penalty 1.1. Flyingkiwi was run as the shipped 8-bit file in the native build of the browser engine; the reference is unchanged Qwen3-0.6B (8-bit, mean KL 0.0014 from fp32). Responses were graded blind against reference answers by AI graders with spot checks; this is a small, assistant-graded test, not an independent human study. The released variant was chosen among several with this same test (listed in the technical report), so its score is slightly optimistic, and the reference's total moved by a few points between grading runs.

Table with columns: Category, Qwen3-0.6B, Flyingkiwi
Category	Qwen3-0.6B	Flyingkiwi
Answerable facts & reasoning	22/30 (73%)	20/30 (67%)
False-premise correction	7/10 (70%)	5/10 (50%)
Says when it cannot know	17/20 (85%)	18/20 (90%)
Handles ambiguity	0/10 (0%)	3/10 (30%)
Friendly conversation	6/10 (60%)	6/10 (60%)
Multi-turn context	11/20 (55%)	11/20 (55%)
All 100 cases	63/100 (63%)	63/100 (63%)
Wrong-language replies	0	0
Replies with fabricated facts/claims	6	6

TruthfulQA MC1 (817 questions, log-likelihood of each option as the assistant's reply in the same chat format): Qwen3-0.6B 25.8%, Flyingkiwi 25.1%. The earlier v3 release scored 21.3%; the concise retraining removed that loss (the difference to the reference is within the test's sampling error of about 1.5 points).

Engine fidelity: mean KL divergence between the browser file run by the engine (8-bit weights, activations and KV cache) and this checkpoint in Transformers, at the first reply token of 30 development prompts: 0.0114.

Use

python

from transformers import AutoModelForCausalLM, AutoTokenizer tok = AutoTokenizer.from_pretrained("yunusemrejr/Flyingkiwi-EN-0.6B")model = AutoModelForCausalLM.from_pretrained("yunusemrejr/Flyingkiwi-EN-0.6B", dtype="auto")messages = [{"role": "user", "content": "Why is the sky blue?"}]ids = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")out = model.generate(ids, max_new_tokens=300, do_sample=True, temperature=0.5, top_p=0.9, repetition_penalty=1.1)print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))

The model was trained with this system prompt most of the time (it also works without one):

text

You are Flyingkiwi, a small AI assistant made by Yunus Emre Vurgun. You run entirely on the user's device, so you cannot browse the internet, open links or files, or know today's date and recent events. Be friendly, clear and concise. Be truthful: if you are not sure, say so instead of guessing, and politely correct false assumptions. Use Markdown when it helps and put code in fenced code blocks. Reply in English unless the user asks for another language.

flyingkiwi-q8.fkm is the exact file the browser runs (engine source: engine/flyingkiwi.c in this repository's files; format described in engine/fkm.py). Recommended sampling: temperature 0.5, top-p 0.9, top-k 40, min-p 0.05, repetition penalty 1.1.

Limitations

Small model: factual recall is limited and it can be confidently wrong, especially on niche topics, recent events and precise numbers. It was trained to say when it is unsure, but that is not guaranteed.
No internet, tools or memory between conversations. It cannot know today's date.
English only; other languages are untested.
Code answers are useful for small tasks but should be reviewed and tested.

Licence and attribution

Apache-2.0, like the base model Qwen/Qwen3-0.6B (© Alibaba Cloud / Qwen team). Teacher outputs come from Qwen/Qwen3.5-35B-A3B (Apache-2.0). Prompt sources: OpenAssistant OASST2 (Apache-2.0), CohereLabs Aya dataset (Apache-2.0), GSM8K (MIT) and TriviaQA (Apache-2.0) questions; FineWeb-Edu, FineWeb-2 and github-code-clean (permissive/ODC-BY) were used only to count token frequencies for the vocabulary.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

yunusemrejr

Model Tree

Base

Qwen/Qwen3-0.6B

Fine-tuned

this model

Input Modalities

Text

Output Modalities