BlazingCustoms

acrostic-4b-v4a

README

License: apache-2.0

⚠️ Read this before you use the weights

The safety property of this model is not in the weights. It is in the wrapper.

The thing that makes this usable in a product is verify-and-resample: every generation is checked by a deterministic program oracle, and rejected and redrawn if the first letters do not actually spell the target. That check is not part of what you download here.

Without it you get silent, invisible failures. The failure mode is not a crash, a refusal, or obvious garbage — it is a poem that reads perfectly and spells the word wrong by one letter:

markdown
CONCERNS  ->  CONCERSN     (transposed)
DEXTROUS  ->  DECTROUS     (X substituted)
COXCOMB   ->  COWCOMB      (X substituted)
CLUCKS    ->  CLUCJS       (K substituted)

You would not notice while reading it. If the point of your application is that the hidden word is correct, an unverified generation is worthless — and worthless in a way that looks like success.

The oracle is ~30 lines of pure string operations with no dependencies and is reproduced in full below. Use it.

Accuracy

Measured on a 380-item held-out split whose target words and topics are disjoint from training at every length, graded by the oracle below. Lengths 3–9 were trained; length 10 was held out entirely and never trained, as a true out-of-distribution structure test.

All figures are content-level: the <n>. <L> | label is stripped first and the check is applied to the poem text itself, which is the stricter of the two possible readings.

Per-length curve

Exact counts, not just percentages:

Table with columns: target length, 3, 4, 5, 6, 7, 8, 9, 10 (OOD)
target length	3	4	5	6	7	8	9	10 (OOD)
temp 0	40/40	40/40	40/40	39/40	39/40	34/40	36/40	98/100
	100%	100%

Table with columns: tier, temp 0, temp 0.8
tier	temp 0	temp 0.8
in-distribution, lengths 3–9	268/280 = 95.7% [92.7, 97.5]	264/280 = 94.3% [90.9, 96.5]
out-of-distribution, length 10	98/100 = 98.0% [93.0, 99.4]	96/100 = 96.0% [90.2, 98.4]

(95% Wilson intervals.)

The honest capability statement, in one paragraph: this model works at target lengths 3–10. Over that whole range it is 366/380 = 96.3% [93.9, 97.8] at temp 0. Its worst cell is length 8 at 34/40 = 85.0% — quote that as the realistic worst case. Roughly one generation in seven at length 8 spells the word wrong, and you will not see it happen unless you check, which is why the shipped experience depends on the verify-and-resample wrapper described at the top of this card rather than on the raw model. There is no length cap below 10; the 3B predecessor did need a hard 6-letter cap — see The wall that wasn't fundamental.

Degradation is NOT monotonic — longer is not simply worse. Length 8 (34/40 = 85.0%) is the weakest cell, below length 9 (36/40 = 90.0%) and far below length 10 (98/100 = 98.0%). We do not have a mechanism for this and are not going to invent one. Length 10 is a separate 100-item word bank, so some of the gap may be word composition rather than length. Do not reason about this model as if accuracy fell off smoothly with length; it does not.

This is not a sampling artifact: it reproduces on an entirely different inference stack. Under onnxruntime (q4f16, pure greedy) the same shape appears — len 7 97.5%, len 8 92.5%, len 9 95.0%, len 10 98.0% — length 8 again the weakest of the three and length 10 again the strongest.

The base model is not zero — and that matters

Table with columns: temp 0, temp 0.8
	temp 0	temp 0.8
Qwen3-4B-Instruct-2507, untuned, same prompt	ID 231/280 = 82.5%, OOD 52/100 = 52.0%	ID 82.5%, OOD 47.0%
acrostic-4b-v4a	ID 268/280 = 95.7%, OOD 98/100 = 98.0%	ID 94.3%, OOD 96.0%

The untuned Qwen3-4B can already do a lot of this task. This is a genuine change from the previous generation: the same recipe on Qwen2.5-3B-Instruct had a base that scored 0/380 — literally zero — so that model's capability was entirely LoRA-conferred. That claim is not true here and we are not making it.

What the LoRA does buy, measured pairwise on the identical 380 items:

89 items the tune gets right that the base gets wrong, against 6 the other way (exact McNemar p = 4.7 × 10⁻²⁰ at temp 0; 86 vs 4, p = 4.3 × 10⁻²¹ at temp 0.8).
The largest gain is out of distribution: 52.0% → 98.0% at length 10.
In distribution: 82.5% → 95.7%.

So the tune is worth having — it roughly quarters the in-distribution error rate and effectively removes the OOD failure — but the headline "a small model that can do something the base cannot" is a description of the 3B ancestor, not of this model.

Failure mode

At temp 0 there are 14 failures across all 380 items, and they are word-locked: only 7 distinct words fail, each failing identically on both of its topics. Greedy failure is deterministic per word, not random — which is exactly why the wrapper retries at temperature 0.8 rather than at 0.

Taxonomy of those 14: 4 substitutions, 4 transpositions, 6 drop/double. The substitutions cluster on rare initial letters (X, K) — DEXTROUS→DECTROUS, COXCOMB→COWCOMB, CLUCKS→CLUCJS. That reads as a lexical problem (it is genuinely hard to open a poem line with X) rather than the positional counting defect that dominated the 3B.

Quantized and converted builds

Every number below is the oracle re-run on the converted weights, full 380-item set, temp 0 — not inherited from the f16 model.

GGUF (llama.cpp), for local/server use:

Table with columns: build, bytes, GB, ID 3–9, OOD 10, vs f16 (paired exact McNemar)
build	bytes	GB	ID 3–9	OOD 10	vs f16 (paired exact McNemar)
f16	8,051,283,232	8.05	268/280	98/100	reference
Q6_K	3,306,259,232	3.31	268/280	97/100	p = 1.000 (ID), 1.000 (OOD)
Q5_K_M	2,889,511,712

No quantization is statistically distinguishable from f16 at either temperature on any slice. However, across all six out-of-distribution comparisons (3 quants × 2 temperatures) the point estimate favours f16 every time. Those tests share a reference and overlapping items so they are not independent, but the direction is consistent enough that Q6_K or Q5_K_M is recommended over Q4_K_M if you care about words of length 9–10. Q4_K_M is fine for short words.

ONNX (onnxruntime). Sizes are measured file bytes (model.onnx + model.onnx_data).

Read the accuracy column carefully. Only q4f16 has been oracle-tested. A dash below means NOT MEASURED — it does not mean zero, and it does not mean "fine". We deliberately did not spend the compute to grade the larger ONNX builds, because none of them is a plausible deployment target next to the GGUFs. If you intend to ship one of the unmeasured builds, measure it yourself with the oracle below; do not assume it inherits q4f16's numbers.

Table with columns: build, bytes, GB, ID 3–9, temp 0, OOD 10, temp 0, overall
build	bytes	GB	ID 3–9, temp 0	OOD 10, temp 0	overall
fp32	17,647,501,324	17.65	not measured	not measured	not measured
fp16	8,824,673,009	8.82	not measured	not measured	not measured
int8 (tfjs `q8`)

q4f16 at temp 0.8, same 380 items: ID 268/280 = 95.7%, OOD 94/100 = 94.0%, overall 362/380 = 95.3%. Per length at temp 0.8: 3:40/40, 4:40/40, 5:40/40, 6:38/40, 7:39/40, 8:36/40, 9:35/40, 10:94/100.

Per length at temp 0: 3:40/40, 4:40/40, 5:40/40, 6:40/40, 7:39/40, 8:37/40, 9:38/40, 10:98/100 — the same non-monotonic dip at length 8 the GGUF shows.

ONNX q4f16 scores 372/380 = 97.9% at temp 0 — indistinguishable from the f16 GGUF's 366/380 (p = 0.180), and directionally ahead. Conversion to ONNX plus 4-bit web quantization does not damage letter placement. Note the two runs use different decoding stacks (pure greedy under onnxruntime vs LM Studio's defaults), so this is not a controlled A/B; it is sufficient to show no degradation, not to claim an improvement.

Two things to know if you use the ONNX builds:

Qwen3 sets head_dim = 128 explicitly, but hidden_size / num_attention_heads = 2560/32 = 80. Tooling that derives the KV-cache shape from the latter will fail with Got invalid dimensions for input: past_key_values.N.key … Got: 80 Expected: 128. The published ONNX folders carry hidden_size = 4096 in their runtime config.json so that the derived value is correct; the graph itself is unmodified.
⚠️ THIS MODEL HAS NEVER BEEN RUN IN A BROWSER. Do not treat the ONNX build as a browser-ready artifact. Every ONNX number on this page was measured with onnxruntime on CPU, server-side. Nobody has loaded these weights in any browser, on any platform, ever. Client-side inference is unverified, not merely untuned.

The smallest build, q4f16, is 3,105,596,691 bytes (3.11 GB) of weights — 3,121,475,862 bytes (3.12 GB) for the complete folder including the tokenizer. That is the real, per-visitor download. It is above the ~2 GB single-ArrayBuffer ceiling enforced by some browser and platform combinations, and above the Cache-API/storage quotas many mobile browsers grant a single origin. There is a live possibility that this model simply cannot be loaded in a given browser at all. We have not established that it can be.

The wall that wasn't fundamental

This is the most interesting thing we measured, and it is a negative result about our own earlier conclusion.

The 3B predecessor (acrostic-3b-v3a, Qwen2.5-3B-Instruct, identical corpus, identical recipe, identical oracle) hit a hard wall at length 9 that would not move: 22/40 = 55.0%, and it was the same 22/40 in two independently trained variants on different data formats (a third variant scored 21/40 at temp 0 and 25/40 at temp 0.8). Because three attempts could not shift it, we concluded it was a character-sequencing limit of a 3B and shipped that model with a hard 6-letter cap.

Swapping the base to Qwen3-4B-Instruct-2507 and changing nothing else:

Table with columns: length 9, length 10 (OOD), ID 3–9
	length 9	length 10 (OOD)	ID 3–9
acrostic-3b-v3a (Qwen2.5-3B)	22/40 = 55.0%	50/100 = 50.0%	85.4%
acrostic-4b-v4a (Qwen3-4B)	36/40 = 90.0%	98/100 = 98.0%	95.7%

The wall was a property of that base model, not of the task, the data, or the format. Part of the gain is simply that the newer base starts far higher (0/380 → 82.5% ID untuned), so this is a generational and scale effect confounded together — we changed both parameter count and model generation at once and cannot separate them from this experiment.

The transferable lesson: "three training variants failed to move it" is evidence about the base, not evidence that a limit is fundamental. We stated it too strongly the first time.

The oracle

It checks that the output is exactly N non-empty lines whose first alphabetic characters spell the target word. That is all it checks.

It proves nothing about whether the poem is good, on topic, grammatical, evocative or funny. It is blind to quality by construction. Poem quality was never measured and is not claimed. Every accuracy number on this page is a letter-placement number.

python
import re

def check(word, text):
    """Return (pass, note). Pure string ops — no dictionary, no model, no judge."""
    lines = [l.strip() for l in (text or '').strip().split('\n') if l.strip()]
    if len(lines) != len(word):
        return False, f'line count {len(lines)} != {len(word)}'
    got = []
    for l in lines:
        m = re.search(r'[A-Za-z]', l)
        if not m:
            return False, 'line with no letter'
        got.append(m.group(0).upper())
    if ''.join(got) != word.upper():
        return False, f'initials {"".join(got)} != {word.upper()}'
    return True, 'ok'

Apply it to the poem text with the <n>. <L> | labels stripped — on the raw labelled output the first alphabetic character of each line is the label, which is the easier check.

Prompt format

Use the Qwen3 chat template with a single user message of this shape. Note that Qwen3-4B-Instruct-2507 inserts no default system prompt, and the model was trained that way.

markdown
You write acrostic poems in a numbered, labeled format. Write each line EXACTLY as
'<n>. <LETTER> | <poem text>' where <n> counts the lines, <LETTER> is that line's required
capital letter, and the poem text also begins with that letter. Output only the numbered lines.

Write a 5-line acrostic poem about a tide pool. The first letters of the 5 lines must spell
STASH. Format EVERY line EXACTLY as '<number>. <LETTER> | <poem text>' where <number> counts 1
to 5, <LETTER> is that line's required capital letter, and <poem text> is a poem line that also
begins with that letter. Output ONLY the 5 formatted lines, nothing else.

Deviating from this format is untested and the numbers above do not transfer to it.

Other measured properties

Table with columns: property, measurement
property	measurement
Lexical diversity (distinct-2, pooled, temp 0.8)	0.5648 vs untuned base 0.5688 = 99.3% retention; 0 exact-duplicate outputs in 380
General instruction-following (200 verifiable IFEval-style probes, 10 categories × 20)	199/200 = 99.5%, vs untuned base 193/200 = 96.5% — the tune improves general instruction-following by 3.0pp rather than taxing it

The narrow stylistic tune did not cause mode collapse and did not degrade unrelated instruction adherence. The base's only weak IF category was word_limit (14/20), which the tune fixes (19/20).

Training

Base: Qwen/Qwen3-4B-Instruct-2507 (Apache-2.0)
Method: LoRA, r=16, α=32, dropout 0.05, on q,k,v,o,gate,up,down; merged into the base weights for release. 33,030,144 trainable of 4,055,498,240 total parameters.
Schedule: 3 epochs, 1347 steps, lr 2e-4 cosine with 5% warmup, batch 1 × grad-accum 4, bf16, completion-only loss masking (the prompt is masked out). Final loss 0.7829.
Data: 1,995 training examples. Train/test words and topics disjoint at every length; length 10 excluded from training entirely.
Corpus construction: acrostics generated by a local teacher, openai/gpt-oss-20b (Apache-2.0), then labelled by execution — every candidate was run through the oracle above and kept or discarded on what the oracle actually returned, never on what the generator intended.
Target words: drawn deterministically from the Debian wamerican word list intersected with cracklib-small, so the vocabulary is ordinary English rather than dictionary tail.

Limitations

Length 8 is the realistic worst case (85.0%), not the longest length. See the non-monotonic curve above.
Rare initial letters (X, K, Q, Z) are the main lexical failure mode.
Untested above length 10 and untested below length 3.
English only.
Poem quality is unmeasured and unclaimed. The oracle is blind to it.
Failures are silent without the oracle. See the warning at the top.
The base can already do much of this (82.5% ID untuned) — do not use this card to argue that a fine-tune conferred a capability from nothing.
Never run in a browser. The ONNX build is server-measured only. See the ONNX section.

What is in this repo

Table with columns: path, what it is, size
path	what it is	size
`model.safetensors` + configs	the merged fp16 model — start here	8,044,982,080 B
`adapter/`	the LoRA adapter alone, if you would rather apply it yourself	132,187,888 B
`gguf/acrostic-4b-v4a-Q6_K.gguf`	llama.cpp, recommended local build	3,306,259,232 B
`gguf/acrostic-4b-v4a-Q5_K_M.gguf`	llama.cpp, smaller, statistically equivalent

Deliberately not published: the f16 GGUF (reproducible from model.safetensors with llama.cpp/convert_hf_to_gguf.py, and 8 GB of upload for nothing), and the ONNX fp32/fp16/int8/q4 builds (never oracle-tested — publishing untested weights alongside tested ones invites people to assume they are equivalent). Sizes for all of them are in the tables above so you can decide whether to build them yourself.

Licence

Apache License 2.0.

This model is a derivative of Qwen/Qwen3-4B-Instruct-2507, distributed by Alibaba Cloud under the Apache License 2.0. Its training corpus was distilled from openai/gpt-oss-20b, distributed under the Apache License 2.0; that licence and its accompanying usage policy place no restriction on training on model outputs or on redistributing a model trained on them.

There is no non-commercial restriction, no research-only clause, and no attribution-naming requirement on any component of this release.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.