GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
wowsocool123
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
wowsocool123
Model Tree
Qwen/Qwen3-4B-Instruct-2507
Input Modalities
Output Modalities
Supported Functionality
Evaluated on 24 hand-written held-out tasks in two tiers. A task passes only if its pattern satisfies every held-out test case; some test cases are shown in the prompt and the rest are withheld, so literal-matching scores zero.
| tier | what it tests | before | after |
|---|---|---|---|
| hard | composition (lookaheads, backrefs, counting) | 0.250 | 0.417 |
| seed | canonical patterns (postcode, IPv4, email) | 0.333 | 0.167 |
Fine-tuning traded memorised canonical patterns for compositional skill. The hard tier improved 67% relative. The seed tier regressed — those are patterns the base model already half-knew from pretraining, and training on composition tasks partially overwrote them. Both numbers are reported because the trade is the actual finding.
For reference, Gemini Flash-Lite scores 1.000 / 0.750 on the same tiers.
python
from peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizerbase = "Qwen/Qwen3-4B-Instruct-2507"tok = AutoTokenizer.from_pretrained(base)model = AutoModelForCausalLM.from_pretrained(base, device_map="auto")model = PeftModel.from_pretrained(model, "wowsocool123/qwen3-4b-regex-lora")prompt = """Write a Python regular expression for this task.Task: Match a string of 8 to 16 characters with no whitespace, containing at \least one lowercase letter, at least one uppercase letter, and at least one digit.The pattern is applied with re.fullmatch (it must match the ENTIRE string).Reply with ONLY the regular expression pattern. No explanation, no code fences,no quotes, no re.compile() call. Just the raw pattern."""text = tok.apply_chat_template([{"role": "user", "content": prompt}],tokenize=False, add_generation_prompt=True)out = model.generate(**tok(text, return_tensors="pt").to(model.device),max_new_tokens=256, do_sample=False)print(tok.decode(out[0], skip_special_tokens=True))
Prompt format matters. The adapter was trained on the exact template above —
task description, re.fullmatch or re.search note, a few example strings,
then the "reply with ONLY the pattern" instruction. Straying far from it will
degrade output.
The model prefixes every answer with an empty <think>\n\n</think> block.
Qwen3's chat template injects a reasoning block into the final assistant turn,
so all training targets carried one and the model learned to reproduce it.
Strip it before using the output:
python
import repattern = re.sub(r"<think>.*?</think>", "", raw, flags=re.DOTALL).strip()
Other limitations: 24 held-out tasks is a small evaluation, so a one-task swing moves the score by 0.083; results come from a single training run with no seed-variance estimate; and training instructions were templated rather than natural English, so unusual phrasing may degrade output.
| Method | QLoRA (4-bit NF4 base, LoRA adapter) |
| Rank / alpha | 16 / 32 |
| Target modules | q, k, v, o, gate, up, down |
| Learning rate | 2e-4, linear schedule |
| Epochs | 3 |
| Effective batch | 8 (2 × 4 grad accum) |
| Loss | responses only (prompt masked) |
| Hardware | Colab T4, ~5 min |
adapter_config.json records the base as
unsloth/qwen3-4b-instruct-2507-unsloth-bnb-4bit — Unsloth's pre-quantised
mirror of Qwen/Qwen3-4B-Instruct-2507, which is what was actually trained
against. It loads fine on either.
Training data was generated programmatically, not by an LLM. Nine templates (one per regex technique) emit the instruction, the reference pattern, and the test cases together, then every candidate is rejected unless the pattern passes its own tests.
An earlier LLM-based generator produced 9 verified tasks in ~20 minutes with six of nine techniques stuck at zero — asking a model to "use a backreference" does not make it use one. Templates produced 400 verified tasks in 0.4 seconds with every technique evenly represented.
Code, evaluation harness, and full write-up: github.com/asadusman123/regex-finetune