GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
felipe53
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
felipe53
Model Tree
unsloth/Qwen3-1.7B-unsloth-bnb-4bit
Input Modalities
Output Modalities
Supported Functionality
| Passage | Correct call |
|---|---|
| "Newton was frustrated when the experiment failed." | tag Newton (person) |
| "We applied the Newton method to approximate the root." | don't tag (method) |
| "I visited Chelsea last summer and loved it." | don't tag (place) |
| "Chelsea helped me revise my thesis statement." | tag Chelsea (person) |
| "thanks, Sam — that explanation finally clicked" | tag Sam (first-name-only person) |
Same token, opposite call, decided by context. Run the side-by-side yourself with
markdown
python -m src.demo --adapter <this-adapter>
Evaluated on 51 quarantined hard cases (ambiguous names in context). Base = the prompted
4-bit Qwen3-1.7B on the same backend; tuned = base + this adapter. The comparison is prompt vs.
fine-tune, which is the assignment's baseline. Full tables + 95% bootstrap CIs: docs/results.md
→ gpt551.
| Metric | Prompted base | Fine-tuned (gpt551) |
|---|---|---|
| F5 (recall-weighted F) | 0.507 | 0.847 |
| Recall | 0.518 | 0.852 |
| Over-tag rate ↓ | 0.549 | 0.157 |
| Integrity violation ↓ | 0.588 | 0.020 |
| Leakage rate ↓ | 0.255 | 0.078 |
| Pass rate |
Per-category: person_vs_eponym recall 0.33 → 1.00, person_vs_place recall → 1.00,
negative_trap pass 0.40 → 1.00. Generalization to names never seen in training was verified
on a held-out-names probe (base→tuned recall 0.08 → 1.00) and an OOD probe (recall 0.05 → 0.89) —
the judgment generalized, it did not memorize.
For context, a frontier API model (gpt-4.1) scored pass 0.88 on the same 51 hard cases; the
1.7B tunes are competitive with it and actually beat it on recall (the frontier model is
precision-first and under-recalls). See docs/eval-engine-comparison.md.
| Base model | unsloth/Qwen3-1.7B-unsloth-bnb-4bit (4-bit, non-thinking mode) |
| Method | QLoRA (unsloth + bitsandbytes 4-bit), bfloat16 compute (A100/Ampere) |
| LoRA | r=32, α=32, dropout 0.0; q/k/v/o + gate/up/down proj |
| Optim | lr=2e-4, 3 epochs, eff. batch 16 (2×8), linear schedule, warmup 0.05, wd 0.01, seed 0 |
| Masking / seq | completion-only masking; max seq 2048 |
| Config | configs/train.yaml — frozen since Day 4 (data-only iteration, no HP tuning) |
| Run | 156 steps / 3 epochs, loss 0.474 → ~2e-4, no NaN |
Config was frozen deliberately: a project rule forbids touching hyperparameters to paper over data problems — failure modes were fixed by generating targeted data, not by tuning lr/r/epochs.
818 train / 90 val (sft-v3-gpt551 splits). Generated by a live OpenAI-compatible teacher
(via the TrueFoundry LLM Gateway) writing passages, plus an independent verifier pass that
rejects teacher/verifier disagreements. Built from matched minimal pairs
(person_vs_common / _place / _eponym / possessive) so the model sees the same token used
both ways. Dataset card: docs/dataset-card-v3.md.
Quality-gate + leakage drops: 134 verifier-disagreement, 98 eval-surface-leak, 48 negative-trap-has-name, plus a few label-integrity rejects.
The quarantined evaluation sets are physically separate and were never fed to the teacher,
augmentation, or training. Independently re-verified post-hoc: 0 exact and 0 substring overlaps
between the 818/90 training splits and all 201 quarantined eval inputs (hardcases / adversarial /
held-out-names / OOD). Enforced in CI by tests/test_no_eval_leakage.py.
unwrap(project(...)) == original by construction). Prefer the projected path in production.possessive, n=3).python
from peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizerBASE = "Qwen/Qwen3-1.7B" # or the 4-bit unsloth base on CUDAtok = AutoTokenizer.from_pretrained(BASE)model = AutoModelForCausalLM.from_pretrained(BASE)model = PeftModel.from_pretrained(model, "<this-repo-id>")# Easiest path — the repo's Tagger contract (handles the non-thinking chat template):# from src.infer import load_hf_tagger# tagger = load_hf_tagger(adapter="<adapter-path>")# tagger.tag("Chelsea helped me revise my thesis.")# → "⟨NAME⟩Chelsea⟨/NAME⟩ helped me revise my thesis."
The model expects the system prompt in src/common/prompts.py and Qwen3's non-thinking chat
template. See src/infer.py and src/demo.py in the repo.
sft-v3-gpt551 (r=32/α=32, checkpoint-156).notebooks/v3_colab_train_eval.ipynb — live teacher via --provider openai
with OPENAI_BASE_URL/TEACHER_MODEL → frozen configs/train.yaml QLoRA → base-vs-tuned on
eval/hardcases.docs/model-card-gpt551.md, docs/model-card-v3.md.| 0.353 |
| 0.824 |
| Consistency | 0.375 | 0.562 |