GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
Rychanfox
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
Rychanfox
Model Tree
Qwen/Qwen2.5-1.5B-Instruct
Input Modalities
Output Modalities
Supported Functionality
This model produces better labels than the classifier when it works, but it is only reliable 43% of the time and ~20x+ slower:
| Exact-match micro F1 | Faithful generations | Latency | |
|---|---|---|---|
| This model (faithful examples only) | 0.23 | 9/21 (43%) | ~21.5s/abstract |
| Classifier | 0.16 | n/a (always well-formed) | <1s/abstract |
"Faithful" means the tag-stripped output exactly matches the source text, character for character. The other 57% of the time, this model duplicates or drops small pieces of the source text while tagging it -- specific, diagnosed failure modes (not random paraphrasing), documented in the findings log. Because a highlighter that silently alters the text it's supposed to be annotating is a correctness problem, not just a quality one, the classifier is the model actually used in the project's interactive demo. This adapter is published for the comparison itself, not as a recommended production choice.
python
import torchfrom peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfigtokenizer = AutoTokenizer.from_pretrained("Rychanfox/semantic-highlighting-qwen2.5-1.5b-lora")quant_config = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",bnb_4bit_compute_dtype=torch.bfloat16, bnb_4bit_use_double_quant=True,)base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct", quantization_config=quant_config, device_map="auto")model = PeftModel.from_pretrained(base_model, "Rychanfox/semantic-highlighting-qwen2.5-1.5b-lora")
Prompt format, faithfulness checking, and tag parsing utilities are in the
project repo's slm/ module (build_messages, parse_tags, predict).
Trained on the same Rychanfox/semantic-highlighting-abstracts dataset as
the classifier, serialized into the [Label]...[/Label] inline-tag format
as the target completion (system prompt describes the label schema; loss is
masked to the completion only, not the prompt).