GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
trajrealm
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
trajrealm
Model Tree
Qwen/Qwen2.5-1.5B-Instruct
Input Modalities
Output Modalities
Supported Functionality
On a rhyme that was held out of training:
| Original | Fine-tuned |
|---|---|
| There was an old woman tossed in a basket,Seventeen times as high as the moon;But where she was going no mortal could tell,For under her arm she carried a broom. | There was an old woman tossed in a coffin,Seventeen times as low as the grave;But where she was going no mortal could tell,For under her arm she carried a broom. |
| There was an old man in a Marsh,Whose manners were futile and harsh;He sate on a log, and sang songs to a frog,That instructive old man in a Marsh. | There was an old stranger in a marsh,Whose manners were pallid and harsh;He sat by the graveside at night, and sang waltzes to frogs,That instructive old stranger in a marsh. |
Given the same request, the base model writes long, generic Dracula ballads that ignore the original rhyme's form.
python
import torchfrom peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer base = "Qwen/Qwen2.5-1.5B-Instruct"tokenizer = AutoTokenizer.from_pretrained(base)model = AutoModelForCausalLM.from_pretrained(base, dtype=torch.float16, device_map="auto")model = PeftModel.from_pretrained(model, "trajrealm/vamprhyme-qwen2.5-1.5b-lora") messages = [ {"role": "system", "content": "You are a creative writing assistant. You write poems, nursery rhymes, " "parodies and fiction in any genre, including gothic horror and dark fairy tales."}, {"role": "user", "content": "Write a Dracula version of this nursery rhyme:\n\n" "Jack and Jill went up the hill\nTo fetch a pail of water;\n" "Jack fell down and broke his crown,\nAnd Jill came tumbling after."},]text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)inputs = tokenizer(text, return_tensors="pt").to(model.device)output = model.generate(**inputs, max_new_tokens=150, do_sample=True, temperature=0.8, top_p=0.9)print(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))Use the system prompt above: the adapter was trained with it. Quoting the full rhyme in the request works best. On a CPU, load the model with dtype=torch.float32 and drop device_map.
Data
Setup
| Setting | Value |
|---|---|
| Method | LoRA, rank 16, alpha 32, dropout 0.05 |
| Target modules | q, k, v, o, gate, up and down projections in all 28 layers |
| Trainable parameters | 18.5M (1.18% of the model) |
| Optimizer | AdamW, peak learning rate 2e-4, warmup then cosine decay |
| Batch | 8 (4 x 2 gradient accumulation) |
| Precision | fp16 mixed precision, T4 GPU |
| Checkpoint | Best validation loss, step 150 of 321 (end of the first epoch) |
Validation loss (per token) at the saved checkpoint, against the base model:
| Base | Fine-tuned | |
|---|---|---|
| Rhyme pairs | 1.989 | 1.418 |
| Nursery rhymes | 3.260 | 2.685 |
| Gothic horror | 3.447 | 3.061 |
| Children's dark fiction | 3.011 | 2.561 |
The adapter is released under Apache 2.0, the license of the base model.
import torchfrom peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer base = "Qwen/Qwen2.5-1.5B-Instruct"tokenizer = AutoTokenizer.from_pretrained(base)model = AutoModelForCausalLM.from_pretrained(base, dtype=torch.float16, device_map="auto")model = PeftModel.from_pretrained(model, "trajrealm/vamprhyme-qwen2.5-1.5b-lora") messages = [ {"role": "system", "content": "You are a creative writing assistant. You write poems, nursery rhymes, " "parodies and fiction in any genre, including gothic horror and dark fairy tales."}, {"role": "user", "content": "Write a Dracula version of this nursery rhyme:\n\n" "Jack and Jill went up the hill\nTo fetch a pail of water;\n" "Jack fell down and broke his crown,\nAnd Jill came tumbling after."},]text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)inputs = tokenizer(text, return_tensors="pt").to(model.device)output = model.generate(**inputs, max_new_tokens=150, do_sample=True, temperature=0.8, top_p=0.9)print(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))