s-g-labs

linlu-lora-v0.4-qwen3.6-27b

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Get help setting up a custom Dedicated Endpoints.

Talk with our engineer to get a quote for reserved GPU instances with discounts.

Output format

markdown
<think>
{Chinese inner monologue — optional}
</think>
[19:05] 晚上好呀
[19:06] [[sticker:挠头]]
[19:06] 在忙吗？

Each line is one WeChat-style bubble. [HH:MM] stamps encode the reply pacing (story-world clock — use deltas to drive typing delays, display wall-clock time). Resolve [[sticker:ref]] against your sticker catalog.

Quick start

python
import torch
import transformers
from peft import PeftModel
from transformers import AutoTokenizer, AutoConfig

BASE = "Qwen/Qwen3.6-27B"
ADAPTER = "s-g-labs/linlu-lora-v0.4-qwen3.6-27b"

# Use the exact architecture class — AutoModelForCausalLM resolves to a
# mismatched module tree and PEFT silently attaches nothing.
arch = AutoConfig.from_pretrained(BASE, trust_remote_code=True).architectures[0]
cls = getattr(transformers, arch)
tok = AutoTokenizer.from_pretrained(BASE, trust_remote_code=True)
model = cls.from_pretrained(BASE, dtype=torch.bfloat16, device_map="auto",
                            trust_remote_code=True)
model = PeftModel.from_pretrained(model, ADAPTER)
model.eval()

n = sum(1 for k, m in model.named_modules()
        if k.endswith("lora_B.default") and m.weight.abs().max() > 0)
assert n > 0, "adapter not loaded"

system = """你将扮演「林路」，一个真实、有血有肉的角色，正在用中文和「对方」聊天。
【消息格式】每条消息单独一行，以发送时刻开头，格式为「[HH:MM] 消息内容」。
发表情包时整条消息写作「[HH:MM] [[sticker:表情名]]」。"""

msgs = [{"role": "system", "content": system},
        {"role": "user", "content": "[19:02] 林老师晚上好呀"}]
prompt = "".join(f"<|im_start|>{m['role']}\n{m['content']}<|im_end|>\n"
                 for m in msgs) + "<|im_start|>assistant\n"
ids = tok(prompt, return_tensors="pt").to(model.device)
eos = [tok.eos_token_id, tok.convert_tokens_to_ids("<|im_end|>")]
out = model.generate(**ids, max_new_tokens=512, do_sample=True,
                     temperature=0.7, top_p=0.95, top_k=20,
                     eos_token_id=eos, pad_token_id=tok.pad_token_id or eos[0])
print(tok.decode(out[0][ids.input_ids.shape[1]:], skip_special_tokens=True))

Requires ~55 GB VRAM (bf16). For an OpenAI-compatible server, use the serve/ bundle from the v0.3 repo — identical protocol, just pass --base Qwen/Qwen3.6-27B --adapter <this repo>.

Sampling — IMPORTANT

Never use no_repeat_ngram_size — the timed format legitimately repeats [HH:MM] prefixes, and n-gram blocking forces the model into corrupted stamps ([18:48h], stray ] lines). A moderate repetition_penalty is fine and recommended: at 1.15 stamps stay 100% well-formed and the model pushes back on repeated questions instead of template-looping.

Table with columns: Parameter, Value
Parameter	Value
`temperature`	0.7
`top_p`	0.95
`top_k`	20
`repetition_penalty`	1.15 (keep ≤ 1.3)
`max_new_tokens`	512–1024

Conversation-history contract — IMPORTANT

Long-chat quality depends on feeding history back in the exact training shape. Consecutive same-role messages were merged into one turn during training (bubbles joined by \n), and every assistant turn starts with its <think>…</think> block (empty block if none). If your client appends each bubble as a separate assistant entry and strips <think>, the model drifts out of distribution and degenerates into one-word replies after ~30 turns.

markdown
user:      [19:02] 在吗\n[19:02] 林老师
assistant: <think>\n{monologue}\n</think>\n[19:05] 在的\n[19:05] 怎么了

Known limitations

Canon slip: unlike v0.2, v0.4 sometimes answers mental math correctly (persona says Lin Lu is bad at math). The 27B's ability leaks through.
Sticker refs are usually valid catalog entries but occasionally invented — render a text fallback for unknown refs.

Model provider

s-g-labs

Model tree

Base

Qwen/Qwen3.6-27B

Adapter

this model

Modalities

Input

Video, Text, Image

Output

Text

Pricing

Dedicated Endpoints

View details

Supported Functionality

Model APIs

Dedicated Endpoints

Container

More information

Model card

Explore FriendliAI today

Get started Talk to an engineer

Output format

markdown
<think>
{Chinese inner monologue — optional}
</think>
[19:05] 晚上好呀
[19:06] [[sticker:挠头]]
[19:06] 在忙吗？

Quick start

python
import torch
import transformers
from peft import PeftModel
from transformers import AutoTokenizer, AutoConfig

BASE = "Qwen/Qwen3.6-27B"
ADAPTER = "s-g-labs/linlu-lora-v0.4-qwen3.6-27b"

# Use the exact architecture class — AutoModelForCausalLM resolves to a
# mismatched module tree and PEFT silently attaches nothing.
arch = AutoConfig.from_pretrained(BASE, trust_remote_code=True).architectures[0]
cls = getattr(transformers, arch)
tok = AutoTokenizer.from_pretrained(BASE, trust_remote_code=True)
model = cls.from_pretrained(BASE, dtype=torch.bfloat16, device_map="auto",
                            trust_remote_code=True)
model = PeftModel.from_pretrained(model, ADAPTER)
model.eval()

n = sum(1 for k, m in model.named_modules()
        if k.endswith("lora_B.default") and m.weight.abs().max() > 0)
assert n > 0, "adapter not loaded"

system = """你将扮演「林路」，一个真实、有血有肉的角色，正在用中文和「对方」聊天。
【消息格式】每条消息单独一行，以发送时刻开头，格式为「[HH:MM] 消息内容」。
发表情包时整条消息写作「[HH:MM] [[sticker:表情名]]」。"""

msgs = [{"role": "system", "content": system},
        {"role": "user", "content": "[19:02] 林老师晚上好呀"}]
prompt = "".join(f"<|im_start|>{m['role']}\n{m['content']}<|im_end|>\n"
                 for m in msgs) + "<|im_start|>assistant\n"
ids = tok(prompt, return_tensors="pt").to(model.device)
eos = [tok.eos_token_id, tok.convert_tokens_to_ids("<|im_end|>")]
out = model.generate(**ids, max_new_tokens=512, do_sample=True,
                     temperature=0.7, top_p=0.95, top_k=20,
                     eos_token_id=eos, pad_token_id=tok.pad_token_id or eos[0])
print(tok.decode(out[0][ids.input_ids.shape[1]:], skip_special_tokens=True))

Requires ~55 GB VRAM (bf16). For an OpenAI-compatible server, use the serve/ bundle from the v0.3 repo — identical protocol, just pass --base Qwen/Qwen3.6-27B --adapter <this repo>.

Sampling — IMPORTANT

Table with columns: Parameter, Value
Parameter	Value
`temperature`	0.7
`top_p`	0.95
`top_k`	20
`repetition_penalty`	1.15 (keep ≤ 1.3)
`max_new_tokens`	512–1024

Conversation-history contract — IMPORTANT

markdown
user:      [19:02] 在吗\n[19:02] 林老师
assistant: <think>\n{monologue}\n</think>\n[19:05] 在的\n[19:05] 怎么了

Known limitations

Canon slip: unlike v0.2, v0.4 sometimes answers mental math correctly (persona says Lin Lu is bad at math). The 27B's ability leaks through.
Sticker refs are usually valid catalog entries but occasionally invented — render a text fallback for unknown refs.

linlu-lora-v0.4-qwen3.6-27b

Get help setting up a custom Dedicated Endpoints.

README

Output format

Quick start

Sampling — IMPORTANT

Conversation-history contract — IMPORTANT

Known limitations

Explore FriendliAI today

README

Output format

Quick start

Sampling — IMPORTANT

Conversation-history contract — IMPORTANT

Known limitations