GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
bmo-story-lab
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
bmo-story-lab
Model Tree
Qwen/Qwen3-Embedding-0.6B
Input Modalities
Output Modalities
Supported Functionality
MASSIVE ru test, 1,000 utterances. "Unseen 15" = choosing among the 15 intents never seen in training. BTZSC = mean of AG News, Emotion, Banking77 (English). ECE = calibration error after temperature scaling, lower is better.
| Model | Size | MASSIVE ru | Unseen 15 | BTZSC | typed-decisions | ECE ↓ |
|---|---|---|---|---|---|---|
| bmo-intent 1.0 | 0.6B | 86.7 | 95.3 | 65.3 | 38.2 | 3.3 |
| Qwen3-Embedding-0.6B (base) | 0.6B | 59.5 | 88.4 | 65.3 | 40.7 | 4.0 |
| multilingual-e5-small, distilled the same way | 0.12B | 82.4 | 90.2 | 53.3 | 34.4 | 2.8 |
| rubert-tiny2, distilled the same way | 0.03B | 81.3 | 86.0 | 35.4 | 36.1 | 4.5 |
| Qwen3-Embedding-4B + heads (teacher) | 4B | 86.7 | 95.7 | 65.0 | — | 3.8 |
| Qwen3-Reranker-8B | 8B | 78.9 | 96.3 | 67.7 | 51.5 | 4.9 |
| Jev 1.13 (TypeSafe, published numbers) | — | — | — | 75.3 | 72.7 | — |
bmo-intent numbers are the mean of three seeds (86.5 / 86.4 / 87.1). This repository holds the seed-2 adapter: 87.1, reproduced from these files with the code below.
A LoRA adapter (18 MB) on top of Qwen/Qwen3-Embedding-0.6B. The utterance goes in with the Qwen instruction, each intent as "description (key)"; the vector is the last token, L2-normalised.
python
import torch, torch.nn.functional as Ffrom transformers import AutoModel, AutoTokenizerfrom peft import PeftModel tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-Embedding-0.6B")tok.padding_side = "right"base = AutoModel.from_pretrained("Qwen/Qwen3-Embedding-0.6B")model = PeftModel.from_pretrained(base, "bmo-story-lab/bmo-intent-1.0").eval() # Russian instruction: "Determine the voice-assistant user's intent from the utterance"TASK = "Определи намерение пользователя голосового ассистента по его реплике" @torch.no_grad()def embed(texts): enc = tok(texts, padding=True, truncation=True, max_length=512, return_tensors="pt") h = model(**enc).last_hidden_state v = h[torch.arange(len(texts)), enc["attention_mask"].sum(-1) - 1] # last token return F.normalize(v, dim=-1) intents = { "alarm_set": "поставить будильник, разбудить в определённое время", # set an alarm, wake me at a time "weather_query": "узнать погоду", # check the weather "play_music": "включить музыку, песню, исполнителя", # play music, a song, an artist}q = embed([f"Instruct: {TASK}\nQuery:разбуди меня завтра в семь"]) # "wake me up at seven tomorrow"o = embed([f"{d} ({k})" for k, d in intents.items()])probs = (33.2 * q @ o.T).softmax(-1)[0] # 33.2 = learned logit scaleprint(dict(zip(intents, probs.round(decimals=3).tolist())))Intents are just text: add or rename them without retraining. For calibrated confidence, fit a temperature on a few hundred labelled utterances from your own domain.
Qwen3-Embedding-0.6B, LoRA r=16 in the encoder, fp16 base with fp32 adapters, 2× T4 on Kaggle (~12 min per seed). Loss: KL to Qwen3-Embedding-4B soft labels + cross-entropy to the gold intent. 45 of 60 MASSIVE ru intents in training.
Модель выбора намерения для локального голосового ассистента: Qwen3-Embedding-0.6B с LoRA, обученная на разметке Qwen3-Embedding-4B. На MASSIVE ru (60 намерений) — 86,7, как у учителя 4B, при размере 1,2 ГБ. Слабые места — вопросы «да/нет» и распознанная речь (−6,4 пункта на Whisper small). Полный отчёт на русском: bmo-story.tech/ru/lab/bmo-intent.
Apache 2.0 · bmo lab
import torch, torch.nn.functional as Ffrom transformers import AutoModel, AutoTokenizerfrom peft import PeftModel tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-Embedding-0.6B")tok.padding_side = "right"base = AutoModel.from_pretrained("Qwen/Qwen3-Embedding-0.6B")model = PeftModel.from_pretrained(base, "bmo-story-lab/bmo-intent-1.0").eval() # Russian instruction: "Determine the voice-assistant user's intent from the utterance"TASK = "Определи намерение пользователя голосового ассистента по его реплике" @torch.no_grad()def embed(texts): enc = tok(texts, padding=True, truncation=True, max_length=512, return_tensors="pt") h = model(**enc).last_hidden_state v = h[torch.arange(len(texts)), enc["attention_mask"].sum(-1) - 1] # last token return F.normalize(v, dim=-1) intents = { "alarm_set": "поставить будильник, разбудить в определённое время", # set an alarm, wake me at a time "weather_query": "узнать погоду", # check the weather "play_music": "включить музыку, песню, исполнителя", # play music, a song, an artist}q = embed([f"Instruct: {TASK}\nQuery:разбуди меня завтра в семь"]) # "wake me up at seven tomorrow"o = embed([f"{d} ({k})" for k, d in intents.items()])probs = (33.2 * q @ o.T).softmax(-1)[0] # 33.2 = learned logit scaleprint(dict(zip(intents, probs.round(decimals=3).tolist())))