GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
abubakar-siddik
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
abubakar-siddik
Model Tree
Qwen/Qwen3.5-2B
Input Modalities
Output Modalities
Supported Functionality
The adapter is meant for research on tool use in Bengali and Banglish, for example as a baseline on the Jontro benchmark or as a starting point for further fine-tuning. Given a system prompt, a list of tools and a conversation, it decides whether to call a tool, which tools to call (several at once when the request needs it) and with which argument values, and it answers in Bengali script. It was trained on synthetic conversations with simulated services, so it is not ready to act on real accounts, payments or bookings, and it should not be used for medical, financial or legal advice.
The example below loads the base model with the adapter, takes one held-out conversation from the dataset and asks for the next assistant turn. Use the system prompt and tool format of the dataset, since the model was trained with them, and keep thinking off, as it was off in training and evaluation.
python
import jsonimport torchfrom datasets import load_datasetfrom peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer BASE = "Qwen/Qwen3.5-2B"ADAPTER = "abubakar-siddik/jontro-qwen3.5-2b-lora" tok = AutoTokenizer.from_pretrained(BASE)model = AutoModelForCausalLM.from_pretrained( BASE, dtype=torch.bfloat16, device_map="auto")model = PeftModel.from_pretrained(model, ADAPTER) def drop_nulls(x): # Released schemas list unused parameters as null. if isinstance(x, dict): return {k: drop_nulls(v) for k, v in x.items() if v is not None} return x def as_tool(t): schema = drop_nulls(json.loads(t["parameters_json_schema"])) return {"type": "function", "function": {"name": t["name"], "description": t["description"], "parameters": schema}} # A held-out test conversation, cut at its first user turn.row = load_dataset("abubakar-siddik/Jontro", split="test")[0]tools = [as_tool(t) for t in row["tools"]]messages = [ {"role": "system", "content": row["system_prompt"]}, {"role": "user", "content": row["messages"][0]["content"]},] inputs = tok.apply_chat_template( messages, tools=tools, add_generation_prompt=True, enable_thinking=False, return_dict=True, return_tensors="pt",).to(model.device)out = model.generate(**inputs, max_new_tokens=512, do_sample=False)new = out[0, inputs["input_ids"].shape[1]:]print(tok.decode(new, skip_special_tokens=False))The model writes tool calls in the Qwen3.5 format, as <tool_call> blocks that hold a <function=...> element with one <parameter=...>
element per argument. To serve it with vLLM, run the base model with --enable-lora, --enable-auto-tool-choice and
--tool-call-parser qwen3_coder, load this repository as the LoRA module, and pass enable_thinking: false in chat_template_kwargs;
the server then returns the calls in the OpenAI tool_calls format. After a call, append the tool result as a tool message and generate
again for the next turn.
Only the training split was used. It holds 1,300 conversations and shares no task goal with the 700 test conversations, so every test goal is unseen. Each assistant turn is one training example, with the loss on assistant tokens only; every turn that makes a tool call is kept, together with a random quarter of the turns that answer in text.
This variant adds 748 replay examples to the Jontro training conversations: the base model's own greedy answers to unrelated prompts, 469 of them given the Jontro system prompt and tools (the right behaviour is to answer without calling a tool) and 279 without tools. The prompts come from the aya dataset for Bengali and OpenAssistant (oasst2) for English, both Apache-2.0, and do not overlap the retention prompts. Answers longer than 512 tokens were cut at that length, and the end token of a cut answer was not trained (422 of 748). The replay data were added after an adapter trained on the Jontro conversations alone failed two of the retention checks described below, so this design is a response to that result rather than a plan made in advance.
The adapter was trained with LoRA at rank 32 (alpha equal to the rank, dropout 0.05) on all linear layers, including the
linear-attention projections of the Qwen3.5 architecture. Optimisation used AdamW at a learning rate of 0.0002 with a cosine schedule,
3% warm-up, gradient clipping at 1.0, an effective batch of 16 and 2 epochs, in bf16 on one H100 80GB
for about 64 minutes. No hyperparameters were searched, and the adapter from the final epoch is the one
released. The run used seed 0, code commit 4c77e6e and data hash 775a4047d8ebb48b.
The benchmark scores every tool-calling turn of the 700 test conversations (1,720 decision points over 48 held-out goals). At each decision point the model sees the true conversation so far and must produce the right set of calls; this is the teacher-forced protocol of the paper. Tool selection asks whether the set of tool names is right, exact arguments whether the names and every argument value are right, and a conversation counts as fully correct only if every one of its decision points has exact arguments. Decoding was greedy with thinking off. All scores are percentages; brackets give 95% bootstrap intervals, which resample task goals rather than conversations and are therefore wide.
| Model | Tool selection | Exact arguments | Slot accuracy | Fully correct conversations |
|---|---|---|---|---|
| Qwen3.5-2B, no adapter | 41.5 [34.7, 48.4] | 22.4 [17.5, 27.6] | 37.7 | 7.3 [1.8, 16.4] |
| Qwen3.5-4B, no adapter | 44.3 [36.7, 52.9] | 24.6 [19.0, 30.8] | 36.9 | 10.1 [3.2, 20.4] |
| Qwen3.5-9B, no adapter | 44.2 [36.7, 52.5] | 24.0 [18.9, 29.9] | 35.9 | 8.4 [2.0, 17.9] |
| This adapter | 84.6 [79.7, 89.2] | 54.0 [45.2, 62.5] | 82.2 | 27.1 [17.2, 38.6] |
The larger base models do no better than the 2B model without fine-tuning, mostly because they often answer in text where a tool call is needed. The adapter was trained on data from the same generator as the test set, so these numbers show what the training split teaches, not how the model would fare with real users.
Fine-tuning on a narrow task can damage general behaviour, so the adapter was compared with the same base model on checks fixed before any adapter was evaluated: 48 TruthfulQA forced-choice items, 100 unrelated requests asked with the Jontro system prompt and tools offered (the right behaviour is not to call a tool), 100 general chat prompts (50 in Bengali or Banglish, 50 in English) scored for coherence from 0 to 100 by a Gemini judge, whether the reply is in the user's language, and the change in negative log-likelihood of held-out human-written replies (lower is better).
| Check | Qwen3.5-2B, no adapter | This adapter |
|---|---|---|
| TruthfulQA, correct of 48 | 35 | 33 |
| Tool calls on unrelated requests | 11% | 0% |
| Reply in the user's language | 98% | 98% |
| Coherence, all prompts (0 to 100) | 62.0 | 55.8 |
| Coherence, Bengali prompts | 38.1 | 27.2 |
| Coherence, English prompts | 86.0 | 84.4 |
| Held-out Bengali NLL, change from base | -1.09 | |
| Held-out English NLL, change from base | +0.76 |
This adapter makes no unrelated tool calls (0%, base 11%) and keeps English coherence (84.4 against 86.0). Bengali chat coherence is 27.2 against 38.1 for the base model, so fine-tuning still costs some Bengali chat quality, although much less than without the replay data. It passes all four gates of the retention protocol as written, although the overall coherence gap (6.2 points, limit 10) is helped by the English prompts. Likelihood of held-out English text is worse than the base (+0.76 nats per token), probably because the replay data are the model's own greedy outputs.
TruthfulQA at 48 items cannot detect small differences, and the judge is a Gemini model from the same family that generated the corpus, whose Bengali scores have not been checked against human ratings.
All results come from one training run with one seed, so run-to-run variation is unknown. Only the teacher-forced protocol has been run; when the model's own calls feed later turns, errors can compound and conversation-level success is likely lower. The training and test conversations were written by a language model playing the user, so the adapter has seen a tidier Banglish and a narrower range of requests than real users produce. Bengali chat quality outside the tool-use setting is below that of the base model.
If you use the adapter, please cite the paper:
bibtex
@misc{siddik2026jontro, author = {Siddik, Abu Bakar and Islam, Afroza}, title = {Jontro: A Multi-Turn {Bangla} and {Banglish} Tool-Use Corpus and Benchmark}, year = {2026}, note = {Preprint}}The entry will be updated with the arXiv identifier once the preprint is posted. To refer to the released adapter itself, you can also cite:
bibtex
@misc{siddik2026jontromodel, author = {Siddik, Abu Bakar}, title = {Jontro Qwen3.5-2B {LoRA}: A Tool-Use Adapter for {Bangla} and {Banglish}}, year = {2026}, publisher = {Hugging Face}, howpublished = {\url{https://huggingface.co/abubakar-siddik/jontro-qwen3.5-2b-lora}}, note = {Apache 2.0, fine-tuned from Qwen3.5-2B on Jontro 2.0}}The training data should be cited as well; its entry is on the dataset page.
The adapter is released under the Apache 2.0 license, the license of the base model. The Jontro dataset is released under CC BY 4.0.
import jsonimport torchfrom datasets import load_datasetfrom peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer BASE = "Qwen/Qwen3.5-2B"ADAPTER = "abubakar-siddik/jontro-qwen3.5-2b-lora" tok = AutoTokenizer.from_pretrained(BASE)model = AutoModelForCausalLM.from_pretrained( BASE, dtype=torch.bfloat16, device_map="auto")model = PeftModel.from_pretrained(model, ADAPTER) def drop_nulls(x): # Released schemas list unused parameters as null. if isinstance(x, dict): return {k: drop_nulls(v) for k, v in x.items() if v is not None} return x def as_tool(t): schema = drop_nulls(json.loads(t["parameters_json_schema"])) return {"type": "function", "function": {"name": t["name"], "description": t["description"], "parameters": schema}} # A held-out test conversation, cut at its first user turn.row = load_dataset("abubakar-siddik/Jontro", split="test")[0]tools = [as_tool(t) for t in row["tools"]]messages = [ {"role": "system", "content": row["system_prompt"]}, {"role": "user", "content": row["messages"][0]["content"]},] inputs = tok.apply_chat_template( messages, tools=tools, add_generation_prompt=True, enable_thinking=False, return_dict=True, return_tensors="pt",).to(model.device)out = model.generate(**inputs, max_new_tokens=512, do_sample=False)new = out[0, inputs["input_ids"].shape[1]:]print(tok.decode(new, skip_special_tokens=False))@misc{siddik2026jontro, author = {Siddik, Abu Bakar and Islam, Afroza}, title = {Jontro: A Multi-Turn {Bangla} and {Banglish} Tool-Use Corpus and Benchmark}, year = {2026}, note = {Preprint}}@misc{siddik2026jontromodel, author = {Siddik, Abu Bakar}, title = {Jontro Qwen3.5-2B {LoRA}: A Tool-Use Adapter for {Bangla} and {Banglish}}, year = {2026}, publisher = {Hugging Face}, howpublished = {\url{https://huggingface.co/abubakar-siddik/jontro-qwen3.5-2b-lora}}, note = {Apache 2.0, fine-tuned from Qwen3.5-2B on Jontro 2.0}}