GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
emretmrk
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
emretmrk
Model Tree
Qwen/Qwen3.5-4B
Input Modalities
Output Modalities
Supported Functionality
Bu repo, Qwen/Qwen3.5-4B modeli üzerinde önce DAPT, ardından SFT ile eğitilmiş final LoRA adapter dosyalarını içerir.
Bu adapter:
amaçlarıyla eğitilmiştir.
Benchmark sistemleri doğrudan model adresi istediğinde merged model repo'sunun kullanılması önerilir. Bu adapter repo'su ise yeniden üretilebilirlik, düşük depolama maliyeti, PEFT inference ve ek fine-tuning için uygundur.
| Item | Value |
|---|---|
| Base model | Qwen/Qwen3.5-4B |
| Adapter type | LoRA |
| PEFT | Yes |
| Target modules | all-linear |
Rank (r) | 32 |
| LoRA alpha | 64 |
| LoRA dropout | 0.05 |
| Training sequence | DAPT → SFT |
| Domain | General Turkish + Turkish Finance |
| Training modality | Text-only |
Qwen3.5-4B is a multimodal upstream model. This adapter was trained using text-only datasets; visual capabilities were not explicitly optimized or evaluated.
Datasets
Mixture
The mixture was constructed on a sample/row basis.
DAPT hyperparameters
| Parameter | Value |
|---|---|
| LoRA target | all-linear |
Rank (r) | 32 |
| Alpha | 64 |
| Dropout | 0.05 |
| Learning rate | 5e-5 |
| Optimizer steps | 1500 |
| Max sequence length | 2048 |
| Packing |
Datasets
Data strategy
alpaca-gpt4-tr examplesSFT hyperparameters
| Parameter | Value |
|---|---|
| Starting adapter | DAPT adapter |
| LoRA target | all-linear |
Rank (r) | 32 |
| Alpha | 64 |
| Dropout | 0.05 |
| Learning rate | 1e-4 |
| Epochs | 2 |
| Max sequence length |
python
import torchfrom transformers import AutoProcessor, AutoModelForMultimodalLMfrom peft import PeftModel BASE_MODEL = "Qwen/Qwen3.5-4B"ADAPTER_ID = "emretmrk/Qwen3.5-4B-TR-Finance-LoRA" processor = AutoProcessor.from_pretrained(BASE_MODEL) base_model = AutoModelForMultimodalLM.from_pretrained( BASE_MODEL, torch_dtype="auto", device_map="auto",) model = PeftModel.from_pretrained( base_model, ADAPTER_ID, is_trainable=False,) model.eval()python
messages = [ { "role": "user", "content": [ { "type": "text", "text": "Merkez bankası politika faizini artırdığında ekonomide hangi kanallar etkilenebilir?" } ], }] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt",).to(model.device) with torch.no_grad(): outputs = model.generate( **inputs, max_new_tokens=256, do_sample=False, ) answer = processor.decode( outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True,) print(answer)python
model = PeftModel.from_pretrained( base_model, ADAPTER_ID, is_trainable=True,)python
merged_model = model.merge_and_unload()merged_model.save_pretrained( "./Qwen3.5-4B-TR-Finance", safe_serialization=True,)processor.save_pretrained("./Qwen3.5-4B-TR-Finance")| Benchmark | Status |
|---|---|
| TurkBench | Pending |
| OpenLLMTurkishLeaderboard | Pending |
| Held-out Turkish Finance QA | Pending |
| Base vs DAPT vs DAPT+SFT ablation | Pending |
Qwen/Qwen3.5-4B base model.The upstream Qwen/Qwen3.5-4B model is released under the Apache License 2.0.
The adapter was trained using third-party datasets. Users should separately review the dataset cards and any applicable usage terms. Not all upstream dataset cards expose the same level of licensing detail, and this repository's license metadata does not override any applicable dataset terms.
Merged standalone model:
text
emretmrk/Qwen3.5-4B-TR-FinanceBuilt with:
| Yes |
| Gradient accumulation | 16 |
| Scheduler | Cosine |
| Evaluation interval | 100 steps |
| Checkpoint interval | 250 steps |
| 2048 |
| Packing | Yes |
| Loss | Completion-only |
| Gradient accumulation | 16 |
| Scheduler | Cosine |
emretmrk/Qwen3.5-4B-TR-Financeimport torchfrom transformers import AutoProcessor, AutoModelForMultimodalLMfrom peft import PeftModel BASE_MODEL = "Qwen/Qwen3.5-4B"ADAPTER_ID = "emretmrk/Qwen3.5-4B-TR-Finance-LoRA" processor = AutoProcessor.from_pretrained(BASE_MODEL) base_model = AutoModelForMultimodalLM.from_pretrained( BASE_MODEL, torch_dtype="auto", device_map="auto",) model = PeftModel.from_pretrained( base_model, ADAPTER_ID, is_trainable=False,) model.eval()messages = [ { "role": "user", "content": [ { "type": "text", "text": "Merkez bankası politika faizini artırdığında ekonomide hangi kanallar etkilenebilir?" } ], }] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt",).to(model.device) with torch.no_grad(): outputs = model.generate( **inputs, max_new_tokens=256, do_sample=False, ) answer = processor.decode( outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True,) print(answer)model = PeftModel.from_pretrained( base_model, ADAPTER_ID, is_trainable=True,)merged_model = model.merge_and_unload()merged_model.save_pretrained( "./Qwen3.5-4B-TR-Finance", safe_serialization=True,)processor.save_pretrained("./Qwen3.5-4B-TR-Finance")