anilkaracay

qwen3-4b-legal-tr-qlora

Deploy Dedicated

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Get help setting up a custom Dedicated Endpoints.

Talk with our engineer to get a quote for reserved GPU instances with discounts.

Training

Base model: Qwen/Qwen3-4B-Instruct-2507
Method: QLoRA (4-bit NF4 base + FP16 LoRA adapters)
LoRA config: r=16, alpha=16, all attention + MLP projections (~33M trainable params)
Data: 4,900 supervised QA examples (Turkish legal)
Hardware: single NVIDIA Tesla T4 (15.6 GB)

Usage

python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = "Qwen/Qwen3-4B-Instruct-2507"
adapter = "anilkaracay/qwen3-4b-legal-tr-qlora"

tokenizer = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base, torch_dtype=torch.float16, device_map="auto")
model = PeftModel.from_pretrained(model, adapter)
model.eval()

Note

This model emits empty <think></think> blocks (a Qwen3-Instruct chat-template artifact). Suppress token IDs 151657 and 151658 via bad_words_ids during generation, or strip the blocks in post-processing.

Model provider

anilkaracay

Model tree

Base

Qwen/Qwen3-4B-Instruct-2507

Adapter

this model

Modalities

Input

Text

Output

Text