iskiai

iski-lora

README

License: apache-2.0

Model Information

Table with columns: Field, Value
Field	Value
Model type	PEFT LoRA Adapter
Base model	`unsloth/Qwen3.6-35B-A3B`
Framework	Transformers + PEFT (Unsloth)
Precision	bfloat16
LoRA rank	r=32, alpha=64
LoRA target modules	q_proj, k_proj, v_proj, o_proj
LoRA dropout	0
Training samples	~7,100 İSKİ Q&A pairs
Language	Turkish
Domain	Municipal water & sewerage administration

Training Configuration

Table with columns: Parameter, Value
Parameter	Value
Epochs	3
Learning rate	5e-5
LR scheduler	cosine
Optimizer	paged_adamw_8bit
Batch size (per device)	8
Gradient accumulation	4 (effective batch: 32)
Weight decay	0.01
Warmup ratio	0.03
Max sequence length	4096

Data Pipeline

Dataset is split using GroupShuffleSplit (group key: source_file) with a 95/5 train/eval ratio to prevent data leakage across documents from the same source.

Each training sample is formatted as a three-turn conversation using the Qwen3 chat template:

text

System    → domain context (category, source_file, source_year)User      → instruction [+ optional input]Assistant → expected output

Loss is computed only on assistant responses (train_on_responses_only).

Usage

Load the Adapter

python

from transformers import AutoTokenizer, AutoModelForCausalLMfrom peft import PeftModelimport torch base_model_id = "unsloth/Qwen3.6-35B-A3B"adapter_id = "iskiai/iski-lora" tokenizer = AutoTokenizer.from_pretrained(base_model_id)base_model = AutoModelForCausalLM.from_pretrained(    base_model_id,    torch_dtype=torch.bfloat16,    device_map="auto") model = PeftModel.from_pretrained(base_model, adapter_id)model.eval()

Inference

python

SYSTEM_BASE = "Sen kurum ici dokumanlar konusunda uzman, yardimci bir asistansin." def build_system_message(category="", source_file="", source_year=""):    return (f"{SYSTEM_BASE} Baglam Bilgisi - Kategori: {category}, "            f"Kaynak: {source_file} ({source_year}).") messages = [    {"role": "system", "content": build_system_message()},    {"role": "user",   "content": "İSKİ su aboneliği nasıl açılır?"},] prompt = tokenizer.apply_chat_template(    messages,    tokenize=False,    add_generation_prompt=True,    # Qwen3: thinking mode kapatılmalı    chat_template_kwargs={"enable_thinking": False}) inputs = tokenizer(prompt, return_tensors="pt").to(model.device) with torch.no_grad():    outputs = model.generate(        **inputs,        max_new_tokens=512,        do_sample=False,        use_cache=True,    ) response = tokenizer.decode(    outputs[0][inputs["input_ids"].shape[1]:],    skip_special_tokens=True)print(response)

⚠️ Thinking Mode: Qwen3 modelleri varsayılan olarak chain-of-thought düşünme modunu etkinleştirir. Üretim ortamında enable_thinking: false set edilmezse yanıtlar <think>...</think> bloğu içerebilir.

Serving with vLLM

bash

vllm serve unsloth/Qwen3.6-35B-A3B \  --enable-lora \  --lora-modules iski-lora=iskiai/iski-lora \  --max-lora-rank 32 \  --max-model-len 8192 \  --port 5001

python

from openai import OpenAI client = OpenAI(base_url="http://localhost:5001/v1", api_key="dummy") response = client.chat.completions.create(    model="iski-lora",    messages=[        {"role": "system", "content": build_system_message()},        {"role": "user",   "content": "Su sayacı arızası nasıl bildirilir?"},    ],    extra_body={"chat_template_kwargs": {"enable_thinking": False}})print(response.choices[0].message.content)

Files

text

iski-lora/├── adapter_model.safetensors   # LoRA weights├── adapter_config.json         # PEFT configuration├── tokenizer_config.json└── README.md

Training Outputs

The training script produces the following artifacts alongside the adapter:

Table with columns: Artifact, Description
Artifact	Description
`egitim_loglari.jsonl`	Step-by-step training logs (loss, eval_loss, lr, epoch) in JSON Lines format
`benchmark_sonuclari.json`	Final metrics: perplexity, VRAM usage, sample generations
`grafikler/`	Per-metric PNG plots + combined summary dashboard
`runs/`	TensorBoard logs (local, no external connection required)

View training curves locally:

bash

tensorboard --logdir runs

Evaluation

Final model quality is assessed by:

Perplexity on the held-out eval split (computed from eval_loss)
Sample generations — 5 randomly drawn eval examples are generated greedily and saved to benchmark_sonuclari.json for manual inspection
DeepEval GEval — downstream evaluation using a vLLM-backed judge model

License

This repository contains a LoRA adapter trained on the Qwen3.6-35B-A3B base model. Users must obtain and use the base model in accordance with its original license. This adapter is distributed under the Apache 2.0 License.

Author

Created by İSKİ AI Team · iskiai

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

iskiai

Model Tree

Base

unsloth/Qwen3.6-35B-A3B

Adapter

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

Model Information

Table with columns: Field, Value
Field	Value
Model type	PEFT LoRA Adapter
Base model	`unsloth/Qwen3.6-35B-A3B`
Framework	Transformers + PEFT (Unsloth)
Precision	bfloat16
LoRA rank	r=32, alpha=64
LoRA target modules	q_proj, k_proj, v_proj, o_proj
LoRA dropout	0
Training samples	~7,100 İSKİ Q&A pairs
Language	Turkish
Domain	Municipal water & sewerage administration

Training Configuration

Table with columns: Parameter, Value
Parameter	Value
Epochs	3
Learning rate	5e-5
LR scheduler	cosine
Optimizer	paged_adamw_8bit
Batch size (per device)	8
Gradient accumulation	4 (effective batch: 32)
Weight decay	0.01
Warmup ratio	0.03
Max sequence length	4096

Data Pipeline

Dataset is split using GroupShuffleSplit (group key: source_file) with a 95/5 train/eval ratio to prevent data leakage across documents from the same source.

Each training sample is formatted as a three-turn conversation using the Qwen3 chat template:

text

System    → domain context (category, source_file, source_year)User      → instruction [+ optional input]Assistant → expected output

Loss is computed only on assistant responses (train_on_responses_only).

Usage

Load the Adapter

python

from transformers import AutoTokenizer, AutoModelForCausalLMfrom peft import PeftModelimport torch base_model_id = "unsloth/Qwen3.6-35B-A3B"adapter_id = "iskiai/iski-lora" tokenizer = AutoTokenizer.from_pretrained(base_model_id)base_model = AutoModelForCausalLM.from_pretrained(    base_model_id,    torch_dtype=torch.bfloat16,    device_map="auto") model = PeftModel.from_pretrained(base_model, adapter_id)model.eval()

Inference

python

SYSTEM_BASE = "Sen kurum ici dokumanlar konusunda uzman, yardimci bir asistansin." def build_system_message(category="", source_file="", source_year=""):    return (f"{SYSTEM_BASE} Baglam Bilgisi - Kategori: {category}, "            f"Kaynak: {source_file} ({source_year}).") messages = [    {"role": "system", "content": build_system_message()},    {"role": "user",   "content": "İSKİ su aboneliği nasıl açılır?"},] prompt = tokenizer.apply_chat_template(    messages,    tokenize=False,    add_generation_prompt=True,    # Qwen3: thinking mode kapatılmalı    chat_template_kwargs={"enable_thinking": False}) inputs = tokenizer(prompt, return_tensors="pt").to(model.device) with torch.no_grad():    outputs = model.generate(        **inputs,        max_new_tokens=512,        do_sample=False,        use_cache=True,    ) response = tokenizer.decode(    outputs[0][inputs["input_ids"].shape[1]:],    skip_special_tokens=True)print(response)

⚠️ Thinking Mode: Qwen3 modelleri varsayılan olarak chain-of-thought düşünme modunu etkinleştirir. Üretim ortamında enable_thinking: false set edilmezse yanıtlar <think>...</think> bloğu içerebilir.

Serving with vLLM

bash

vllm serve unsloth/Qwen3.6-35B-A3B \  --enable-lora \  --lora-modules iski-lora=iskiai/iski-lora \  --max-lora-rank 32 \  --max-model-len 8192 \  --port 5001

python

from openai import OpenAI client = OpenAI(base_url="http://localhost:5001/v1", api_key="dummy") response = client.chat.completions.create(    model="iski-lora",    messages=[        {"role": "system", "content": build_system_message()},        {"role": "user",   "content": "Su sayacı arızası nasıl bildirilir?"},    ],    extra_body={"chat_template_kwargs": {"enable_thinking": False}})print(response.choices[0].message.content)

Files

text

iski-lora/├── adapter_model.safetensors   # LoRA weights├── adapter_config.json         # PEFT configuration├── tokenizer_config.json└── README.md

Training Outputs

The training script produces the following artifacts alongside the adapter:

Table with columns: Artifact, Description
Artifact	Description
`egitim_loglari.jsonl`	Step-by-step training logs (loss, eval_loss, lr, epoch) in JSON Lines format
`benchmark_sonuclari.json`	Final metrics: perplexity, VRAM usage, sample generations
`grafikler/`	Per-metric PNG plots + combined summary dashboard
`runs/`	TensorBoard logs (local, no external connection required)

View training curves locally:

bash

tensorboard --logdir runs

Evaluation

Final model quality is assessed by:

Perplexity on the held-out eval split (computed from eval_loss)
Sample generations — 5 randomly drawn eval examples are generated greedily and saved to benchmark_sonuclari.json for manual inspection
DeepEval GEval — downstream evaluation using a vLLM-backed judge model

License

Author

Created by İSKİ AI Team · iskiai