ayushshah

Qwen3-1.7B-Chat

Deploy Dedicated

README

License: apache-2.0

Model Overview

Qwen3-1.7B-Chat has the following features:

Type: Causal Language Model
Training Stage: Pretraining → Supervised Fine-Tuning → Direct Preference Optimization
Number of Parameters: 1.7B
Number of Parameters (Non-Embedding): 1.4B
Number of Layers: 28
Number of Attention Heads (GQA): 16 for Q and 8 for KV
Context Length: 32,768

Training

Stage 1: Supervised Fine-Tuning

Method: Supervised Fine-Tuning (SFT)
Dataset: UltraChat-200k
Framework: Unsloth
Fine-Tuning: LoRA + embed_tokens + lm_head

Stage 2: Preference Alignment

Method: Direct Preference Optimization (DPO)
Frameworks: TRL + Unsloth
Fine-Tuning: LoRA
Precision: BF16
Environment: Windows Subsystem for Linux (WSL)
GPU: RTX 4090 24 GB using FlashAttention 2, xFormers and Triton

Dataset Preprocessing

The datasets were first filtered by removing rows with low chosen ratings. For argilla/distilabel-capybara-dpo-7k-binarized and argilla/ultrafeedback-binarized-preferences-cleaned, rows where the chosen rating was < 4 were removed. Similarly, for argilla/distilabel-intel-orca-dpo-pairs, rows where the preference was tie, the chosen rating was < 8, or in_gsm8k_train was true were removed.
For argilla/ultrafeedback-binarized-preferences-cleaned, only rows where the difference between the chosen and rejected ratings was >= 1 were retained.
Any row where either the chosen or rejected conversation exceeded 1024 tokens was removed.
The argilla/ultrafeedback-binarized-preferences-cleaned dataset was significantly larger than the other datasets; therefore, it was undersampled by randomly selecting rows.

DPO Configuration

Table with columns: Parameter, Value
Parameter	Value
Epochs	1
Batch Size	2
Gradient Accumulation	16
Effective Batch Size	32
Gradient Checkpointing	True
Optimizer	AdamW 8-bit
Learning Rate	5e-6
LoRA Rank	32
LoRA Alpha	64

LoRA was applied to: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj.

Evaluation

Evaluation was performed using EleutherAI's Language Model Evaluation Harness.

Table with columns: Benchmark, SFT, Chat, Δ
Benchmark	SFT	Chat	Δ
MMLU-Pro	37.91	39.86	+1.95
GSM8K	71.27	72.48	+1.21
HellaSwag	62.28	62.26	-0.02
TruthfulQA mc2	51.62	53.09	+1.47

Blind A/B Evaluation tests with LLM-as-a-judge and Human-In-The-Loop calibration preferred the DPO model over the SFT model in majority of the runs.

Usage

Unsloth (Recommended)

python
from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="ayushshah/Qwen3-1.7B-Chat",
    max_seq_length=2048,
    dtype="bfloat16",
)

FastLanguageModel.for_inference(model)

transformers

python
from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained(
    "ayushshah/Qwen3-1.7B-Chat"
)

model = AutoModelForCausalLM.from_pretrained(
    "ayushshah/Qwen3-1.7B-Chat",
    torch_dtype="auto",
    device_map="auto"
)

Limitations

Although preference optimization improves conversational quality, reasoning, and helpfulness, this model is still limited by the capabilities and knowledge of the underlying Qwen3-1.7B architecture. It may occasionally generate incorrect or hallucinated information, struggle with complex multi-step reasoning, or fail to follow ambiguous instructions accurately. Furthermore, this model has not undergone reinforcement learning from human feedback (RLHF) or additional safety-specific alignment beyond DPO and should not be relied upon for high-stakes applications without appropriate human oversight.

Compared to the SFT model, the primary observed trade-off was a slight reduction in strict instruction-following ability, accompanied by a tendency to generate more elaborate responses and a slightly higher likelihood of hallucinations on complex queries.

Citation

bibtex
@misc{qwen3technicalreport,
  title={Qwen3 Technical Report},
  author={Qwen Team},
  year={2025},
  eprint={2505.09388},
  archivePrefix={arXiv},
  primaryClass={cs.CL},
  url={https://arxiv.org/abs/2505.09388}
}

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

ayushshah

Model Tree

Base

ayushshah/Qwen3-1.7B-UltraChat-SFT

Fine-tuned

this model

Input Modalities

Text

Output Modalities