GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Wothmag07
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
Wothmag07
Model Tree
meta-llama/Llama-3.1-8B-Instruct
Input Modalities
Output Modalities
Supported Functionality
Stage 1 — Supervised Fine-Tuning (SFT)
| Parameter | Value |
|---|---|
| Method | QLoRA (4-bit NF4 + double quantization) |
| LoRA Rank / Alpha | 64 / 128 |
| Learning Rate | 2e-4 (cosine scheduler) |
| Epochs | 2 |
| Effective Batch Size | 16 |
| Training Data | 36K multi-source counseling examples |
| GPU | NVIDIA H100 80GB |
| Training Time | ~3 hours |
Stage 2 — Direct Preference Optimization (DPO)
| Parameter | Value |
|---|---|
| Method | QLoRA on SFT-merged base |
| LoRA Rank / Alpha | 16 / 32 |
| Beta (KL penalty) | 0.5 |
| Learning Rate | 1e-5 (cosine scheduler) |
| Epochs | 1 |
| Effective Batch Size | 8 |
| Training Data | ~2K preference-filtered pairs |
| GPU | NVIDIA H100 80GB |
| Training Time | ~30 minutes |
SFT (36K examples from 5 sources)
| Source | Examples | Type |
|---|---|---|
| MentalChat16K | ~16K | Synthetic + clinical |
| empathetic_dialogues | ~10K | Real human multi-turn |
| Psych8k | ~8K | Real therapist transcripts |
| counsel-chat | ~940 | Real therapist Q&A |
| ESConv | ~910 | Real human + strategy labels |
DPO (~2K preference pairs)
| Source | Pairs | Selection |
|---|---|---|
| PsychoCounsel-Preference | ~2K | Rating-gap filtered across 7 dimensions |
| Metric | Base | SFT | DPO |
|---|---|---|---|
| Perplexity | 4.18 | 3.64 | 3.13 |
| BERTScore F1 | 0.8598 | 0.8527 | 0.8492 |
| ROUGE-L F1 | 0.1065 | 0.0772 | 0.0790 |
| Distinct-1 | 0.273 | 0.331 | 0.262 |
| Distinct-2 | 0.658 | 0.807 | 0.712 |
| Avg Response Length | 98 | 119 | 198 |
| Dimension | Base | SFT | DPO |
|---|---|---|---|
| Empathy | 4.40 | 3.48 | 4.88 |
| Safety | 4.28 | 3.84 | 4.60 |
| Relevance | 4.68 | 3.72 | 4.88 |
| Helpfulness | 4.04 | 3.04 | 4.48 |
| Overall | 4.35 | 3.52 | 4.71 |
Evaluated on 25 curated prompts across 18 mental health categories (anxiety, depression, grief, crisis, relationships, trauma, etc.).
This repo contains three artifacts:
| Path | Format | Size | Description |
|---|---|---|---|
/ (root) | Full merged model | ~16 GB | Ready-to-use Llama 3.1 8B + SFT + DPO merged |
sft/ | LoRA adapter | ~640 MB | Stage-1 SFT adapter (r=64, α=128) — load on top of base Llama 3.1 8B |
dpo/ | LoRA adapter | ~160 MB | Stage-2 DPO adapter (r=16, α=32) — load on top of SFT-merged base |
Load an adapter with PEFT:
python
from peft import PeftModelfrom transformers import AutoModelForCausalLM base = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.1-8B-Instruct", torch_dtype="bfloat16", device_map="auto")model = PeftModel.from_pretrained(base, "Wothmag07/counseLLM", subfolder="dpo")python
from transformers import AutoModelForCausalLM, AutoTokenizerimport torch model_id = "Wothmag07/counseLLM" tokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained( model_id, torch_dtype=torch.bfloat16, device_map="auto",) messages = [ {"role": "system", "content": "You are a mental health counselor providing supportive, empathetic guidance. Respond by first acknowledging the person's feelings, then explore their situation with open-ended questions. Use techniques like reflective listening, validation, and gentle reframing. Keep responses warm, conversational, and non-judgmental."}, {"role": "user", "content": "I've been feeling really anxious about work lately and I can't sleep."},] input_ids = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)outputs = model.generate( input_ids, max_new_tokens=512, temperature=0.7, top_p=0.9, repetition_penalty=1.1,)response = tokenizer.decode(outputs[0][input_ids.shape[1]:], skip_special_tokens=True)print(response)| Component | Technology |
|---|---|
| Base Model | Meta Llama 3.1 8B Instruct |
| Training | HuggingFace TRL (SFTTrainer, DPOTrainer) |
| Quantization | QLoRA via bitsandbytes (4-bit NF4) |
| Adapters | PEFT (LoRA) |
| Infrastructure | Modal (H100 GPUs) |
| Experiment Tracking | Weights & Biases |
| Evaluation | BERTScore, ROUGE-L, GPT-4o Judge |
bibtex
@misc{counseLLM2026, author = {Gowtham Arulmozhii}, title = {CounseLLM: Empathy-Aligned Conversational Support LLM}, year = {2026}, publisher = {HuggingFace}, url = {https://huggingface.co/Wothmag07/counseLLM}}from peft import PeftModelfrom transformers import AutoModelForCausalLM base = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.1-8B-Instruct", torch_dtype="bfloat16", device_map="auto")model = PeftModel.from_pretrained(base, "Wothmag07/counseLLM", subfolder="dpo")from transformers import AutoModelForCausalLM, AutoTokenizerimport torch model_id = "Wothmag07/counseLLM" tokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained( model_id, torch_dtype=torch.bfloat16, device_map="auto",) messages = [ {"role": "system", "content": "You are a mental health counselor providing supportive, empathetic guidance. Respond by first acknowledging the person's feelings, then explore their situation with open-ended questions. Use techniques like reflective listening, validation, and gentle reframing. Keep responses warm, conversational, and non-judgmental."}, {"role": "user", "content": "I've been feeling really anxious about work lately and I can't sleep."},] input_ids = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)outputs = model.generate( input_ids, max_new_tokens=512, temperature=0.7, top_p=0.9, repetition_penalty=1.1,)response = tokenizer.decode(outputs[0][input_ids.shape[1]:], skip_special_tokens=True)print(response)@misc{counseLLM2026, author = {Gowtham Arulmozhii}, title = {CounseLLM: Empathy-Aligned Conversational Support LLM}, year = {2026}, publisher = {HuggingFace}, url = {https://huggingface.co/Wothmag07/counseLLM}}