GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
srimworksa
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
srimworksa
Model Tree
Qwen/Qwen3-14B
Input Modalities
Output Modalities
Supported Functionality
adapter_model.safetensors (256.98 MB) — COMPLETE & PRODUCTION-READY ADAPTER:
This file contains all 560 LoRA A/B tensors across all 40 layers (layers.0 through layers.39) for all 7 target projections (q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj = 280 LoRA modules, Rank r=32, α=64). It binds cleanly to Qwen/Qwen3-14B (or Qwen/Qwen3-14B-AWQ).
ven_hermes_awq_4bit_group128.safetensors (681.25 MB) — INCOMPLETE DIAGNOSTIC SHARD (NOT A STANDALONE MODEL):
This file contains only 56 quantized tensors covering layers 0..3 only (4 out of 40 layers). It lacks layers 4..39, model.embed_tokens.weight, lm_head.weight, and RMSNorm weights. It is an incomplete 4-layer diagnostic artifact and CANNOT be loaded as a standalone AWQ model. A full standalone 14B 4-bit model requires ≥7.4 GB of payload. For production serving, load the base model Qwen/Qwen3-14B and attach adapter_model.safetensors.
Reconciliation of Historical Metrics:
An earlier automated run on October 22 temporarily uploaded a smaller fallback artifact (196 modules / 34.9M claimed params). That discrepancy has been resolved. The current repository contains the verified 280-module, 40-layer Qwen/Qwen3-14B adapter with 128,450,560 trainable parameters.
transformers + peft:python
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModel base_model_id = "Qwen/Qwen3-14B"adapter_id = "kireetialugunulla/VEN-Hermes-14B-AWQ" tokenizer = AutoTokenizer.from_pretrained(base_model_id, trust_remote_code=True)base_model = AutoModelForCausalLM.from_pretrained( base_model_id, torch_dtype=torch.float16, device_map="auto", trust_remote_code=True,) # Attach the VEN-Hermes 14B PEFT adaptermodel = PeftModel.from_pretrained(base_model, adapter_id)model.eval() # Example prompt with Hermes /think directiveprompt = "<|im_start|>system\nYou are an elite B2B SDR.<|im_end|>\n<|im_start|>user\n/think\nExplain your pgvector RAG capabilities.<|im_end|>\n<|im_start|>assistant\n"inputs = tokenizer(prompt, return_tensors="pt").to(model.device)outputs = model.generate(**inputs, max_new_tokens=150, temperature=0.2)print(tokenizer.decode(outputs[0], skip_special_tokens=True))vLLM:bash
vllm serve Qwen/Qwen3-14B \ --enable-lora \ --lora-modules ven-hermes=kireetialugunulla/VEN-Hermes-14B-AWQ \ --dtype float16 \ --tensor-parallel-size 2 \ --max-model-len 2048 \ --port 8000Telemetry Source: training_and_quantization_metrics.json (Run Marker: qwen3-14b-unified-1793180715)
| Parameter / Metric | Measured Value |
|---|---|
| Base Model Architecture | Qwen/Qwen3-14B (40 layers, D=5120, intermediate =17408) |
| Execution Hardware | 2x NVIDIA Tesla T4 (sm_75, 32 GB VRAM combined, CUDA 12.8) |
| Base Model Parameters | 14,768,307,200 (14.77B) |
| LoRA Adapter Rank & Alpha | r=32,α=64, Dropout =0.05 |
| Target Projection Modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj (280 modules across 40 layers) |
| Total LoRA Tensors in Adapter | 560 tensors (base_model.model.model.layers.0..39.{proj}.lora_{A,B}.weight) |
| Trainable LoRA Parameters | 128,450,560 (0.8623% of 14,896,757,760 total) |
| SFT Training Dataset | 925 rows (849 Email, 36 Voice, 40 WhatsApp in data/ven_unified_3channel_sft_train.jsonl) |
| SFT Training Loss | 2.2861 (Initial) ⟶ 0.0003 (Final batch loss, 60 steps in 283.15s) |
| DPO Alignment Dataset | 574 contrastive pairs (543 Email, 11 Voice, 20 WhatsApp, Bradley-Terry β=0.1) |
| In-Sample DPO Accuracy | 91.67% (22/24 training steps with positive chosen > rejected margin in 201.68s) |
| Peak GPU Memory | 12.358 GB peak single-GPU VRAM (24.373 GB combined across both Tesla T4 GPUs) |
Note on Generalization: The reported SFT loss (
0.0003) and DPO accuracy (91.67%) are in-sample training-loop measurements reflecting template memorization during training. They do not constitute independent proof of generalization on unseen out-of-distribution dialogues. An independent held-out benchmark suite (56 unseen scenarios) is maintained separately indata/ven_unseen_benchmark_56.json.
/think Mode): Deep ICP buyer qualification, multi-page company research, anti-slop cold outreach generation (75–150 words), reviewer revision/QC, and autonomous reply drafting./no_think Mode): Sub-second conversational turn generation (8–28 words, 1–2 spoken sentences), SDR (Maya) → Solutions Architect (Rajan) warm handoff, truthful AI disclosure (DISCLOSE_AI), and fail-closed E.164 phone DNC compliance (RESPECTFUL_DNC)./no_think & /think Mode): Fast conversational replies and calendar booking via /no_think, switching dynamically to /think for complex technical architecture questions (e.g. pgvector row-level security, VPC isolation).adapter_model.safetensors: Full Qwen/Qwen3-14B PEFT LoRA adapter (256.98 MB, 560 tensors across 40 layers, 128,450,560 params).adapter_config.json: PEFT configuration for Qwen/Qwen3-14B (r=32,α=64).config.json & generation_config.json: Official Qwen/Qwen3-14B base model configuration.training_and_quantization_metrics.json: Hardware-measured training and quantization telemetry.ven_hermes_awq_4bit_group128.safetensors: Incomplete 4-layer diagnostic shard (layers.0..3 only, 681.25 MB). Not suitable for standalone inference.csrc/: Custom C++/CUDA kernel sources (ven_cuda_kernels.cu, bindings.cpp), compiled binary (ven_cuda_ops.so), and benchmark report (cuda_benchmark_results.json).data/: Unified 3-channel datasets (ven_unified_3channel_sft_train.jsonl, ven_unified_3channel_dpo_pairs.jsonl, ven_unseen_benchmark_56.json).import torchfrom transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModel base_model_id = "Qwen/Qwen3-14B"adapter_id = "kireetialugunulla/VEN-Hermes-14B-AWQ" tokenizer = AutoTokenizer.from_pretrained(base_model_id, trust_remote_code=True)base_model = AutoModelForCausalLM.from_pretrained( base_model_id, torch_dtype=torch.float16, device_map="auto", trust_remote_code=True,) # Attach the VEN-Hermes 14B PEFT adaptermodel = PeftModel.from_pretrained(base_model, adapter_id)model.eval() # Example prompt with Hermes /think directiveprompt = "<|im_start|>system\nYou are an elite B2B SDR.<|im_end|>\n<|im_start|>user\n/think\nExplain your pgvector RAG capabilities.<|im_end|>\n<|im_start|>assistant\n"inputs = tokenizer(prompt, return_tensors="pt").to(model.device)outputs = model.generate(**inputs, max_new_tokens=150, temperature=0.2)print(tokenizer.decode(outputs[0], skip_special_tokens=True))vllm serve Qwen/Qwen3-14B \ --enable-lora \ --lora-modules ven-hermes=kireetialugunulla/VEN-Hermes-14B-AWQ \ --dtype float16 \ --tensor-parallel-size 2 \ --max-model-len 2048 \ --port 8000