GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Aleton
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
Aleton
Model Tree
empero-ai/Qwen3.8-4B-Distill
Input Modalities
Output Modalities
Supported Functionality
| Параметр | Значение |
|---|---|
| Датасет | WiNE-iNEFF/1M-OpenOrca_be |
| Метод | QLoRA (NF4, double quant), BF16 compute |
| LoRA rank / alpha | 8 / 16 |
| Target modules | q_proj, k_proj, v_proj, o_proj |
| Обучаемых параметров | 1.57M (0.04%) |
| Эпохи | 2 |
| Sequence length | 512 (packing) |
| Learning rate | 2e-4, cosine |
| Финальный loss | 1.08 |
| Token accuracy | 71% |
python
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfigfrom peft import PeftModelmodel_id = "empero-ai/Qwen3.8-4B-Distill"adapter_id = "Aleton/Bel_qwen3.8-4B"# 4-bit квантование для экономии VRAMbnb_config = BitsAndBytesConfig(load_in_4bit=True,bnb_4bit_quant_type="nf4",bnb_4bit_use_double_quant=True,bnb_4bit_compute_dtype=torch.bfloat16,)base = AutoModelForCausalLM.from_pretrained(model_id,quantization_config=bnb_config,device_map={"": 0},)model = PeftModel.from_pretrained(base, adapter_id)tokenizer = AutoTokenizer.from_pretrained(adapter_id)messages = [{"role": "user", "content": "Прывітанне! Як справы?"}]inputs = tokenizer.apply_chat_template(messages,add_generation_prompt=True,return_tensors="pt",return_dict=True).to(model.device)out = model.generate(**inputs, max_new_tokens=200, do_sample=True, temperature=0.7)input_length = inputs.input_ids.shape[-1]response = tokenizer.decode(out[0][input_length:], skip_special_tokens=True)print(response)
Требования:
bash
pip install torch transformers peft bitsandbytes accelerate
python
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModelmodel_id = "empero-ai/Qwen3.8-4B-Distill"adapter_id = "Aleton/Bel_qwen3.8-4B"base = AutoModelForCausalLM.from_pretrained(model_id,dtype=torch.bfloat16,device_map={"": 0},)model = PeftModel.from_pretrained(base, adapter_id)tokenizer = AutoTokenizer.from_pretrained(adapter_id)messages = [{"role": "user", "content": "Прывітанне! Як справы?"}]inputs = tokenizer.apply_chat_template(messages,add_generation_prompt=True,return_tensors="pt",return_dict=True).to(model.device)out = model.generate(**inputs, max_new_tokens=200, do_sample=True, temperature=0.7)input_length = inputs.input_ids.shape[-1]response = tokenizer.decode(out[0][input_length:], skip_special_tokens=True)print(response)