GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
leandronunes
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
leandronunes
Model Tree
empero-ai/Qwen3.8-4B-Distill
Input Modalities
Output Modalities
Supported Functionality
| Parametro | Valor |
|---|---|
| Metodo | QLoRA (4-bit NF4) |
| Base | empero-ai/Qwen3.8-4B-Distill |
| Framework | Unsloth + TRL (SFTTrainer) |
| Dataset | Mingyin0312/Genome-Bench (2.671 amostras de treino) |
| LoRA rank (r) | 16 |
| LoRA alpha | 16 |
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Max seq length | 2048 |
| Batch efetivo | 8 (2 × 4 acumulacao) |
| Learning rate | 2e-4 |
| Steps | 300 (~1 epoch) |
| Otimizador | adamw_8bit |
| Precisao | float32 (Qwen3.5 nao suporta float16) |
| Hardware | Google Colab T4 (14.5 GB VRAM) |
O Genome-Bench e um benchmark de 3.332 questoes de multipla escolha sobre CRISPR e engenharia genomica, extraidas de discussoes reais de especialistas. O split de treino tem 2.671 amostras.
Formato de saida aprendido:
text
Explicacao: <rationale da escolha>Resposta final: <letra>python
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfigfrom peft import PeftModelimport torch bnb_config = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_compute_dtype=torch.bfloat16, bnb_4bit_quant_type="nf4", bnb_4bit_use_double_quant=True,) # 1. Carregar o modelo basebase = AutoModelForCausalLM.from_pretrained( "empero-ai/Qwen3.8-4B-Distill", quantization_config=bnb_config, device_map="auto", trust_remote_code=True,) # 2. Aplicar o adaptador LoRAmodel = PeftModel.from_pretrained(base, "leandronunes/qwen3.8-4b-genomebench-qlora")tokenizer = AutoTokenizer.from_pretrained("leandronunes/qwen3.8-4b-genomebench-qlora")python
messages = [{"role": "user", "content": "What is the most appropriate CRISPR delivery method for primary T cells?"}]inputs = tokenizer.apply_chat_template( messages, return_tensors="pt", add_generation_prompt=True).to(model.device) with torch.no_grad(): outputs = model.generate(**inputs, max_new_tokens=300, do_sample=False) print(tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True))| Metrica | Base (Qwen3.8-4B-Distill) | Adaptador |
|---|---|---|
| Deducao Logica (BBH) | 78% | a medir |
| Causal Judgement (BBH) | 46% | a medir |
| Acuracia Genome-Bench (test) | ? | a medir |
flash-linear-attention e causal-conv1d nao estavam instaladas, entao o treino usou a implementacao padrao do PyTorch.Genome-Bench e focado em CRISPR/genomica. O adaptador nao cobre outras areas da biologia.Apache 2.0 (herdada do Qwen3.5-4B e do Qwen3.8-4B-Distill).
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfigfrom peft import PeftModelimport torch bnb_config = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_compute_dtype=torch.bfloat16, bnb_4bit_quant_type="nf4", bnb_4bit_use_double_quant=True,) # 1. Carregar o modelo basebase = AutoModelForCausalLM.from_pretrained( "empero-ai/Qwen3.8-4B-Distill", quantization_config=bnb_config, device_map="auto", trust_remote_code=True,) # 2. Aplicar o adaptador LoRAmodel = PeftModel.from_pretrained(base, "leandronunes/qwen3.8-4b-genomebench-qlora")tokenizer = AutoTokenizer.from_pretrained("leandronunes/qwen3.8-4b-genomebench-qlora")messages = [{"role": "user", "content": "What is the most appropriate CRISPR delivery method for primary T cells?"}]inputs = tokenizer.apply_chat_template( messages, return_tensors="pt", add_generation_prompt=True).to(model.device) with torch.no_grad(): outputs = model.generate(**inputs, max_new_tokens=300, do_sample=False) print(tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True))