GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Bioaligned
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
Bioaligned
Model Tree
microsoft/phi-4
Input Modalities
Output Modalities
Supported Functionality
| Metric | Base Phi-4 | This adapter |
|---|---|---|
| Δpup | −0.1195 | −0.0020 |
| Improvement | — | +0.1175 |
| Parse rate | — | 100% (50/50) |
Δpup = mean difference in success probability assigned to biological vs. synthetic R&D approaches across 50 benchmark prompts. Higher (less negative) = more bioaligned.
| Parameter | Value |
|---|---|
| Base model | microsoft/phi-4 |
| Method | QLoRA (4-bit NF4, double quantization) |
| LoRA rank / alpha | 32 / 32 |
| LoRA dropout | 0.05 |
| Target modules | all-linear |
| Learning rate | 2e-4 (cosine decay) |
| Effective batch size | 16 (batch 2 × grad accum 8) |
| Epochs | 2 |
| Total optimizer steps | 770 |
| Warmup steps | 38 (5%) |
| Max grad norm |
All CPT (continues pretraining) examples were converted to Phi-4 instruction chat format to prevent format drift — the key fix vs. earlier Qwen3-14B training.
python
from transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModelimport torch base = AutoModelForCausalLM.from_pretrained( "microsoft/phi-4", torch_dtype=torch.bfloat16, device_map="auto")model = PeftModel.from_pretrained(base, "Bioaligned/Phi-4-instruct-bioaligned-qlora")model = model.merge_and_unload() tokenizer = AutoTokenizer.from_pretrained("microsoft/phi-4")For the ready-to-use merged model see Bioaligned/Phi-4-Instruct-Bioaligned.
| 0.3 |
| Sequence length | 2048 |
| Optimizer | PagedAdamW8bit |
| Compute dtype | bfloat16 |
| Training examples | 6160 (3984 CPT abstracts + 2176 instruction) |
| Validation examples | 664 |
| Best val loss | 1.5943 (step 700) |
| Hardware | NVIDIA A40 48GB |
from transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModelimport torch base = AutoModelForCausalLM.from_pretrained( "microsoft/phi-4", torch_dtype=torch.bfloat16, device_map="auto")model = PeftModel.from_pretrained(base, "Bioaligned/Phi-4-instruct-bioaligned-qlora")model = model.merge_and_unload() tokenizer = AutoTokenizer.from_pretrained("microsoft/phi-4")