Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Get help setting up a custom Dedicated Endpoints.
Talk with our engineer to get a quote for reserved GPU instances with discounts.
README
License: mitBioalignment results
| Metric | Base Phi-4 | This adapter |
|---|---|---|
| Δpup | −0.1195 | −0.0020 |
| Improvement | — | +0.1175 |
| Parse rate | — | 100% (50/50) |
Δpup = mean difference in success probability assigned to biological vs. synthetic R&D approaches across 50 benchmark prompts. Higher (less negative) = more bioaligned.
Training details
| Parameter | Value |
|---|---|
| Base model | microsoft/phi-4 |
| Method | QLoRA (4-bit NF4, double quantization) |
| LoRA rank / alpha | 32 / 32 |
| LoRA dropout | 0.05 |
| Target modules | all-linear |
| Learning rate | 2e-4 (cosine decay) |
| Effective batch size | 16 (batch 2 × grad accum 8) |
| Epochs | 2 |
| Total optimizer steps | 770 |
| Warmup steps | 38 (5%) |
| Max grad norm | 0.3 |
| Sequence length | 2048 |
| Optimizer | PagedAdamW8bit |
| Compute dtype | bfloat16 |
| Training examples | 6160 (3984 CPT abstracts + 2176 instruction) |
| Validation examples | 664 |
| Best val loss | 1.5943 (step 700) |
| Hardware | NVIDIA A40 48GB |
All CPT (continues pretraining) examples were converted to Phi-4 instruction chat format to prevent format drift — the key fix vs. earlier Qwen3-14B training.
Usage
python
from transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModelimport torchbase = AutoModelForCausalLM.from_pretrained("microsoft/phi-4", torch_dtype=torch.bfloat16, device_map="auto")model = PeftModel.from_pretrained(base, "Bioaligned/Phi-4-instruct-bioaligned-qlora")model = model.merge_and_unload()tokenizer = AutoTokenizer.from_pretrained("microsoft/phi-4")
For the ready-to-use merged model see Bioaligned/Phi-4-Instruct-Bioaligned.
Model provider
Bioaligned
Model tree
Base
microsoft/phi-4
Adapter
this model
Modalities
Input
Text
Output
Text
Pricing
Dedicated Endpoints
View detailsSupported Functionality
Model APIs
Dedicated Endpoints
Container
More information