GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Shivenys
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
Shivenys
Model Tree
microsoft/Phi-3-mini-4k-instruct
Input Modalities
Output Modalities
Supported Functionality
microsoft/Phi-3-mini-4k-instruct| Parameter | Value |
|---|---|
| Base Model | Phi-3 Mini 4K Instruct |
| Fine-Tuning Method | QLoRA |
| LoRA Rank (r) | 8 |
| LoRA Alpha | 16 |
| LoRA Dropout | 0.05 |
| Task | Causal Language Modeling |
| Quantization | 4-bit NF4 |
| Compute Type | FP16 |
The adapter was fine-tuned on a small custom instruction-response dataset containing sample examples such as:
This dataset was intentionally kept small for educational purposes to demonstrate the end-to-end fine-tuning process.
Install the required libraries:
bash
pip install transformers peft bitsandbytes accelerate torchLoad the adapter:
python
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfigfrom peft import PeftModel BASE_MODEL = "microsoft/Phi-3-mini-4k-instruct" bnb_config = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.float16, bnb_4bit_use_double_quant=True,) tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL) base_model = AutoModelForCausalLM.from_pretrained( BASE_MODEL, quantization_config=bnb_config, device_map="auto",) model = PeftModel.from_pretrained( base_model, "YOUR_USERNAME/YOUR_REPOSITORY_NAME")Generate text:
python
prompt = """### Instruction:Who created Python? ### Response:""" inputs = tokenizer(prompt, return_tensors="pt").to(model.device) outputs = model.generate( **inputs, max_new_tokens=100,) print(tokenizer.decode(outputs[0], skip_special_tokens=True))text
adapter_config.jsonadapter_model.safetensorsREADME.mdThis repository contains only the LoRA adapter.
Please follow the license terms of the original Phi-3 Mini 4K Instruct model when using the base model.
pip install transformers peft bitsandbytes accelerate torchimport torchfrom transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfigfrom peft import PeftModel BASE_MODEL = "microsoft/Phi-3-mini-4k-instruct" bnb_config = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.float16, bnb_4bit_use_double_quant=True,) tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL) base_model = AutoModelForCausalLM.from_pretrained( BASE_MODEL, quantization_config=bnb_config, device_map="auto",) model = PeftModel.from_pretrained( base_model, "YOUR_USERNAME/YOUR_REPOSITORY_NAME")prompt = """### Instruction:Who created Python? ### Response:""" inputs = tokenizer(prompt, return_tensors="pt").to(model.device) outputs = model.generate( **inputs, max_new_tokens=100,) print(tokenizer.decode(outputs[0], skip_special_tokens=True))