GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
sarathi-balakrishnan
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
sarathi-balakrishnan
Model Tree
microsoft/Phi-3-mini-4k-instruct
Input Modalities
Output Modalities
Supported Functionality
| Base model | microsoft/Phi-3-mini-4k-instruct (3.8B) |
| Method | LoRA SFT (TRL SFTTrainer), BF16 |
| Data | 1,000 synthetic Q&A examples (900 train / 100 validation), 8 US counties |
| Hardware / time | NVIDIA DGX Spark (GB10), ~51 minutes |
| Adapter actually trained | o_proj + down_proj only, 35,651,584 parameters (≈0.93% of the base), 142.6 MB (FP32 safetensors) |
adapter_config.json lists
q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj. But Phi-3 uses fused
projections (qkv_proj and gate_up_proj), so PEFT matched only o_proj and down_proj.
The adapter weights confirm it: 64 tensors for each of those two modules and nothing else
(32 layers × 2 modules × LoRA A/B).
The correct targets for Phi-3 would have been:
python
target_modules = ["qkv_proj", "o_proj", "gate_up_proj", "down_proj"] # or "all-linear"model.print_trainable_parameters() and list the LoRA-wrapped module names before every run.| Metric | Base Phi-3-mini | Fine-tuned | Change |
|---|---|---|---|
| Content Coverage | 50.0% | 63.3% | +13.3 pp (+26.7% relative) |
| Structure Quality | 56.7% | 83.3% | +26.6 pp (+47.1% relative) |
| Avg end-to-end response time | 101.9 s | 62.4 s | −38.8% (see note 4 above) |
Training / eval loss by epoch (training loss start → end of epoch):
| Epoch | Training loss |
|---|---|
| 1 | 1.605 → 0.115 |
| 2 | 0.115 → 0.020 |
| 3 | 0.020 → 0.019 |
| Final eval loss | 0.0186 (token accuracy 99.3%) |
training_args.bin and adapter_config.json)| Parameter | Value |
|---|---|
| LoRA rank / alpha / dropout | 64 / 128 / 0.05 |
| Modules effectively trained | o_proj, down_proj (see above) |
| Epochs | 3 |
| Learning rate / schedule | 2e-4, linear, warmup ratio 0.1 |
| Per-device batch × grad accumulation | 16 × 2 = 32 effective |
| Optimizer | adamw_torch_fused, weight decay 0 |
| Max sequence length | 2048, no packing |
| Loss | computed on the full sequence (assistant_only_loss=False) |
| Precision | BF16, gradient checkpointing on |
| Eval / save | every epoch, best checkpoint loaded at end |
| Seed | 42 |
| Library versions | PEFT 0.18.0 |
Counties covered: Fresno CA, Kern CA, Lancaster PA, Sioux IA, Yakima WA, Imperial CA, Monterey CA, Deaf Smith TX.
python
from transformers import AutoTokenizer, AutoModelForCausalLMfrom peft import PeftModelimport torch model_name = "microsoft/Phi-3-mini-4k-instruct"tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)base_model = AutoModelForCausalLM.from_pretrained( model_name, torch_dtype=torch.bfloat16, device_map="auto", trust_remote_code=True, attn_implementation="eager",)model = PeftModel.from_pretrained(base_model, "sarathi-balakrishnan/phi3-agricultural-analyst") prompt = """You are an agricultural analyst. Analyze the following query using the provided data context. ### QueryWhat are the key investment opportunities in Fresno County agriculture? ### Data ContextCounty: Fresno, CAOperators: 5,847Average Farm Size: 423 acresIrrigation Coverage: 89%Revenue per Acre: $2,340 ### Analysis""" inputs = tokenizer(prompt, return_tensors="pt").to(model.device)outputs = model.generate(**inputs, max_new_tokens=500, temperature=0.7, do_sample=True)print(tokenizer.decode(outputs[0], skip_special_tokens=True))(The data-context numbers above are illustrative example inputs, not verified statistics.)
To merge for deployment: model = model.merge_and_unload().
The model was trained to produce: a regional summary, key insights, opportunities, risk factors, and a confidence level.
o_proj / down_proj were adapted, so this is a much weaker adapter than the config implies.Retrain with the correct fused-module targets, use loss on assistant tokens only, run a rank sweep (r = 8 / 16 / 64), and evaluate on real, human-written questions with a documented rubric.
MIT (same as the Phi-3-mini base model).
target_modules = ["qkv_proj", "o_proj", "gate_up_proj", "down_proj"] # or "all-linear"from transformers import AutoTokenizer, AutoModelForCausalLMfrom peft import PeftModelimport torch model_name = "microsoft/Phi-3-mini-4k-instruct"tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)base_model = AutoModelForCausalLM.from_pretrained( model_name, torch_dtype=torch.bfloat16, device_map="auto", trust_remote_code=True, attn_implementation="eager",)model = PeftModel.from_pretrained(base_model, "sarathi-balakrishnan/phi3-agricultural-analyst") prompt = """You are an agricultural analyst. Analyze the following query using the provided data context. ### QueryWhat are the key investment opportunities in Fresno County agriculture? ### Data ContextCounty: Fresno, CAOperators: 5,847Average Farm Size: 423 acresIrrigation Coverage: 89%Revenue per Acre: $2,340 ### Analysis""" inputs = tokenizer(prompt, return_tensors="pt").to(model.device)outputs = model.generate(**inputs, max_new_tokens=500, temperature=0.7, do_sample=True)print(tokenizer.decode(outputs[0], skip_special_tokens=True))