GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
alfeenafsal
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
alfeenafsal
Model Tree
microsoft/phi-2
Input Modalities
Output Modalities
Supported Functionality
The adapter was evaluated on a leak-free, out-of-distribution 50-task algorithmic benchmark spanning eight programming categories:
| Model / Configuration | Training Scope | Target Modules | Resp Loss | Core Pass% (Greedy) | EOS Term% | Max-Token Cutoff% | Mean Length (tok) |
|---|---|---|---|---|---|---|---|
| Raw Phi-2 Base | Pretrained | None | 0.5792 | 52.0% (26/50) | 18.0% | 84.0% | 284.6 |
| Config E Step 100 (Selected Adapter) | 5,000 samples | q_proj, v_proj | 0.4803 | 68.0% (34/50) | 96.0% | 4.0% | 115.4 |
microsoft/phi-2 (2.78B parameters)q_proj, v_projPhase2CompletionCollator) focusing gradients solely on completion tokensbash
pip install torch transformers peft bitsandbytes acceleratepython
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfigfrom peft import PeftModel # Configure 4-bit NF4 quantizationbnb_config = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_use_double_quant=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.bfloat16,) # Load frozen base modelbase_model = AutoModelForCausalLM.from_pretrained( "microsoft/phi-2", quantization_config=bnb_config, device_map="auto", trust_remote_code=True,) # Attach LoRA adaptermodel = PeftModel.from_pretrained(base_model, "alfeenafsal/phi2-qlora-finetune")model.eval() tokenizer = AutoTokenizer.from_pretrained("alfeenafsal/phi2-qlora-finetune", trust_remote_code=True)tokenizer.pad_token = tokenizer.eos_token # Format promptprompt = "### Instruction:\nWrite a Python function that checks if two strings are anagrams of each other (case-insensitive).\n\n### Response:\n"inputs = tokenizer(prompt, return_tensors="pt").to(model.device) with torch.no_grad(): outputs = model.generate( **inputs, max_new_tokens=200, do_sample=False, # Deterministic greedy decoding eos_token_id=tokenizer.eos_token_id, pad_token_id=tokenizer.pad_token_id, ) completion = tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)print(completion)Because LoRA modifies intermediate projections linearly without copying weights, you can easily compare the fine-tuned adapter against the raw base model on the same in-memory instance:
python
# Raw base model generationwith model.disable_adapter(): with torch.no_grad(): base_out = model.generate(**inputs, max_new_tokens=200, do_sample=False) print("Base:", tokenizer.decode(base_out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)) # Fine-tuned adapted generationwith torch.no_grad(): adapted_out = model.generate(**inputs, max_new_tokens=200, do_sample=False)print("Adapted:", tokenizer.decode(adapted_out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))iamtarun/python_code_instructions_18k_alpaca (4,500 train / 500 validation, seed 42).text
### Instruction:{instruction}### Input:{input}### Response:{output}<|endoftext|>reports/phase3_benchmark_ood_verification.json).adamw_torch2e-40.03