GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
FardinJP
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
FardinJP
Model Tree
Qwen/Qwen2.5-7B-Instruct
Input Modalities
Output Modalities
Supported Functionality
Give the model a task profile (task description, data modality, target hardware,
memory budget) and it returns one ## Hyperparameters block with a dimension,
encoder, quantization, and number of retraining passes. In the accompanying
application, this recommendation seeds a memory-constrained search that
measures every candidate on the user's data.
The recommendation is a starting point. Always verify it, which the application does automatically.
python
from peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-7B-Instruct", torch_dtype="auto", device_map="auto")model = PeftModel.from_pretrained(base, "FardinJP/rag-to-lora-qwen2.5-7b")tok = AutoTokenizer.from_pretrained("FardinJP/rag-to-lora-qwen2.5-7b")For the exact prompt format, use the rag-to-lora repository
(src/recommender/prompts.py), or simply run its Streamlit app.
| Setting | Value |
|---|---|
| Base model | Qwen/Qwen2.5-7B-Instruct (bfloat16) |
| LoRA rank / alpha / dropout | 32 / 64 / 0.05 |
| Target modules | q, k, v, o, gate, up, down projections |
| Trainable parameters | 80.7 M (about 1.1%) |
| Training pairs | 1,519 (168 validation) |
| Epochs, learning rate | 4, 2e-4 |
| Loss | completion tokens only |
Released under Apache 2.0, the license of the base model.