GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Famali
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
Famali
Model Tree
Qwen/Qwen2.5-Coder-7B-Instruct
Input Modalities
Output Modalities
Supported Functionality
| Base model | Qwen/Qwen2.5-Coder-7B-Instruct |
| Method | QLoRA (NF4, r=16, α=32) |
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Training data | 100,000 samples (12.9% ABAP, L7-style 2x boost) |
| Training time | 29.7 hours (RTX 4000 Ada) |
| Adapter size | ~161 MB |
| Epochs | 1 |
| Learning rate | 2e-4, cosine schedule |
python
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfigfrom peft import PeftModel # Load base model in NF4quant_cfg = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype="auto",) base = AutoModelForCausalLM.from_pretrained( "Qwen/Qwen2.5-Coder-7B-Instruct", quantization_config=quant_cfg, device_map="auto",)model = PeftModel.from_pretrained(base, "ChayannFamali/qwen7b-abap-sql-lora")tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-Coder-7B-Instruct") # Generatemessages = [ {"role": "system", "content": "You are an expert ABAP programmer."}, {"role": "user", "content": "Implement ABAP class for customer data handling"},]chatml = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)inputs = tokenizer(chatml, return_tensors="pt").to(model.device)outputs = model.generate(**inputs, max_new_tokens=1024, do_sample=False)print(tokenizer.decode(outputs[0]))For best results, use with the Hybrid RAG pipeline (see GitHub repo for full instructions):
python
from src.rag.retriever import HybridRetriever retriever = HybridRetriever( chroma_path="data/rag_index", collection_name="code_corpus", chunks_dir="data/rag_corpus/chunks", model_name="BAAI/bge-m3", device="cuda:0",) # Retrieve 3 ABAP examplesresults = retriever.retrieve("Implement ABAP class for sorting", language="ABAP", k=3) # Build few-shot system promptexamples = "\n".join(f"Example {i+1}:\n```\n{r['code'][:800]}\n```\n" for i, r in enumerate(results))system = f"You are an expert ABAP programmer.\nHere are 3 relevant ABAP code examples:\n{examples}"| Language | Metric | Baseline | FT 7B v2 | FT 7B v2 + RAG |
|---|---|---|---|---|
| ABAP | chrf | 0.325 | 0.418 | 0.498 |
| ABAP | syntax_valid | 0.994 | 0.956 | 0.978 |
| ABAP | exact_match | 0.000 | 0.017 | 0.028 |
| Model | Switching rate |
|---|---|
| Baseline 7B | 0.0% |
| FT 7B v2 | 10.0% |
| FT 7B v2 + RAG | 2.2% |
RAG reduces switching by 78% (10.0% → 2.2%) without additional training.
configs/qlora_7b_v2_abap_boost.yamldata/splits/train_v2_abap_boost.jsonl (100k, 12.9% ABAP, repeat 1.464x)MIT
| SQL |
| exact_match |
| 0.040 |
| 0.320 |
| 0.300 |
| SQL | chrf | 0.767 | 0.842 | 0.804 |
| Python | chrf | 0.376 | 0.418 | 0.389 |
| Java | chrf | 0.348 | 0.392 | 0.360 |