qwen14b-abap-sql-lora API & Inference Endpoint

Model Details

Table

Base model	Qwen/Qwen2.5-Coder-14B-Instruct
Method	QLoRA (NF4, r=16, α=32)
Target modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Training data	50,000 samples (12.9% ABAP, stratified subsample)
Training time	28.2 hours (RTX 4000 Ada)
Adapter size	~320 MB
Epochs	1
Learning rate	2e-4, cosine schedule

Note: 14B trained on 50k (not 100k) due to VRAM/time constraints (RTX 4000 Ada, 20.5 GB). This is a resource constraint, not a scientific choice. See PLANNING.md T6.1 for details.

Usage

python
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel

# Load base model in NF4
quant_cfg = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype="auto",
)

base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-Coder-14B-Instruct",
    quantization_config=quant_cfg,
    device_map="auto",
)
model = PeftModel.from_pretrained(base, "ChayannFamali/qwen14b-abap-sql-lora")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-Coder-14B-Instruct")

# Generate
messages = [
    {"role": "system", "content": "You are an expert ABAP programmer."},
    {"role": "user", "content": "Implement ABAP class for customer data handling"},
]
chatml = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(chatml, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=1024, do_sample=False)
print(tokenizer.decode(outputs[0]))

Usage with RAG

For best results, use with the Hybrid RAG pipeline (see GitHub repo for full instructions):

python
from src.rag.retriever import HybridRetriever

retriever = HybridRetriever(
    chroma_path="data/rag_index",
    collection_name="code_corpus",
    chunks_dir="data/rag_corpus/chunks",
    model_name="BAAI/bge-m3",
    device="cuda:0",
)

# Retrieve 3 ABAP examples
results = retriever.retrieve("Implement ABAP class for sorting", language="ABAP", k=3)

# Build few-shot system prompt
examples = "\n".join(f"Example {i+1}:\n```\n{r['code'][:800]}\n```\n"
                     for i, r in enumerate(results))
system = f"You are an expert ABAP programmer.\nHere are 3 relevant ABAP code examples:\n{examples}"

Performance (Test Split)

Table with columns: Language, Metric, Baseline 14B, FT 14B, FT 14B + RAG
Language	Metric	Baseline 14B	FT 14B	FT 14B + RAG
ABAP	chrf	0.319	0.450	0.531
ABAP	syntax_valid	1.000	0.983	0.917
ABAP	exact_match	0.000	0.039	0.028

ABAP chrf: +66% (0.319 → 0.531, baseline → FT+RAG) SQL exact_match: +3.8x (0.100 → 0.380, baseline → FT+RAG)

Python-Switching (Val Split)

Table with columns: Model, Switching rate
Model	Switching rate
Baseline 14B	0.0%
FT 14B	17.8%
FT 14B + RAG	0.0%

RAG completely eliminates switching (17.8% → 0.0%) without additional training.

Training Details

Config: configs/qlora_14b_final.yaml
Data: data/splits/train_v2_abap_boost_50k.jsonl (50k, resource constraint)
gradient_checkpointing: true (mandatory for 14B)
save_steps: 50 (~30 min between checkpoints, 14B-specific)
Merge: CPU merge required (14B bf16 = ~28 GB > 20.5 GB VRAM)
Full reproduction: See REPRODUCE.md

License

MIT

Base model

Qwen/Qwen2.5-Coder-14B-Instruct

Method

QLoRA (NF4, r=16, α=32)

Target modules

q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj

Training data

50,000 samples (12.9% ABAP, stratified subsample)

Training time

28.2 hours (RTX 4000 Ada)

Adapter size

~320 MB

Epochs

Learning rate

2e-4, cosine schedule

python

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel

# Load base model in NF4
quant_cfg = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype="auto",
)

base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-Coder-14B-Instruct",
    quantization_config=quant_cfg,
    device_map="auto",
)
model = PeftModel.from_pretrained(base, "ChayannFamali/qwen14b-abap-sql-lora")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-Coder-14B-Instruct")

# Generate
messages = [
    {"role": "system", "content": "You are an expert ABAP programmer."},
    {"role": "user", "content": "Implement ABAP class for customer data handling"},
]
chatml = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(chatml, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=1024, do_sample=False)
print(tokenizer.decode(outputs[0]))

python

from src.rag.retriever import HybridRetriever

retriever = HybridRetriever(
    chroma_path="data/rag_index",
    collection_name="code_corpus",
    chunks_dir="data/rag_corpus/chunks",
    model_name="BAAI/bge-m3",
    device="cuda:0",
)

# Retrieve 3 ABAP examples
results = retriever.retrieve("Implement ABAP class for sorting", language="ABAP", k=3)

# Build few-shot system prompt
examples = "\n".join(f"Example {i+1}:\n```\n{r['code'][:800]}\n```\n"
                     for i, r in enumerate(results))
system = f"You are an expert ABAP programmer.\nHere are 3 relevant ABAP code examples:\n{examples}"

Language

Metric

Baseline 14B

FT 14B

FT 14B + RAG

ABAP

chrf

0.319

0.450

0.531

ABAP

syntax_valid

1.000

0.983

0.917

ABAP

exact_match

0.000

0.039

0.028

Model

Switching rate

Baseline 14B

0.0%

FT 14B

17.8%

FT 14B + RAG

0.0%

qwen14b-abap-sql-lora

README

Model Details

Usage

Usage with RAG

Performance (Test Split)

Python-Switching (Val Split)

Training Details

Links

License

Explore FriendliAI today

README

Model Details

Usage

Usage with RAG

Performance (Test Split)

Python-Switching (Val Split)

Training Details

Links

License