GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
ai-with-ramadhan
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
ai-with-ramadhan
Model Tree
Qwen/Qwen2.5-3B-Instruct
Input Modalities
Output Modalities
Supported Functionality
The reported loss is a training metric, not a measure of factual accuracy or proof that the model outperforms its base model.
InukaLM-LLM-Tutor is intended for:
The model was adapted using the following workflow:
| Parameter | Value |
|---|---|
LoRA rank (r) | 16 |
| LoRA alpha | 32 |
| LoRA dropout | 0.05 |
| Learning rate | 0.0002 |
| Epochs | 3 |
| Batch size per device | 1 |
| Gradient accumulation steps | 8 |
| Maximum sequence length | 512 tokens |
| Training precision | FP16 training with a 4-bit quantized base model |
The training dataset is small and was designed for a learning demonstration rather than broad domain coverage.
Install the required libraries:
bash
pip install -U transformers peft accelerate bitsandbytes torchLoad the base model and adapter:
python
import torchfrom transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfigfrom peft import PeftModel base_model_id = "Qwen/Qwen2.5-3B-Instruct"adapter_id = "ai-with-ramadhan/InukaLM-LLM-Tutor" quant_config = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.float16, bnb_4bit_use_double_quant=True,) tokenizer = AutoTokenizer.from_pretrained(adapter_id) base_model = AutoModelForCausalLM.from_pretrained( base_model_id, quantization_config=quant_config, device_map="auto",) model = PeftModel.from_pretrained(base_model, adapter_id)model.eval() messages = [ { "role": "system", "content": "You are an AI engineering tutor. Explain concepts clearly and accurately." }, { "role": "user", "content": "Explain self-attention in a transformer to a beginner." }] inputs = tokenizer.apply_chat_template( messages, tokenize=True, add_generation_prompt=True, return_tensors="pt", return_dict=True,).to(model.device) with torch.no_grad(): outputs = model.generate( **inputs, max_new_tokens=200, do_sample=False, ) new_tokens = outputs[0][inputs["input_ids"].shape[-1]:]print(tokenizer.decode(new_tokens, skip_special_tokens=True))Replace YOUR_HUGGINGFACE_USERNAME with the account that owns this repository. A compatible GPU environment is recommended for practical inference with this base model. Exact memory requirements depend on the runtime and configuration.
The model completed its initial fine-tuning run with a reported training loss of approximately 3.17.
This is an initial development result, not a comprehensive evaluation. No independent benchmark or controlled comparison has yet been established in this model card.
Future evaluation should compare the fine-tuned adapter with the original base model on held-out questions, checking factual correctness, clarity, relevance, consistency, and hallucination rates.
This repository contains a LoRA adapter based on Qwen2.5-3B-Instruct. Refer to the base model repository for its model card, license, and usage conditions.
The repository's license metadata should accurately reflect the applicable terms for the adapter, training materials, and any redistributed files.
Thanks to the Qwen team for releasing the base model and to the open-source communities behind Transformers, PEFT, Accelerate, and bitsandbytes.
Project goal: To make the mechanics of LLM fine-tuning easier to understand through a practical, reproducible learning experiment.
pip install -U transformers peft accelerate bitsandbytes torchimport torchfrom transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfigfrom peft import PeftModel base_model_id = "Qwen/Qwen2.5-3B-Instruct"adapter_id = "ai-with-ramadhan/InukaLM-LLM-Tutor" quant_config = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.float16, bnb_4bit_use_double_quant=True,) tokenizer = AutoTokenizer.from_pretrained(adapter_id) base_model = AutoModelForCausalLM.from_pretrained( base_model_id, quantization_config=quant_config, device_map="auto",) model = PeftModel.from_pretrained(base_model, adapter_id)model.eval() messages = [ { "role": "system", "content": "You are an AI engineering tutor. Explain concepts clearly and accurately." }, { "role": "user", "content": "Explain self-attention in a transformer to a beginner." }] inputs = tokenizer.apply_chat_template( messages, tokenize=True, add_generation_prompt=True, return_tensors="pt", return_dict=True,).to(model.device) with torch.no_grad(): outputs = model.generate( **inputs, max_new_tokens=200, do_sample=False, ) new_tokens = outputs[0][inputs["input_ids"].shape[-1]:]print(tokenizer.decode(new_tokens, skip_special_tokens=True))