GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
zaid646
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
zaid646
Model Tree
TinyLlama/TinyLlama-1.1B-Chat-v1.0
Input Modalities
Output Modalities
Supported Functionality
The adapter was trained using peft + bitsandbytes with the following configuration:
| Hyperparameter | Value |
|---|---|
| LoRA rank (r) | 16 |
| LoRA alpha | 32 |
| LoRA dropout | 0.1 |
| Target modules | q_proj, v_proj, k_proj, o_proj |
| Quantization | 4-bit NF4, double quant |
| Batch size | 2 (effective 16 with grad accum) |
| Learning rate | 1e-4 |
| Epochs | 2 |
| Max sequence length | 512 |
| Warmup steps | 50 |
| Optimizer | AdamW (paged) |
| Metric | Value |
|---|---|
| Final loss | 1.22 |
| Train samples/sec | 6.96 |
| Train steps/sec | 0.43 |
python
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfigfrom peft import PeftModel # Base modelbase_model_name = "TinyLlama/TinyLlama-1.1B-Chat-v1.0"adapter_name = "zaid646/tinyllama-1.1b-alpaca-qlora" # Load tokenizertokenizer = AutoTokenizer.from_pretrained(base_model_name)tokenizer.pad_token = tokenizer.eos_token # Load model with 4-bit quantizationquant_config = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_compute_dtype=torch.bfloat16, bnb_4bit_use_double_quant=True, bnb_4bit_quant_type="nf4",) model = AutoModelForCausalLM.from_pretrained( base_model_name, quantization_config=quant_config, device_map="auto", torch_dtype=torch.bfloat16,) # Load adaptermodel = PeftModel.from_pretrained(model, adapter_name) # Inferenceprompt = "### Instruction:\nExplain what machine learning is.\n### Response:\n"inputs = tokenizer(prompt, return_tensors="pt").to(model.device)outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.7)print(tokenizer.decode(outputs[0], skip_special_tokens=True))This model was trained using the fine-tuning-recipes framework. To reproduce:
bash
git clone https://github.com/ZAID646/fine-tuning-recipes.gitcd fine-tuning-recipespip install -e .python -m src.cli train --config recipes/qlora.yaml### Instruction:\n...\n### Response:\n)import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfigfrom peft import PeftModel # Base modelbase_model_name = "TinyLlama/TinyLlama-1.1B-Chat-v1.0"adapter_name = "zaid646/tinyllama-1.1b-alpaca-qlora" # Load tokenizertokenizer = AutoTokenizer.from_pretrained(base_model_name)tokenizer.pad_token = tokenizer.eos_token # Load model with 4-bit quantizationquant_config = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_compute_dtype=torch.bfloat16, bnb_4bit_use_double_quant=True, bnb_4bit_quant_type="nf4",) model = AutoModelForCausalLM.from_pretrained( base_model_name, quantization_config=quant_config, device_map="auto", torch_dtype=torch.bfloat16,) # Load adaptermodel = PeftModel.from_pretrained(model, adapter_name) # Inferenceprompt = "### Instruction:\nExplain what machine learning is.\n### Response:\n"inputs = tokenizer(prompt, return_tensors="pt").to(model.device)outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.7)print(tokenizer.decode(outputs[0], skip_special_tokens=True))git clone https://github.com/ZAID646/fine-tuning-recipes.gitcd fine-tuning-recipespip install -e .python -m src.cli train --config recipes/qlora.yaml