GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Nikhil1581
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
Nikhil1581
Model Tree
HuggingFaceTB/SmolLM2-1.7B-Instruct
Input Modalities
Output Modalities
Supported Functionality
| Field | Details |
|---|---|
| Model name | DAVID-fine-tuned-SmolLM2-1.7B-Instruct |
| Base model | HuggingFaceTB/SmolLM2-1.7B-Instruct |
| Model type | Causal language model with LoRA adapter |
| Intended task | Instruction-following and text generation |
| Fine-tuning method | Supervised fine-tuning (SFT) with LoRA |
| Training dataset | HuggingFaceTB/smoltalk |
| Training subset | 5,000 selected examples before train/validation split |
| Epochs | 1 |
| Maximum sequence length | 1,024 tokens |
LoRA rank (r) | 16 |
| LoRA alpha | 32 |
| LoRA dropout | 0.05 |
| LoRA target modules | q_proj, k_proj, v_proj, o_proj |
| Training quantization | 4-bit NF4 base-model loading (QLoRA-style setup) |
DAVID is an experimental fine-tuned adapter for evaluating instruction-following responses, structured output, and general text-generation prompts. It may be useful for prototyping and research.
It has not yet been shown to outperform the original base model on a controlled benchmark. Evaluate it on your own use case before relying on it in production.
This example uses 4-bit loading and requires a compatible CUDA GPU with bitsandbytes support.
bash
pip install -U torch transformers peft accelerate bitsandbytes sentencepiecepython
import torchfrom transformers import ( AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig,)from peft import PeftModel BASE_MODEL_ID = "HuggingFaceTB/SmolLM2-1.7B-Instruct"ADAPTER_MODEL_ID = "Nikhil1581/DAVID-fine-tuned-SmolLM2-1.7B-Instruct" compute_dtype = torch.float16 quantization_config = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_use_double_quant=True, bnb_4bit_compute_dtype=compute_dtype,) tokenizer = AutoTokenizer.from_pretrained(ADAPTER_MODEL_ID) base_model = AutoModelForCausalLM.from_pretrained( BASE_MODEL_ID, quantization_config=quantization_config, torch_dtype=compute_dtype, device_map="auto",) model = PeftModel.from_pretrained(base_model, ADAPTER_MODEL_ID)model.eval() messages = [ { "role": "user", "content": "List exactly three benefits of unit testing.", }] prompt = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True,) inputs = tokenizer(prompt, return_tensors="pt").to(model.device) with torch.no_grad(): output_ids = model.generate( **inputs, max_new_tokens=160, do_sample=False, pad_token_id=tokenizer.eos_token_id, ) new_tokens = output_ids[0][inputs["input_ids"].shape[1]:]print(tokenizer.decode(new_tokens, skip_special_tokens=True))For CPU or non-CUDA deployment, use a compatible loading configuration and inference runtime; the 4-bit bitsandbytes example above is intended for supported CUDA environments.
The initial experiment used:
HuggingFaceTB/SmolLM2-1.7B-InstructHuggingFaceTB/smoltalkq_proj, k_proj, v_proj, o_projThe base model was loaded in 4-bit NF4 for training, with LoRA adapter parameters trained while the base weights remained frozen.
| Metric | Step 100 | Step 200 | Final logged step (304) |
|---|---|---|---|
| Training loss | 0.7132 | 0.7201 | 0.7432 |
| Validation loss | 0.6588 | 0.6546 | 0.6541 |
| Entropy | 0.6919 | 0.6989 | 0.7032 |
| Mean token accuracy | 81.12% | 81.25% | 81.29% |
The trainer's aggregate final training loss was approximately 0.7423. These metrics describe the training run; they do not by themselves demonstrate improved instruction-following.
Three manual smoke tests were run:
name and status values.These checks are qualitative and too limited to establish general performance.
Before claiming improvement, compare DAVID with the original base model using the same prompts, generation parameters, and evaluation procedure.
Recommended checks:
Keep benchmark prompts separate from training and hyperparameter selection. Report the benchmark version, decoding settings, scoring method, and baseline scores.
The base model's model card lists its license and usage terms. Review and comply with the base model and dataset terms before redistributing or deploying this adapter:
The license metadata at the top of this card should be checked against the base model's current license and your intended redistribution. Do not assume that uploading an adapter changes the obligations associated with the base model or training data.
Status: Initial adapter training completed. Controlled baseline evaluation remains future work.
pip install -U torch transformers peft accelerate bitsandbytes sentencepieceimport torchfrom transformers import ( AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig,)from peft import PeftModel BASE_MODEL_ID = "HuggingFaceTB/SmolLM2-1.7B-Instruct"ADAPTER_MODEL_ID = "Nikhil1581/DAVID-fine-tuned-SmolLM2-1.7B-Instruct" compute_dtype = torch.float16 quantization_config = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_use_double_quant=True, bnb_4bit_compute_dtype=compute_dtype,) tokenizer = AutoTokenizer.from_pretrained(ADAPTER_MODEL_ID) base_model = AutoModelForCausalLM.from_pretrained( BASE_MODEL_ID, quantization_config=quantization_config, torch_dtype=compute_dtype, device_map="auto",) model = PeftModel.from_pretrained(base_model, ADAPTER_MODEL_ID)model.eval() messages = [ { "role": "user", "content": "List exactly three benefits of unit testing.", }] prompt = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True,) inputs = tokenizer(prompt, return_tensors="pt").to(model.device) with torch.no_grad(): output_ids = model.generate( **inputs, max_new_tokens=160, do_sample=False, pad_token_id=tokenizer.eos_token_id, ) new_tokens = output_ids[0][inputs["input_ids"].shape[1]:]print(tokenizer.decode(new_tokens, skip_special_tokens=True))