GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
knc6
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
knc6
Model Tree
unsloth/mistral-7b-bnb-4bit
Input Modalities
Output Modalities
Supported Functionality
QLoRA (initialized from the DiffractGPT explicit-formula checkpoint), LoRA r=16, alpha=16 on q,k,v,o,gate,up,down; lr 2e-4 linear, effective batch 8, 2 epochs, 4-bit base, seed 3407.
python
from transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModelimport torchbase = "unsloth/mistral-7b-bnb-4bit"tok = AutoTokenizer.from_pretrained(base)model = PeftModel.from_pretrained( AutoModelForCausalLM.from_pretrained(base, device_map="auto"), "knc6/ramangpt_dft_model")prompt = ("### Instruction:\nBelow is a description of a material.\n### Input:\n" "The chemical formula is: MoS2.\nThe Raman spectrum shows main peaks at: " "383 cm-1(1.0), 408 cm-1(0.9).\nGenerate atomic structure description with " "lattice lengths, angles, coordinates and atom types.\n### Output:\n")ids = tok(prompt, return_tensors="pt").to(model.device)print(tok.decode(model.generate(**ids, max_new_tokens=512)[0][ids.input_ids.shape[1]:], skip_special_tokens=True))