GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
d1wash
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
d1wash
Model Tree
mistralai/Mistral-7B-Instruct-v0.2
Input Modalities
Output Modalities
Supported Functionality
This adapter is intended to be loaded on top of the Mistral-7B-Instruct-v0.2 base model to generate conversational responses in a medical Q&A style, for research, demonstration, and educational purposes.
Users should treat all outputs as illustrative only, verify any medical claims with a qualified professional, and not rely on this model for real health decisions.
python
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfigfrom peft import PeftModel base_model_name = "mistralai/Mistral-7B-Instruct-v0.2"adapter_repo = "d1wash/mistral-7b-healthcare-qlora" tokenizer = AutoTokenizer.from_pretrained(base_model_name)tokenizer.pad_token = tokenizer.eos_token bnb_config = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.float16, bnb_4bit_use_double_quant=True,) base_model = AutoModelForCausalLM.from_pretrained( base_model_name, quantization_config=bnb_config, device_map="auto",) model = PeftModel.from_pretrained(base_model, adapter_repo) prompt = """<s>[INST] You are a helpful medical assistant.I have been experiencing persistent headaches for the past week. What could be the cause? Patient: I wake up with a throbbing headache every morning. [/INST] Doctor:""" inputs = tokenizer(prompt, return_tensors="pt").to(model.device)output = model.generate( **inputs, max_new_tokens=200, do_sample=True, temperature=0.7, top_p=0.9, repetition_penalty=1.3, no_repeat_ngram_size=3, pad_token_id=tokenizer.eos_token_id,)print(tokenizer.decode(output[0], skip_special_tokens=True))A filtered, subsampled subset of ChatDoctor-HealthCareMagic-100k:
<s>[INST] ... [/INST] ...q_proj, v_projTraining and validation loss decreased consistently across checkpoints:
| Step | Training Loss | Validation Loss |
|---|---|---|
| 150 | 2.004 | 2.011 |
| 300 | 1.900 | 1.907 |
| 450 | 1.880 | 1.885 |
| 600 | 1.903 | 1.870 |
Validation loss tracked closely with training loss throughout, indicating the model generalized to the held-out set rather than memorizing the training data. Training completed the full 900-step run with the best checkpoint (by evaluation loss) restored at the end.
NVIDIA T4 GPU (16GB VRAM), 4 vCPUs — Lightning AI Studio free tier
transformers, peft, bitsandbytes, accelerate, trlDiwash Adhikari — d1wash on Hugging Face | GitHub: Diwash17
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfigfrom peft import PeftModel base_model_name = "mistralai/Mistral-7B-Instruct-v0.2"adapter_repo = "d1wash/mistral-7b-healthcare-qlora" tokenizer = AutoTokenizer.from_pretrained(base_model_name)tokenizer.pad_token = tokenizer.eos_token bnb_config = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.float16, bnb_4bit_use_double_quant=True,) base_model = AutoModelForCausalLM.from_pretrained( base_model_name, quantization_config=bnb_config, device_map="auto",) model = PeftModel.from_pretrained(base_model, adapter_repo) prompt = """<s>[INST] You are a helpful medical assistant.I have been experiencing persistent headaches for the past week. What could be the cause? Patient: I wake up with a throbbing headache every morning. [/INST] Doctor:""" inputs = tokenizer(prompt, return_tensors="pt").to(model.device)output = model.generate( **inputs, max_new_tokens=200, do_sample=True, temperature=0.7, top_p=0.9, repetition_penalty=1.3, no_repeat_ngram_size=3, pad_token_id=tokenizer.eos_token_id,)print(tokenizer.decode(output[0], skip_special_tokens=True))