GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
darshit-0391
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
darshit-0391
Model Tree
meta-llama/Llama-3.1-8B-Instruct
Input Modalities
Output Modalities
Supported Functionality
| Field | Details |
|---|---|
| Developed by | Patel Darshit |
| Base Model | meta-llama/Meta-Llama-3.1-8B-Instruct |
| Model Type | Causal Language Model (QLoRA Fine-tuned) |
| Languages | English, Hindi |
| License | Apache 2.0 |
| Fine-tuning Method | LoRA (r=16), 4-bit Quantization |
Velora v1 was trained on a curated mix of:
| Parameter | Value |
|---|---|
| LoRA Rank (r) | 16 |
| LoRA Alpha | 16 |
| Optimizer | AdamW 8-bit |
| Learning Rate | 2e-4 |
| Effective Batch Size | 8 |
| Training Steps | 5,000 |
| Precision | BFloat16 |
| GPU | NVIDIA A100 SXM4 40GB |
python
import torchfrom transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfigfrom peft import PeftModel base_model_id = "unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit"adapter_id = "darshit-0391/Velora-v1-8B" tokenizer = AutoTokenizer.from_pretrained(adapter_id) bnb_config = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.bfloat16) base_model = AutoModelForCausalLM.from_pretrained( base_model_id, quantization_config=bnb_config, device_map="auto") model = PeftModel.from_pretrained(base_model, adapter_id)model.eval() prompt = """<|begin_of_text|><|start_header_id|>user<|end_header_id|> Write a Python function to check if a number is prime.<|eot_id|><|start_header_id|>assistant<|end_header_id|>""" inputs = tokenizer(prompt, return_tensors="pt").to("cuda")with torch.no_grad(): outputs = model.generate( **inputs, max_new_tokens=512, temperature=0.3, do_sample=True, pad_token_id=tokenizer.eos_token_id ) print(tokenizer.decode(outputs[0], skip_special_tokens=True))Prompt: Write a Python asyncio script to fetch 3 URLs concurrently and explain how the event loop works.
Velora v1 Response: Correctly generated a production-ready asyncio + aiohttp implementation with a five-point structured explanation of the Python event loop model.
| Field | Details |
|---|---|
| Hardware | NVIDIA A100 SXM4 40GB |
| Training Time | ~6 hours |
| Platform | AIKosh |
| Cloud Region | India |
import torchfrom transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfigfrom peft import PeftModel base_model_id = "unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit"adapter_id = "darshit-0391/Velora-v1-8B" tokenizer = AutoTokenizer.from_pretrained(adapter_id) bnb_config = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.bfloat16) base_model = AutoModelForCausalLM.from_pretrained( base_model_id, quantization_config=bnb_config, device_map="auto") model = PeftModel.from_pretrained(base_model, adapter_id)model.eval() prompt = """<|begin_of_text|><|start_header_id|>user<|end_header_id|> Write a Python function to check if a number is prime.<|eot_id|><|start_header_id|>assistant<|end_header_id|>""" inputs = tokenizer(prompt, return_tensors="pt").to("cuda")with torch.no_grad(): outputs = model.generate( **inputs, max_new_tokens=512, temperature=0.3, do_sample=True, pad_token_id=tokenizer.eos_token_id ) print(tokenizer.decode(outputs[0], skip_special_tokens=True))