Arinkc

pydoc-llama-r16-full

Deploy Dedicated

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Get help setting up a custom Dedicated Endpoints.

Talk with our engineer to get a quote for reserved GPU instances with discounts.

Model Details

Base model: meta-llama/Llama-3.1-8B-Instruct
Fine-tuning method: QLoRA (4-bit NF4 + LoRA rank 16)
Training data: Arinkc/pydoc-llama-codesearchnet-curated
Training: 3 epochs, 22,473 examples, A100 GPU, 4h 51m
Loss: 2.3 → 0.63

Evaluation (200 held-out examples)

Metric	Base	Fine-Tuned
Hallucinated exceptions	11%	0%
Verbose outputs (>80 words)	19.5%	0%
Format compliance	80.5%	100%

Usage

python
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
import torch

BASE = "meta-llama/Llama-3.1-8B-Instruct"
ADAPTER = "Arinkc/pydoc-llama-r16-full"

bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
                          bnb_4bit_compute_dtype=torch.bfloat16)
tokenizer = AutoTokenizer.from_pretrained(BASE)
model = PeftModel.from_pretrained(
    AutoModelForCausalLM.from_pretrained(BASE, quantization_config=bnb, device_map="auto"),
    ADAPTER,
)
model.eval()