Dedicated Endpoints
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Container
Run this model inference with full control and performance in your environment.
Get help setting up a custom Dedicated Endpoints.
Talk with our engineer to get a quote for reserved GPU instances with discounts.
README
License: apache-2.0Evaluation Results
| Dataset | Base Accuracy | FT Accuracy | Base Macro-F1 | FT Macro-F1 |
|---|---|---|---|---|
| FPB in-domain | 0.8908 | 0.9748 | 0.8765 | 0.9725 |
| FiQA-SA OOD | 0.8120 | 0.9402 | 0.6705 | 0.8335 |
Baseline = zero-shot Meta-Llama-3.1-8B-Instruct with the same prompt template.
Quick Start
python
from peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizerbase = AutoModelForCausalLM.from_pretrained("meta-llama/Meta-Llama-3.1-8B-Instruct",load_in_4bit=True,device_map="auto",)model = PeftModel.from_pretrained(base, "jhon53/Llama3_1_8B_Finance_QLoRA")tokenizer = AutoTokenizer.from_pretrained("jhon53/Llama3_1_8B_Finance_QLoRA")
Training Details
| Param | Value |
|---|---|
| Base model | meta-llama/Meta-Llama-3.1-8B-Instruct |
| Method | QLoRA (4-bit NF4 + LoRA bf16) |
| LoRA rank (r) | 16 |
| LoRA alpha | 32 |
| LoRA dropout | 0.05 |
| Target modules | q, k, v, o, gate, up, down projections |
| Training data | FinGPT/fingpt-sentiment-train (~76k) |
| Optimizer | AdamW 8-bit |
| Learning rate | 2e-4 (cosine schedule) |
| Epochs | 3 (early stopping, patience=3) |
| Loss | Completion-only cross-entropy |
Related Repos
- GGUF q4_k_m: jhon53/Llama3_1_8B_Finance_QLoRA-GGUF
- model: jhon53/Llama3_1_8B_Finance_QLoRA
Uploaded finetuned model
- Developed by: jhon53
- License: apache-2.0
- Finetuned from model : unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit
Model provider
jhon53
Model tree
Base
unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit
Fine-tuned
this model
Modalities
Input
Text
Output
Text
Pricing
Dedicated Endpoints
View detailsSupported Functionality
Model APIs
Dedicated Endpoints
Container
More information