GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
tarun7r
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
tarun7r
Model Tree
unsloth/Meta-Llama-3.1-8B
Input Modalities
Output Modalities
Supported Functionality
Overview Finance-Instruct-500k is a comprehensive and meticulously curated dataset designed to train advanced language models for financial tasks, reasoning, and multi-turn conversations. Combining data from numerous high-quality financial datasets, this corpus provides over 500,000 entries, offering unparalleled depth and versatility for finance-related instruction tuning and fine-tuning.
The dataset includes content tailored for financial reasoning, question answering, entity recognition, sentiment analysis, address parsing, and multilingual natural language processing (NLP). Its diverse and deduplicated entries make it suitable for a wide range of financial AI applications, including domain-specific assistants, conversational agents, and information extraction systems.
Key Features of the Dataset
The CFA (Chartered Financial Analyst) exam is widely recognized as one of the most challenging professional certifications in the financial industry, typically requiring over 1000 hours of study across all three levels. The evaluation concept for the CFA Level 1 mock exam was inspired by the work on mukaj/Llama-3.1-Hawkish-8B. Below is a comparison of different models on a sample Level 1 CFA Mock Exam, demonstrating how Finance-Llama-8B performs on the exam. The same prompt was used for all models. The results presented are approximated and have been tested across multiple mock exam papers to ensure consistency. A sample mock exam with a comparison to other models is shown below.
The mock exams are designed to challenge candidates with varying levels of difficulty, reflecting the rigorous nature of the CFA Level 1 exam. Pass rates for these mock exams typically range from 64% to 72%, with an average pass rate of around 67%. This average is notably higher than the 12-year average Minimum Passing Score (MPS) of 65% for all CFA years, indicating the effectiveness of the preparation materials. For more detailed insights, visit 300hours.com/cfa-passing-score.
You can also use this model with Ollama. Pre-built GGUF versions (FP16 and Q4_K_M) are available at: ollama.com/martain7r/finance-llama-8b
To run the FP16 version:
bash
ollama run martain7r/finance-llama-8b:fp16
To run the Q4_K_M quantized version (smaller and faster, with a slight trade-off in quality):
bash
ollama run martain7r/finance-llama-8b:q4_k_m
This model can be used with the transformers library pipeline for text generation.
First, make sure you have the transformers and torch libraries installed:
bash
pip install transformers torch
Usage 🚀 Transformers Pipeline
bash
from transformers import pipeline, AutoModelForCausalLM, AutoTokenizerimport torch# Alternative memory-efficient loading options without bitsandbytesmodel_id = "tarun7r/Finance-Llama-8B"print("Loading model with memory optimizations...")# Option 1: Use FP16 (half precision) - reduces memory by ~50%try:print("Trying FP16 loading...")model = AutoModelForCausalLM.from_pretrained(model_id,torch_dtype=torch.float16, # Half precisiondevice_map="auto", # Automatic device placementlow_cpu_mem_usage=True, # Efficient CPU memory usage during loadingtrust_remote_code=True)print("✓ Model loaded with FP16")except Exception as e:print(f"FP16 loading failed: {e}")# Option 2: CPU offloading - some layers on GPU, some on CPUtry:print("Trying CPU offloading...")model = AutoModelForCausalLM.from_pretrained(model_id,torch_dtype=torch.float16,device_map="balanced", # Balance between GPU and CPUlow_cpu_mem_usage=True,trust_remote_code=True)print("✓ Model loaded with CPU offloading")except Exception as e:print(f"CPU offloading failed: {e}")# Option 3: Full CPU loading as fallbackprint("Loading on CPU...")model = AutoModelForCausalLM.from_pretrained(model_id,torch_dtype=torch.float16,device_map="cpu",low_cpu_mem_usage=True,trust_remote_code=True)print("✓ Model loaded on CPU")# Load tokenizertokenizer = AutoTokenizer.from_pretrained(model_id)if tokenizer.pad_token is None:tokenizer.pad_token = tokenizer.eos_token# Create pipelinegenerator = pipeline("text-generation",model=model,tokenizer=tokenizer)print("✓ Pipeline created successfully!")# Your existing prompt codefinance_prompt_template = """Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.### Instruction:{}### Input:{}### Response:"""# Update the system prompt to provide a more detailed description of the chatbot's rolemessages = [{"role": "system", "content": "You are a highly knowledgeable finance chatbot. Your purpose is to provide accurate, insightful, and actionable financial advice to users, tailored to their specific needs and contexts."},{"role": "user", "content": "What strategies can an individual investor use to diversify their portfolio effectively in a volatile market?"},]# Update the generator call to use the messagesprompt = "\n".join([f"{msg['role'].capitalize()}: {msg['content']}" for msg in messages])print("\n--- Generating Response ---")try:outputs = generator(prompt,#max_new_tokens=250, # Reduced for memory efficiencydo_sample=True,temperature=0.7,top_p=0.9,pad_token_id=tokenizer.eos_token_id,# Memory efficient generation settingsnum_beams=1, # No beam search to save memoryearly_stopping=True,use_cache=True)# Extract responsegenerated_text = outputs[0]['generated_text']response_start = generated_text.rfind("### Response:")if response_start != -1:response = generated_text[response_start + len("### Response:"):].strip()print("\n--- Response ---")print(response)else:print(generated_text)# Clean up GPU memory after generationif torch.cuda.is_available():torch.cuda.empty_cache()except Exception as e:print(f"Generation error: {e}")
Citation 📌
markdown
@misc{tarun7r/Finance-Llama-8B,author = {tarun7r},title = {tarun7r/Finance-Llama-8B: A Llama 3.1 8B Model Fine-tuned on Josephgflowers/Finance-Instruct-500k},year = {2025},publisher = {Hugging Face},journal = {Hugging Face Model Hub},howpublished = {\url{https://huggingface.co/tarun7r/Finance-Llama-8B}}}
This model is an experimental research implementation based on Meta's LLaMA 3.1 architecture and is governed by the LLaMA 3.1 community license terms, with additional restrictions as outlined below. It is designed for academic and research purposes to explore the influence of financial data in training language models. Users are advised that this model is experimental and should be used at their own risk, with full responsibility for any implementation or application. This model is not a financial advisor and should not be used for financial decision-making.
The creators of this model: