Model Details
Model Description
This is a LoRA (Low-Rank Adaptation) adapter fine-tuned on top of google/gemma-4-31B-it, specializing the base model for financial and quantitative reasoning tasks. The adapter was trained as part of a submission to the AutoScientist Challenge 2026 (Finance category) hosted by Adaption Labs, using their Adaptive Data platform for data adaptation and AutoScientist for fine-tuning.
The model is adapted for tasks including:
- Multi-hop financial statement reasoning (cross-period ratio analysis, margin/leverage/liquidity comparisons)
- Verification of narrative claims (10-K MD&A statements) against reported financial figures
- Numeric extraction and calculation over financial tables (revenue, margins, EPS, balance sheet items)
- Tokenomics and crypto-native financial reasoning (spot-based instruments only)
The model is explicitly constrained to avoid reasoning about interest-bearing instruments, leverage/margin trading, derivatives, prediction markets, and meme coins, in line with the dataset's design blueprint (see Training Data below).
- Model type: Causal language model, LoRA adapter
- Language(s): English
- License: Apache 2.0 (inherited from base model)
- Finetuned from model: google/gemma-4-31B-it
- Fine-tuning platform: AutoScientist by Adaption Labs
Model Sources
Uses
Direct Use
This adapter is intended to be loaded on top of google/gemma-4-31B-it for financial reasoning tasks: computing and comparing financial ratios across periods, verifying whether narrative claims in financial disclosures (e.g. MD&A sections of 10-K filings) are numerically consistent with reported figures, and answering extraction/calculation questions over financial statements and tables.
Downstream Use
Suitable as a component in financial research tools, automated 10-K/earnings analysis pipelines, or as a teaching/demonstration model for finance-domain LLM fine-tuning. Not intended for live trading decisions or as a substitute for professional financial advice.
Out-of-Scope Use
This model is not intended for and has not been trained on:
- Interest-bearing instrument analysis (loans, bonds, interest rate products)
- Leverage or margin trading strategies
- Derivatives (futures, options) pricing or analysis
- Prediction market reasoning
- Meme coin / speculative crypto-asset analysis
These exclusions were deliberately built into the training data's blueprint constraints (see Training Data) and reflect the dataset's halal-finance design intent. The model should not be relied upon for any of the above use cases, financial advice, or investment decisions generally. The model may inherit limitations and biases from both the base model (google/gemma-4-31B-it) and the training data sources.
Bias, Risks, and Limitations
- Training data was partially platform-generated. A portion of the training data's completions were elaborated/enhanced by Adaption Labs' Adaptive Data platform using its own generation model. While base claims and directional labels (e.g. "revenue increased/decreased") were verified against real SEC XBRL data, some enhanced completions may include illustrative figures (e.g. segment-level breakdowns) that were not independently verified against source filings and could be approximate or fabricated for narrative completeness.
- Extraction pipeline limitations. A portion of the training data (MD&A narrative verification rows) was built using a regex-based extraction pipeline rather than an LLM, with validated but inherently incomplete coverage of company-specific terminology (e.g. business segment names, product-line categories). Rows with detectable extraction ambiguity were filtered out, but novel phrasings not seen during development may not generalize perfectly.
- English-language, US-filer focus. Training data is sourced primarily from SEC filings of US-listed companies; performance on non-English financial documents or non-US regulatory contexts is untested.
- Not verified for production financial use. This model has not undergone the level of validation required for use in regulated financial contexts.
Recommendations
Users should independently verify any specific financial figures or claims generated by this model against primary source documents (10-K filings, financial statements) before relying on them for any consequential decision.
How to Get Started with the Model
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base_model = AutoModelForCausalLM.from_pretrained("google/gemma-4-31B-it")
tokenizer = AutoTokenizer.from_pretrained("google/gemma-4-31B-it")
model = PeftModel.from_pretrained(base_model, "0xZaid10/gemma-4-31b-financial-reasoning-lora")
Training Details
Training Data
The model was fine-tuned on the Financial Quantitative and Crypto Dataset, combining four component datasets built and adapted via Adaption Labs' Adaptive Data platform:
- financial_qa_with_reasoning: Multi-hop numeric reasoning over SEC XBRL financial facts for 150 curated US Mega/Large-Cap companies, plus MD&A narrative claims verified against reported financial figures, plus supplementary financial QA (FinQA/ConvFinQA/TAT-QA-derived) reasoning examples.
- mda_financial_extraction: Additional MD&A-derived extraction examples.
- financial_ratio_calculation: Additional financial ratio calculation examples.
- crypto_supply_metrics: Crypto-native financial reasoning examples (spot-based instruments, tokenomics, supply/circulation metrics only).
All component datasets follow a shared blueprint constraint excluding interest-bearing instruments, leverage/margin trading, derivatives, prediction markets, and meme coins.
Combined dataset diagnosis: quality score improved from 7.0 (raw, grade C, 14.4th percentile) to 8.8 (adapted, grade A, 44.8th percentile) on Adaptive Data's evaluation — a 25.7% relative improvement.
Training Procedure
Training Hyperparameters
- Training method: Supervised fine-tuning (SFT)
- Algorithm: LoRA
- Base model size: 31B
- LoRA rank (r): 16
- LoRA alpha: 32
- LoRA dropout: 0
- LoRA target modules:
q_proj, k_proj, v_proj, o_proj (full attention projections)
- Epochs: 1
- Batch size: max (platform-determined)
- Learning rate: 1e-4
- LR scheduler: cosine (min LR ratio 0.1, 0.5 cycles)
- Warmup ratio: 0.1
- Max gradient norm: 1.0
- 0.01
Speeds, Sizes, Times
- Trained adapter size: ~87.7 MB (
adapter_model.safetensors)
- Training platform: AutoScientist (Adaption Labs)
Evaluation
Results
Evaluated via AutoScientist's training win-rate metric, comparing the adapted model against the base google/gemma-4-31B-it model on the training dataset's held-out evaluation split. See the AutoScientist training dashboard for full win-rate and loss curves.
Technical Specifications
Model Architecture and Objective
LoRA adapter (rank 16, alpha 32) targeting attention projections (q/k/v/o) of google/gemma-4-31B-it, a 31B-parameter dense multimodal model with hybrid local/global attention. Fine-tuning objective: supervised next-token prediction on completion tokens only.
Compute Infrastructure
Training was performed via Adaption Labs' AutoScientist platform (managed compute).
Citation
If you use this model, please credit Adaption Labs' Adaptive Data and AutoScientist platforms, which were used to adapt the training data and perform fine-tuning, in addition to this repository.
APA:
Zaid, M. (2026). Gemma 4 31B IT — Financial Quantitative Reasoning (LoRA) [Computer software]. Hugging Face. Built using Adaption Labs' Adaptive Data and AutoScientist, as part of the AutoScientist Challenge 2026.
Framework versions