dustin2050

finrag-trading-expert-qwen3-32b

README

License: apache-2.0

Overview

A domain-specific fine-tuned model distillation of Qwen3-32B for quantitative trading strategy analysis, backtest auditing, and risk management. The model was trained on 6,305 high-quality examples generated by GLM-5.2 as teacher, covering forex, crypto, stocks, and commodities markets.

Training Details

Table with columns: Parameter, Value
Parameter	Value
Base Model	Qwen/Qwen3-32B (4-bit quantized)
Method	QLoRA (4-bit NF4 + LoRA r=64)
Teacher	GLM-5.2 via OpenRouter
Framework	Unsloth 2026.7.5 + TRL 0.24.0
Hardware	NVIDIA RTX PRO 6000 Blackwell (96GB VRAM)
Training Time	11h 20min
Total Steps	1,146 (3 epochs)
Batch Size	16 (8 × 2 gradient accumulation)
Learning Rate	2e-4
Max Sequence Length	2048
LoRA Rank	64
LoRA Alpha	64
LoRA Target Modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Trainable Parameters	536,870,912 (1.61% of 33.3B)
Optimizer	adamw_8bit
Packing	Enabled (Unsloth padding-free)

Training Data

The training dataset contains 6,305 examples from 7 sources:

Table with columns: Source, Examples, Description
Source	Examples	Description
arXiv q-fin Abstracts	2,334	Papers on trading, momentum, arbitrage, risk management (2023-2026)
Self-Instruct v1+v2	1.892	Strategy design, cost analysis, regime detection, risk management scenarios
Trading-Fable5	319	Real failure patterns from 38 trading strategy tracks (look-ahead bias, OOS collapse)
Forex_Renko Phase-Docs	232	Phase 0-40 audit results, Codex reviews, backtest validation
Workspace Projects	827	Bot_KI_Agents, Reverse_Engineering_XAUUSD, stock3, ML-DT-Trading, xauusd-cbot

Data Quality Pipeline

Paper Collection: 6,089 unique papers from arXiv (q-fin.*) and OpenAlex
Fulltext Download: 200 top papers downloaded as PDF, extracted to text (~15k chars each)
Teacher Generation: GLM-5.2 generates detailed answers with chain-of-thought reasoning
Post-Processing:
- <think> tags for reasoning separation (99% tagged)
- Deduplication (max 30 per template for Self-Instruct, 50 for papers)
- Truncation to max 14k chars (seq_length compatible)
- Removal of generic finance Q&A (FinAlpaca removed — not trading-relevant)

Evaluation Results

Evaluated on 40 trading-specific questions comparing Base Qwen3-32B vs. Distilled:

Table with columns: Metric, Base Qwen3-32B, Distilled (LoRA)
Metric	Base Qwen3-32B	Distilled (LoRA)
Avg Response Length	8,450 chars	6,854 chars
Avg Time per Answer	116.7s	152.6s
Questions Answered	40/40	40/40

Qualitative Improvements

The distilled model shows:

Direct Skepticism: Immediately flags suspicious CAGR (>100%) as red flag, unlike base model which explains basic concepts first
Structured Output: Uses numbered lists with concrete bias types (look-ahead, survivorship, data-snooping)
Domain Precision: Mentions specific concepts (Kelly criterion, McLean & Pontiff alpha decay, walk-forward validation, OOS degradation)
Compact Expertise: Less introductory text, more actionable analysis
Real Failure Patterns: Draws on training data from actual failed trading strategies (Trading-Fable5 tracks, Forex_Renko phases)

Category Coverage

Table with columns: Category, Questions, Topics
Category	Questions	Topics
Overfitting Detection	4	CAGR red flags, Sharpe degradation, compound CAGR bias
Validation	4	Walk-forward, train/test split, param vs cost sensitivity
Risk Management	6	Drawdown filters, Kelly criterion, kill-switch, vol-targeting
Bias Detection	4	Look-ahead, survivorship, data snooping vs overfitting
Strategy Design	6	BTC momentum, crypto pairs, execution plans, funding arbitrage
Strategy Comparison

Usage

With Unsloth

python
from unsloth import FastLanguageModel
import torch

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="dustin2050/finrag-trading-expert-qwen3-32b",
    max_seq_length=2048,
    dtype=None,
    load_in_4bit=True,
)
FastLanguageModel.for_inference(model)

messages = [{"role": "user", "content": "Design a momentum strategy for EUR/USD with walk-forward validation."}]
inputs = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt").to("cuda")
outputs = model.generate(input_ids=inputs, max_new_tokens=2048, temperature=0.6, top_p=0.95)
print(tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True))

With PEFT (transformers)

python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-32B", torch_dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(base_model, "dustin2050/finrag-trading-expert-qwen3-32b")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-32B")

messages = [{"role": "user", "content": "Audit this backtest: 567% CAGR, 30% DD. What are the red flags?"}]
inputs = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt").to("cuda")
outputs = model.generate(input_ids=inputs, max_new_tokens=2048, temperature=0.6, top_p=0.95)
print(tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True))

Recommended Inference Settings

Table with columns: Parameter, Value
Parameter	Value
Temperature	0.6 (thinking mode)
Top_P	0.95
Top_K	20
Max New Tokens	2048

Limitations

Not Financial Advice: This model generates analysis for research purposes only. Do not use for real trading decisions without independent validation.
No Live Trading: The model cannot execute trades or access real-time market data.
Knowledge Cutoff: Training data covers papers up to July 2026.
Reasoning Depth: While the model shows structured trading analysis, it cannot replace full backtesting or professional risk management.
GGUF Export: Currently only LoRA adapter is available. GGUF export pending.

Training Infrastructure

Cloud Provider: Vast.ai (RTX PRO 6000 Blackwell, 96GB VRAM)
Training Framework: Unsloth 2026.7.5 (2x faster, 70% less VRAM)
Teacher API: OpenRouter (GLM-5.2 / z-ai/glm-5.2)
Data Pipeline: Custom Python pipeline (arXiv API + OpenAlex API + PyMuPDF)

Acknowledgments

Unsloth for the optimized training framework
Qwen Team for the base model
OpenRouter for API access to GLM-5.2
arXiv and OpenAlex for paper access

License

Apache 2.0 (inherited from Qwen3-32B base model)

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

dustin2050

Model Tree

Base

Qwen/Qwen3-32B

Adapter

this model

Input Modalities

Text

Output Modalities