Results
1,999 held-out complaints from the CFPB Consumer Complaint Database, same test set for every model.
Table with columns: Model, Product accuracy, Issue accuracy, Both correct| Model | Product accuracy | Issue accuracy | Both correct |
|---|
| Claude Sonnet 5.5, 5 retrieved training complaints as examples | 84.7% | 56.6% | 55.8% |
| This adapter (Qwen3-8B + LoRA, rank 16) | 83.5% | 55.0% | 54.4% |
| Reflection Beam (501B mixture of experts, 23B active), prompt only | 82.9% | 33.2% | 31.5% |
| Claude Sonnet 5.5, prompt only | 82.3% | 44.1% | 42.9% |
| GPT-5.4, prompt only, medium reasoning | 82.8% | 38.7% | 37.3% |
| GPT-5.4, prompt only (default: no reasoning) | 82.3% | 33.9% | 32.1% |
| Claude Haiku 5.5, prompt only | 83.6% | 37.8% | 35.9% |
| Claude Haiku 5.5, 5 retrieved examples | 84.4% | 47.9% | — |
| TF-IDF + logistic regression (same 8,000 training complaints) | 80.8% | 52.6% | — |
Product accuracy by class: Mortgage 94% (184 test complaints), Student loan 94% (69), Credit reporting 87% (487),
Checking or savings account 85% (266), Debt collection 82% (422), Credit card or prepaid card 80% (354), Money
transfer, virtual currency or money service 76% (94), Vehicle loan or lease 71% (83), Payday, title or personal loan
60% (40). "Debt or credit management" has no test complaints.
Product accuracy sits near 82-85% for every model we tried, including much larger ones: the label is the consumer's
own choice and some products overlap (credit reporting vs debt collection). The 82-class issue label is where
training on the data pays off: +11 points over Claude Sonnet, +16 over GPT-5.4 with medium reasoning (+21 at its
default), +17 over Claude Haiku and +22 over a 501B model, all prompted without examples. Sonnet given the five most similar training complaints as examples is 1.6 points ahead
on issue, at roughly 600 times the cost per complaint in real time (below). On the first 200 test complaints only, Claude Opus 5.5
prompted without examples reached 61.5% issue accuracy (this adapter: 51.5% on the same 200); not run on the full set.
Cost per 1,000 complaints (USD)
Table with columns: Model, Real-time, Batch| Model | Real-time | Batch |
|---|
| This adapter, self-hosted on one NVIDIA A30 | ~$0.012 | ~0.014∗∗(10,000perbatch)to∗∗ 0.034 (1,000 per batch) |
| Claude Haiku 5.5, prompt only | ~$0.26 | ~$0.07 |
| Claude Sonnet 5.5, prompt only | ~$3.75 | ~$0.73 |
| Claude Sonnet 5.5, 5 examples | ~$7.0 | ~$2.4 |
| GPT-5.4, prompt only (default) | ~$2.7 | ~$1.4 |
| GPT-5.4, prompt only, medium reasoning | ~$5.4 | ~$2.7 |
Estimates, not invoices. Self-hosted: one NVIDIA A30 in India at about 0.41/hour(listprice₹38.88,convertedatUSD1=₹95),about10complaints/s,billedperminute;real−timeassumestheGPUisbusy(atlowvolumeadedicatedGPUcostsabout9.80/day whatever the traffic, so run in
batches or share it across tasks); batch includes about 160 s of GPU start-up. API: list prices (Haiku 5.5 0.10/0.50,
Sonnet 5.5 2/10, GPT-5.4 2.50/15 per million input/output tokens; cached reads 0.20forSonnet,0.25 for GPT-5.4) on our measured tokens per
complaint; real-time without caching; batch applies the 50% batch discount and caches the shared instructions (the
retrieved examples differ per complaint and are not cached). Corrected 22 Oct 2026: earlier figures used older models'
prices. At bank volumes all are small in absolute terms; a small API model such as Haiku is close on cost, so the
adapter's case is fine-grained accuracy, data residency and control as much as cost.
Use
Base model Qwen/Qwen3-8B (revision b968826d9c46dd6066d109eabc6255188de91218). Greedy decoding (temperature 0),
max_tokens 80, thinking disabled. The adapter was trained on exactly this prompt:
- System:
You file US consumer finance complaints by product and issue. Reply with JSON only: {"product": ..., "issue": ...}.
- User:
Complaint: {text}\nAnswer:
With vLLM:
vllm serve Qwen/Qwen3-8B --revision b968826d9c46dd6066d109eabc6255188de91218 \ --enable-lora --max-lora-rank 16 --lora-modules us-cfpb=Gydeai/complaint-classifier-us-cfpb
from openai import OpenAIclient = OpenAI(base_url="http://localhost:8000/v1", api_key="local")r = client.chat.completions.create( model="us-cfpb", temperature=0, max_tokens=80, extra_body={"chat_template_kwargs": {"enable_thinking": False}}, messages=[{"role": "system", "content": 'You file US consumer finance complaints by product and issue. ' 'Reply with JSON only: {"product": ..., "issue": ...}.'}, {"role": "user", "content": "Complaint: My card was charged twice and the bank will not reverse it.\nAnswer:"}])print(r.choices[0].message.content) # {"product": "Credit card or prepaid card", ...}
labels.json lists the 10 products and 82 issues; universal-taxonomy.json maps each product to the shared
categories.
Gyde also serves this model inside India with data-residency, policy and failover controls, and trains versions on
your own categories: gyde.ai.
Training
LoRA rank 16, alpha 32, dropout 0.05 on all attention and MLP projections; 1 epoch over 8,000 complaints, learning
rate 2e-4; about 40 minutes on one A100 40 GB (run.json).
Limits
- US complaints and US categories only. An Indian UPI or ATM complaint will be put in the nearest US category.
- English only. Narratives are as written by US consumers, with the CFPB's redactions.
- For triage and analytics, not for deciding a customer's case. Send low-confidence answers to a person.
- Base model origin: Qwen (Alibaba Cloud). A version on another base model is available on request.
Data and licence
Training and test data: CFPB Consumer Complaint Database,
a US government work. Adapter: Apache 2.0. Base model: Qwen3-8B under its own licence (Apache 2.0).
Built by Gyde (DAI Labs Private Limited, Pune, India).