Results
The 2,000 newest single-component complaints received 6 to 28 October 2026, all newer than every training complaint
(training: January 2025 to 25 September 2026). Same test set for every model.
Table with columns: Model, Accuracy, Macro-F1, Invalid answers| Model | Accuracy | Macro-F1 | Invalid answers |
|---|
| This adapter (Qwen3-8B + LoRA, rank 16) | 87.6% | 0.846 | 0 |
| Claude Sonnet 5.5, component list in the prompt | 86.3% | 0.828 | 0 |
| Claude Haiku 5.5, component list in the prompt | 85.6% | 0.810 | 1 |
| TF-IDF keyword classifier, same training data | 82.2% | | |
| Qwen3-8B untuned, component list in the prompt | 77.5% | 0.729 | 3 |
The adapter and Claude Sonnet are within about one percentage point of each other on this set; read it as a match,
not a lead. Fine-tuning added about 10 points over the untuned base model.
Cost per 1,000 complaints (USD)
Table with columns: Model, Real-time, Nightly batch of 1,000, Batch of 10,000| Model | Real-time | Nightly batch of 1,000 | Batch of 10,000 |
|---|
| This adapter on one NVIDIA A30 | ~$0.004 | ~$0.022 | ~$0.006 |
| This adapter on one NVIDIA L4 | ~$0.007 | ~$0.024 | ~$0.009 |
| Claude Haiku 5.5 | ~$0.11 | ~$0.056 | ~$0.056 |
| Claude Sonnet 5.5 | ~$1.68 | ~$0.84 | ~$0.84 |
- Adapter: measured in India with 64 requests in parallel, same accuracy as above (87.5% to 87.6%):
- A30 (JarvisLabs, $0.41/h, Rs 38.88/h at Rs 95 per USD): 2,000 complaints in 72 s (2 Nov 2026).
- L4 (CloudPe, $0.37/h, Rs 35.33/h): 2,000 complaints in 135 s (30 Oct and 2 Nov 2026).
- Real-time assumes the GPU is kept busy; batches include 160 s for starting the GPU. A GPU kept on all day for low
traffic costs about 9to10 a day whatever the volume.
- Claude: measured token counts on this test set at list prices checked 22 Oct 2026 (Haiku 0.10/0.50, Sonnet
2/10 per million input / output tokens); batch at the 50% batch discount. The tuned adapter needs no component
list in its prompt (about 220 input tokens against about 540).
Use
Base model Qwen/Qwen3-8B (revision b968826d9c46dd6066d109eabc6255188de91218). Greedy decoding (temperature 0),
thinking disabled. The adapter was trained on exactly this prompt:
- System:
You route US vehicle owners' defect complaints to the component team. Reply with JSON only: {"component": ...}.
- User:
Complaint: {text}\nAnswer:
With vLLM:
vllm serve Qwen/Qwen3-8B --revision b968826d9c46dd6066d109eabc6255188de91218 \ --enable-lora --max-lora-rank 16 --lora-modules us-nhtsa=Gydeai/vehicle-complaint-triage-us-nhtsa
from openai import OpenAIclient = OpenAI(base_url="http://localhost:8000/v1", api_key="local")r = client.chat.completions.create( model="us-nhtsa", temperature=0, max_tokens=40, extra_body={"chat_template_kwargs": {"enable_thinking": False}}, messages=[{"role": "system", "content": "You route US vehicle owners' defect complaints to the component team. " 'Reply with JSON only: {"component": ...}.'}, {"role": "user", "content": "Complaint: The brake pedal went to the floor at a stop light and the car " "rolled into the intersection.\nAnswer:"}])print(r.choices[0].message.content) # e.g. {"component": "SERVICE BRAKES"}
labels.json lists the 17 component groups.
Gyde trains versions on your own categories (warranty codes, service-ticket queues) and runs them in your cloud:
gyde.ai.
Training
LoRA rank 16, alpha 32, dropout 0.05 on all attention and MLP projections; 1 epoch over 8,000 complaints, learning
rate 2e-4; 20 minutes on one A100 80 GB (run.json). The loss covered only the answer (the component label), not
the complaint text.
Data
NHTSA Office of Defects Investigation complaints, file COMPLAINTS_RECEIVED_2025-2026 from
static.nhtsa.gov/odi/ffdd/cmpl (sha256 in labels.json), a US
government work. Kept: complaints filed under exactly one component, narratives of 200 to 4,000 characters, with
"Unknown or other" dropped, renamed groups merged (for example "Service brakes, hydraulic" into "Service brakes"),
groups with fewer than 400 complaints dropped, and recall-campaign texts that quote the component removed.
Split by date: 8,000 training (from older complaints), 1,000 validation, 2,000 test (the newest).
The training data is not distributed with the adapter. NHTSA blanks most personal details; a few narratives still
contain a phone number or VIN. The adapter was trained only to output a component label.
Limits
- US vehicles and NHTSA's component groups only. A manufacturer's own warranty codes need their own training.
- English only, as written by owners.
- For routing and analytics, not for safety decisions or defect determinations. Send low-confidence answers to a person.
- Base model origin: Qwen (Alibaba Cloud). A version on another base model is available on request.
Licence
Adapter: Apache 2.0. Base model: Qwen3-8B under its own licence (Apache 2.0). Data: NHTSA, a US government work.
Built by Gyde (DAI Labs Private Limited, Pune, India).