GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
cobra9786
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
cobra9786
Model Tree
Qwen/Qwen2.5-1.5B-Instruct
Input Modalities
Output Modalities
Supported Functionality
This project adapts an open-weight instruction model for structured classification of engineering and sensor fault reports.
The model takes an unstructured engineering report and returns a strict JSON object matching the project schema:
categoryseveritysubsystemsymptomssuspected_causerecommended_actionrequires_human_reviewThe goal is not to claim production-grade triage accuracy from a small dataset. The goal is to demonstrate a reproducible foundation-model adaptation workflow:
Qwen/Qwen2.5-1.5B-InstructSFTTrainerThe base model is loaded locally and adapted with LoRA adapters rather than full-parameter fine-tuning.
Dataset type: synthetic/sanitized engineering-log examples.
Current dataset version: 0.3.0
Split policy: lora_v1
| Split | Records |
|---|---|
| Train | 48 |
| Validation | 7 |
| Test | 3 |
| Total | 58 |
The dataset includes fault reports across these categories:
sensor_or_signal_pathcommunicationspower_or_batteryfirmware_or_softwarecalibration_or_configurationmechanical_or_environmentaldata_pipeline_or_apiunknownThe validation set was used to select the adapter candidate. The test split was held out until after adapter selection.
The model is evaluated against strict JSON output requirements. A response is considered strictly valid only when:
{;};TriageResult schema;The evaluator also records a recovered JSON validity metric for Markdown-fenced JSON, but strict JSON validity is the preferred API-readiness metric.
The prompt-only baseline used the base Qwen model with the same task prompt and greedy decoding.
On the active 7-record validation split:
| Metric | Qwen Prompt Baseline |
|---|---|
| Strict JSON/schema validity | 0.0000 |
| Recovered JSON/schema validity | 0.8571 |
| Markdown fence recovery used | 0.8571 |
| Category accuracy | 0.4286 |
| Severity accuracy | 0.2857 |
| Requires-human-review accuracy | 0.8571 |
| All measured fields correct | 0.1429 |
| Mean latency seconds | 2.0247 |
The baseline frequently produced Markdown-fenced JSON. That was recoverable for evaluation, but not ideal for an API boundary.
Selected adapter run:
qwen_lora_r8_train48_steps72_v1
Training setup:
| Setting | Value |
|---|---|
| LoRA rank | 8 |
| LoRA alpha | 16 |
| LoRA dropout | 0.05 |
| Training records | 48 |
| Max steps | 72 |
| Batch size per device | 1 |
| Gradient accumulation | 2 |
| Max sequence length | 1024 |
| Precision flags | , |
LoRA target modules:
q_projk_projv_projo_projgate_projup_projdown_projAdapter weights are local training artifacts and are not committed to Git. Evaluation reports are committed separately under artifacts/evaluation/.
Validation split: 7 examples.
| Metric | Qwen Baseline | LoRA Adapter |
|---|---|---|
| Strict JSON/schema validity | 0.0000 | 1.0000 |
| Recovered JSON/schema validity | 0.8571 | 1.0000 |
| Markdown fence recovery used | 0.8571 | 0.0000 |
| Category accuracy | 0.4286 | 1.0000 |
| Severity accuracy | 0.2857 | 0.7143 |
| Requires-human-review accuracy | 0.8571 |
The selected adapter significantly improved strict JSON compliance and validation classification quality over the prompt-only baseline.
Test split: 3 examples.
| Metric | LoRA Adapter |
|---|---|
| Strict JSON/schema validity | 1.0000 |
| Recovered JSON/schema validity | 1.0000 |
| Markdown fence recovery used | 0.0000 |
| Category accuracy | 0.6667 |
| Severity accuracy | 0.3333 |
| Requires-human-review accuracy | 0.6667 |
| All measured fields correct | 0.3333 |
| Mean latency seconds | 3.0377 |
The held-out test result shows that the adapter learned the structured output contract reliably. Semantic classification, especially severity and human-review policy, still needs a larger and more diverse dataset.
Because the test split contains only 3 examples, each incorrect field changes the metric by 33.3 percentage points. These test metrics should be treated as directional evidence, not production accuracy estimates.
The strongest result is strict JSON/schema compliance.
The LoRA adapter moved from a prompt-only baseline that usually required Markdown-fence recovery to a selected adapter that produced raw schema-valid JSON on both validation and test splits.
The adapter also improved validation category accuracy and complete measured-field correctness. However, held-out semantic generalization remains limited by dataset size.
A fair project claim is:
Built a reproducible Hugging Face QLoRA training and evaluation pipeline that improved strict JSON compliance and validation triage accuracy over a prompt-only Qwen baseline on a compact engineering-log dataset.
A fair limitation is:
The current dataset is too small for production accuracy claims. Additional examples are needed, especially for severity calibration, human-review policy, water-ingress cases, ambiguous low-evidence reports, and configuration-versus-software distinctions.
Expand the dataset to at least:
Add more hard examples for:
Add a FastAPI inference service:
POST /triage;Current status: adapter training and evaluation pipeline complete.
Next engineering milestone: FastAPI inference service with typed request/response validation.
fp16=Falsebf16=False| Output overwrite policy | training output directories are protected from accidental overwrite |
| 1.0000 |
| All measured fields correct | 0.1429 | 0.7143 |
| Mean latency seconds | 2.0247 | 3.1859 |
Add Docker-based local deployment.
Publish the dataset and adapter to Hugging Face Hub once the project boundary is stable.