Model Description
ETL Error Explainer v2 is a domain-specific instruction-tuned language model designed to analyze ETL and data pipeline failures.
Given an ETL error message and execution context (environment, severity, cloud provider), the model produces a structured JSON response containing:
- category
- root_cause
- immediate_fix
- long_term_fix
The model is intended to assist Data Engineers, Analytics Engineers, Platform Engineers, and DevOps teams during troubleshooting and incident response.
Model Details
Developed by
Fırat Çelik
Model type
Instruction-tuned causal language model
Base model
Qwen/Qwen2.5-1.5B-Instruct
Fine-tuning method
QLoRA (4-bit NF4 Quantization + LoRA)
Language
English
License
Apache-2.0
Model Sources
Hugging Face Repository
https://huggingface.co/firfircelik/etl-error-explainer-v2
Dataset
Synthetic ETL troubleshooting instruction dataset created by the author.
Intended Uses
Direct Use
The model is designed for structured ETL troubleshooting.
Example applications include:
- Spark error explanation
- SQL exception analysis
- Databricks job failures
- Azure Data Factory pipeline failures
- Kafka ingestion errors
- Snowflake query failures
- Schema evolution issues
- Authentication and permission errors
- Data validation failures
Input:
Context:
environment=prod
severity=High
cloud=Azure
Analyze the following Databricks failure:
org.apache.spark.sql.AnalysisException:
cannot resolve 'customer_email'
Output:
{
"category": "...",
"root_cause": "...",
"immediate_fix": "...",
"long_term_fix": "..."
}
Downstream Uses
The model can be integrated into:
- AI Data Engineering copilots
- Internal troubleshooting assistants
- Incident response systems
- RAG applications
- Chatbots
- IDE assistants
- Knowledge base generation
- Support automation
Out-of-Scope Uses
The model is not intended for:
- General-purpose chat
- Code generation
- SQL execution
- Production system diagnosis
- Security analysis
- Legal advice
- Medical advice
Outputs should always be reviewed before applying fixes to production environments.
Bias, Risks and Limitations
The model has been trained specifically for ETL troubleshooting.
Limitations include:
- It does not inspect live systems.
- It may suggest common fixes that are not applicable in every environment.
- It cannot validate infrastructure configurations.
- It may hallucinate missing context if insufficient information is provided.
- It should complement—not replace—human expertise.
Recommendations
Best performance is achieved when prompts include:
- Full error messages
- Stack traces
- Cloud provider
- Environment (dev/test/prod)
- Severity level
- Relevant SQL or Spark snippets
Getting Started
from transformers import AutoTokenizer, AutoModelForCausalLM
model_id = "firfircelik/etl-error-explainer-v2"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)
prompt = """
Context:
environment=prod
severity=High
cloud=Azure
Analyze the following Databricks failure:
org.apache.spark.sql.AnalysisException:
cannot resolve 'customer_email'
"""
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(
**inputs,
max_new_tokens=256
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Training Details
Base Model
Qwen/Qwen2.5-1.5B-Instruct
Fine-Tuning Method
QLoRA
- 4-bit NF4 quantization
- LoRA adapters
- PEFT
LoRA Configuration
Table with columns: Parameter, Value| Parameter | Value |
|---|
| r | 16 |
| alpha | 32 |
| dropout | 0.05 |
| target modules | q_proj, k_proj, v_proj, o_proj |
Optimizer
paged_adamw_8bit
Training Hyperparameters
Table with columns: Parameter, Value| Parameter | Value |
|---|
| Epochs | 3 |
| Batch Size | 4 |
| Gradient Accumulation | 4 |
| Effective Batch Size | 16 |
| Learning Rate | 2e-4 |
| Sequence Length | 512 |
Training Data
The model was trained on a synthetic instruction dataset covering realistic ETL failures.
Covered domains include:
- Apache Spark
- Databricks
- SQL
- Azure Data Factory
- Snowflake
- Kafka
- Delta Lake
- Data validation
- Schema evolution
- Authentication
- File ingestion
- Cloud storage
Each sample contains:
- execution context
- instruction
- structured target JSON
Evaluation
A held-out validation split (10%) was used during training.
Additionally, the model was manually tested using unseen ETL failure scenarios.
Evaluation focused on:
- JSON validity
- Root cause correctness
- Practical remediation quality
- Instruction following
Environmental Impact
Hardware
NVIDIA Tesla T4
Kaggle Notebooks
Training Method
QLoRA (4-bit)
Using QLoRA significantly reduces GPU memory usage and energy consumption compared to full fine-tuning.
Technical Specifications
Architecture
- Transformer Decoder
- Causal Language Modeling
Objective
Instruction-following generation of structured ETL troubleshooting responses.
Output format:
{
"category": "...",
"root_cause": "...",
"immediate_fix": "...",
"long_term_fix": "..."
}
Supported Domains
Table with columns: Domain, Supported| Domain | Supported |
|---|
| Apache Spark | ✅ |
| Databricks | ✅ |
| SQL | ✅ |
| Azure Data Factory | ✅ |
| Kafka | ✅ |
| Snowflake | ✅ |
| Delta Lake | ✅ |
| Schema Evolution | ✅ |
| Authentication | ✅ |
Citation
@misc{celik2026etlexplainer,
author = {Fırat Çelik},
title = {ETL Error Explainer v2},
year = {2026},
publisher = {Hugging Face},
howpublished = {\url{https://huggingface.co/firfircelik/etl-error-explainer-v2}}
}
GitHub
https://github.com/firfircelik
Hugging Face
https://huggingface.co/firfircelik