qwen2.5-1.5b-buildeng-lora

Qwen2.5-1.5B BuildEng LoRA

Repository: Irfanuruchi/qwen2.5-1.5b-buildeng-lora

This model is a civil/building engineering focused LoRA fine-tune of Qwen2.5-1.5B-Instruct.

BuildEng 1.5B was built to move more toward structural reasoning and construction decision behavior. Earlier building-engineering models were more focused on direct calculations, HVAC checks, formula correction, and unit sanity. This version moves much more into structural and site reasoning.

The main goal was making the model more careful with engineering prompts, especially when the input is incomplete or unsafe.

What I Used

Base model:

text
Copy code
Qwen/Qwen2.5-1.5B-Instruct

Training method:

text
Copy code
QLoRA fine-tuning with Unsloth

Dataset used:

text
Copy code
Irfanuruchi/buildeng-v8-1.5b

Local base model used during training:

text
Copy code
models/Qwen2.5-1.5B-Instruct

Training output:

text
Copy code
qwen2.5-1.5b-buildeng-v8-lora

Training stack:

text
Copy code
Unsloth
Transformers
TRL SFTTrainer
bitsandbytes 4-bit QLoRA
LoRA r=16
LoRA alpha=32
Sequence length 2048

Release Roadmap

BuildEng 1.5B is the first public release of the BuildEng project.

Planned releases on June 4 include:

text
Copy code
BuildEng 1.5B
BuildEng 3B
BuildEng 32B preview

The larger models focus on stronger routing consistency, deeper structural reasoning, longer engineering workflows, and more advanced construction and structural behavior.

Main Focus Of BuildEng 1.5B

BuildEng 1.5B focuses on conservative civil/building engineering reasoning.

The model was trained to handle structural and construction-related situations such as beams, columns, slabs, retaining walls, shallow foundations, settlement, masonry, structural connections, temporary works, excavation safety, lateral stability, concrete cracking, waterproofing problems, roof systems, construction sequencing, and renovation cases with missing drawings or uncertain existing conditions.

One of the biggest goals during development was reducing confident wrong answers.

Earlier outputs could still:

text
Copy code
mix beam behavior with column behavior
answer retaining wall drainage as a generic field change
approve from sketches or weak input
accept contractor or client pressure
ignore temporary construction-stage risk
guess when drawings or site information were missing

BuildEng 1.5B adds repair-focused and routing-focused data around those exact problems.

Engineering Areas

The model was trained mainly around simplified civil/building engineering reasoning tasks including structural member behavior, load path reasoning, support conditions, bending and shear checks, column slenderness and buckling, retaining wall lateral pressure, sliding and overturning, drainage and hydrostatic pressure, foundation bearing and settlement, masonry openings, construction sequence, temporary shoring, temporary bracing, excavation safety, roof loading, waterproofing and durability, structural connections, cracking behavior, and inspection-first decision making.

Example Behavior

Example:

text
Copy code
User:
Can I approve a 3 m retaining wall from a sketch?

Assistant:
No. A 3 m retaining wall cannot be approved from a sketch alone. Retaining wall checks include lateral earth pressure, sliding, overturning, bearing pressure, drainage, surcharge loading, wall geometry, reinforcement, and foundation conditions.

Another example:

text
Copy code
User:
Can I check this beam only with axial stress?

Assistant:
No. For a beam, axial stress alone is not the correct safety check. Beam design must also consider bending moment, shear force, span, support condition, load pattern, stiffness, and deflection.

Another example:

text
Copy code
User:
The contractor removed props before slab strength. Is it okay because temporary?

Assistant:
No. Temporary conditions can be more dangerous than the final structure due to instability, excessive deflection, overload, progressive collapse risk, or unsafe load redistribution.

Limitations

This model is intended for research and engineering-assistant workflows only. Final engineering approval and construction sign-off should always be performed by a licensed engineer.

Future Improvements

BuildEng 1.5B is the small-model branch of the BuildEng project.

Future work includes BuildEng 3B and a larger BuildEng 32B model with stronger reasoning depth, more calculations, better structural routing, longer engineering workflows, and larger training data.

Training Notes

One important part during BuildEng 1.5B development was using failed model outputs as feedback for future dataset generation.

When the model repeatedly failed certain engineering cases, additional repair-focused synthetic samples were added around those exact weak areas.

This made the tuning process iterative instead of only generating random engineering calculations.

Usage

This repository contains a LoRA adapter, not full standalone model weights.

Base model required:

text
Copy code
Qwen/Qwen2.5-1.5B-Instruct

Example loading:

python
Copy code
from unsloth import FastLanguageModel

base_model = "Qwen/Qwen2.5-1.5B-Instruct"
adapter = "Irfanuruchi/qwen2.5-1.5b-buildeng-lora"

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name=base_model,
    max_seq_length=2048,
    dtype=None,
    load_in_4bit=True,
)

model.load_adapter(adapter)
FastLanguageModel.for_inference(model)

Author

Irfan Uruchi

Part of my ongoing work on domain-specialized engineering language models and lightweight structural reasoning systems.