lhpku20010120

qwen3.5-2b-k12kgraph

README

License: apache-2.0

Model details

Architecture: Qwen3_5ForConditionalGeneration
Parameters: approximately 2B
Fine-tuning method: LoRA SFT, merged into the base model
LoRA rank: 8
LoRA alpha: 16
LoRA dropout: 0.05
Training framework: LLaMA-Factory
Context length used for preprocessing: 4,096 tokens
Model weight dtype: bfloat16
Languages: Chinese and English
License: Apache License 2.0

Intended use

The model is intended for research on K-12 educational question answering, knowledge-intensive reasoning, and evaluation of models trained with knowledge-graph-derived educational data.

Usage

python
import torch
from transformers import AutoModelForImageTextToText, AutoProcessor

model_id = "lhpku20010120/qwen3.5-2b-k12kgraph"

processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    device_map="auto",
)

messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "text",
                "text": "Explain the difference between the light-dependent and light-independent reactions in photosynthesis.",
            }
        ],
    }
]

inputs = processor.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

with torch.inference_mode():
    outputs = model.generate(
        **inputs,
        max_new_tokens=512,
        do_sample=False,
    )

generated_ids = outputs[:, inputs["input_ids"].shape[-1]:]
answer = processor.batch_decode(
    generated_ids,
    skip_special_tokens=True,
)[0]
print(answer)

The model uses the included chat_template.jinja, which follows the Qwen3.5 multimodal chat format.

Training

The following information was recovered from the exported trainer metadata:

Table with columns: Setting, Value
Setting	Value
Base model	Qwen/Qwen3.5-2B-Base
Epochs	3
Per-device batch size	4
Gradient accumulation	2
Number of devices	8
Effective global batch size	64
Learning rate	1e-4
Scheduler	cosine
Warmup ratio	0.1

Limitations

The model may reproduce errors and biases present in the base model or fine-tuning data.
Generated answers may be incorrect, incomplete, or hallucinated. The model should not be treated as an authoritative source for high-stakes educational decisions.
Its strongest expected domain is K-12 educational content; performance may degrade outside that domain.

Data and copyright

Users are responsible for complying with the licenses and terms associated with the base model, training data, benchmark data, and generated content. The release of model weights does not grant rights to reproduce any third-party textbook or benchmark material.

Acknowledgements

This model is based on Qwen3.5-2B-Base and was trained with LoRA using LLaMA-Factory, then merged into the base model weights for release. Please cite the Qwen3.5 and K12-KGraph projects when using this model in research.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

lhpku20010120

Model Tree

Base

Qwen/Qwen3.5-2B-Base

Fine-tuned

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality