manifesta

scientific-chart-qa-lora-qwen3.5-9b

README

License: apache-2.0

What you need to load it

The exact base model string is:

markdown
Qwen/Qwen3.5-9B

A note on the base model string, read this before you file an issue

adapter_config.json in this repo declares:

json
"base_model_name_or_path": "togethercomputer/Qwen3.5-9B"

That is what Adaption's exporter wrote, and it is left untouched so the export stays verifiable. That repo id does not resolve on the Hugging Face Hub (it returns 404 both anonymously and with a valid token, checked 2026-08-12), so peft cannot auto-resolve the base model from it.

The base is Qwen/Qwen3.5-9B. The evidence is the config.json shipped alongside the adapter in this repo. Flattening both configs and comparing all 66 keys, every architecture-defining value is identical:

Table with columns: key, value
key	value
architectures	`Qwen3_5ForConditionalGeneration`
model_type	`qwen3_5`
hidden_size	4096
num_hidden_layers	32
num_attention_heads	16
num_key_value_heads	4
head_dim	256
intermediate_size	12288
vocab_size	248320
vision_config.depth	27
vision_config.hidden_size	1152

The only differences are serialization and training-time fields (transformers_version, torch_dtype, use_cache, explicit token ids). The LoRA tensor shapes agree with this too: lora_A is [16, 4096] against hidden_size 4096, and q_proj.lora_B is [8192, 16], which is 16 heads times 256 head_dim times the 2x width of this model's gated attention output.

So: pass Qwen/Qwen3.5-9B explicitly when you load, and everything lines up.

How to load it

python
import torch
from transformers import AutoModelForImageTextToText, AutoProcessor
from peft import PeftModel

BASE = "Qwen/Qwen3.5-9B"
ADAPTER = "manifesta/scientific-chart-qa-lora-qwen3.5-9b"

model = AutoModelForImageTextToText.from_pretrained(
    BASE,
    dtype=torch.bfloat16,
    device_map="auto",
)

# The adapter_config points at a base repo id that does not resolve, so pass the
# already-loaded base model object. Do NOT use AutoPeftModel here, it will try to
# resolve "togethercomputer/Qwen3.5-9B" and fail.
model = PeftModel.from_pretrained(model, ADAPTER)
model.eval()

# The processor is included in this repo, so you can load it from either place.
processor = AutoProcessor.from_pretrained(ADAPTER)

messages = [{
    "role": "user",
    "content": [
        {"type": "image", "url": "https://upload.wikimedia.org/wikipedia/commons/2/26/Line_graph_example.png"},
        {"type": "text", "text": "What is the value on the y axis at x = 3?"},
    ],
}]

inputs = processor.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

out = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

If you want a single merged checkpoint instead of runtime adapter application:

python
merged = model.merge_and_unload()
merged.save_pretrained("./chart-qa-qwen3.5-9b-merged")

Merging bakes the adapter into a full-size copy of the base weights. That copy is not distributed here, you produce it yourself.

Tested against peft 0.15.1, which is the version that wrote this adapter. transformers must be new enough to know the qwen3_5 architecture (the export was written by transformers 5.13.0).

LoRA configuration

Taken verbatim from adapter_config.json.

Table with columns: setting, value
setting	value
`peft_type`	LORA
`task_type`	CAUSAL_LM
`r`	16
`lora_alpha`	32 (scaling factor 2.0)
`lora_dropout`	0.0
`target_modules`	, , ,

What actually got adapted

Reading the tensor index of adapter_model.safetensors rather than trusting the config prose:

64 tensors, all float32, 3,932,160 parameters total.
Every tensor is under base_model.model.model.language_model.layers.*.self_attn.*.
Only 8 of the 32 decoder layers carry LoRA: layers 3, 7, 11, 15, 19, 23, 27, 31. Those are exactly the full_attention layers. This model interleaves linear attention and full attention with full_attention_interval 4, and the linear attention layers do not expose q_proj/k_proj/v_proj/o_proj, so they were skipped.
Zero vision tensors. The vision tower is completely untouched. adapter_config.json carries a long exclude_modules list covering all 27 model.visual.blocks.*. The chart reading behaviour that changed is language-side behaviour conditioned on vision features, not a retrained vision encoder.

Practical consequence: only 8 attention blocks out of 32 layers were adapted, on a 9B base. This is a small intervention, which is consistent with the small measured effect described below.

Training run

Facts below come from trainer_state.json, included in this repo so you can check the curve yourself.

Table with columns: field, value
field	value
training data	Adaption dataset `46b5c093-a9e5-45a0-9265-412b003c1d28` (`scientific_chart_qa`), 6,976 rows ingested
`num_train_epochs`	1
`max_steps` / `global_step`	21 / 21 (run completed)
`train_batch_size`	1
`logging_steps`	1

Loss

Training loss, step 1 to step 21:

markdown
1.5476  1.4797  1.4729  1.5156  1.4297  1.5247  1.5907  1.5457  1.5095  1.3912  1.5430
0.7522  1.4543  1.5291  1.3928  1.5344  1.5432  1.4104  1.3699  1.4127  1.5413

Evaluation loss, every 4 steps:

Table with columns: step, eval_loss
step	eval_loss
5	1.4545
9	1.4319
13	1.4124
17	1.3987
21	1.3909

Eval loss fell monotonically from 1.4545 to 1.3909 across the run, about 4.4 percent. Training loss did not visibly trend, it stayed in a 1.37 to 1.59 band with one outlier at step 12 (0.7522, a single batch at batch size 1, so it is one example and not a trend). With train_batch_size 1 and 21 steps the per-step training loss is essentially a per-example reading and is very noisy. The eval curve is the one worth reading.

21 optimizer steps at batch size 1 is 21 examples of gradient signal against a 6,976 row dataset. This is a short run. Read the results section with that in mind.

Results, stated plainly

Adaption's head to head evaluation compared the adapted model against the unmodified base model:

Table with columns: wins
	wins
adapted (this adapter)	51
base `Qwen3.5-9B`	49

51 versus 49 out of 100. That is a two point margin. On 100 paired comparisons the one-sigma spread from coin flipping alone is about 5 points, so this result is well inside noise and should be read as no measurable win, not as a win. The eval loss improvement (1.4545 to 1.3909) is real and monotonic, but it did not convert into a preference margin that survives its own error bars.

I am not going to dress this up. What this adapter demonstrates is a clean, completed, fully documented adaptation run with a downward eval curve. It is not a demonstration of a large capability gain on chart QA.

The other two runs in this batch, for context

Three AutoScientist runs were completed. Only this one had its weights exported, so only this one is published. The other two are listed here for honesty about the full picture, not as a claim of anything. No weights are published for them, and no repository exists for them. Do not go looking for one.

Table with columns: run, dataset, steps, peak grad norm, win rate (adapted vs base)
run	dataset	steps	peak grad norm	win rate (adapted vs base)
this adapter, `Qwen3.5-9B` chart QA	`46b5c093-a9e5-45a0-9265-412b003c1d28`, 6,976 rows	21	4.44	51 vs 49
`gemma-3-27b-it` VLM chart QA 17k	`3f347c8b-5724-4f96-9417-251623d8aaa5`, 17,070 rows	34	0.79	50 vs 50, a dead tie

The math and code run made the model worse. A peak gradient norm of 744.9 against 0.79 and 4.44 on the two chart runs is a roughly three orders of magnitude difference and is the obvious thing to look at first when explaining why. That run is reported here because suppressing a negative result would make the two positive-ish numbers meaningless.

Intended use and limits

Intended use: research on parameter efficient adaptation for chart and scientific figure question answering, and as a reproducible artifact for the AutoScientist Part 2 submission. Load it, read trainer_state.json, check the numbers above against it.

Limits, all of them real:

The measured preference win is inside noise. Do not deploy this expecting better chart reading than plain Qwen/Qwen3.5-9B.
21 optimizer steps at batch size 1. Under-trained by any normal standard.
Vision tower untouched, so nothing here improves figure perception itself.
Only 8 of 32 layers adapted, attention projections only, no MLP.
English only. Training data is English scientific chart QA.
Inherits every limitation, bias and failure mode of Qwen/Qwen3.5-9B. Chart QA models in general will confidently misread axis values, mislabel series and invent gridline numbers. Nothing in this adapter fixes that, and a 51 to 49 preference margin certainly does not.
No safety tuning of any kind was performed.

Files in this repository

Table with columns: file, what it is
file	what it is
`adapter_model.safetensors`	the LoRA weights, 15.7 MB, 64 tensors, fp32
`adapter_config.json`	peft config, unedited from the export, including the unresolvable base repo id
`trainer_state.json`	the full training log, every step, every eval, the evidence for the loss table above
`config.json`	base architecture config as recorded at export time, the evidence for the base model identification
`tokenizer.json`, `tokenizer_config.json`,

Training data

manifesta/scientific-chart-qa-17k on the Hugging Face Hub, and the same dataset on Kaggle. The Adaption side ingested 6,976 rows into dataset 46b5c093-a9e5-45a0-9265-412b003c1d28 for this specific run.

License

The adapter weights are released under Apache 2.0. The base model Qwen/Qwen3.5-9B carries its own license, which applies to you separately whenever you load the base, and it is your responsibility to comply with it.

Citation

bibtex
@misc{navardauskas2026chartqalora,
  title  = {scientific-chart-qa-lora-qwen3.5-9b: a LoRA adapter for scientific chart question answering},
  author = {Navardauskas, Aivaras},
  year   = {2026},
  note   = {Adaption Labs AutoScientist Part 2. Trained on Adaption dataset 46b5c093-a9e5-45a0-9265-412b003c1d28.},
  url    = {https://huggingface.co/manifesta/scientific-chart-qa-lora-qwen3.5-9b}
}

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider