lyrain2001

Auto-Fill-Qwen3-8B-Reasoning

README

License: apache-2.0

Model details

Table

Base model	Qwen/Qwen3-8B
Training	full-parameter SFT with ms-swift, DeepSpeed ZeRO-3, bf16
Data	59,367 examples: chain-of-thought traces distilled from DeepSeek-R1 (`<think>…</think>` + answer). Only traces with a correct answer are kept; the confidence label comes from a two-stage procedure described in the paper (§ Training Specialist Models).
Hyper-parameters	2 epochs, lr 2e-5, cosine schedule, weight decay 0.1, max length 40,960, effective batch size 16
Hardware	4× A100 80 GB
Confidence signal	the verbalized integer confidence (0–100) in the answer JSON, scaled to [0, 1]. Calibrated with isotonic regression fitted on a held-out validation split (`calibrators.json`).
Decoding used in the paper	temperature 0.8, max new tokens 32768, default Qwen3 chat template

Training tables come from public sources only (spreadsheets crawled from a search-engine index, public BI models, Wikipedia, nationalarchives.gov.uk, GitHub CSV/Parquet files); one cell per table is masked and its original value is the target.

Prompt and output format

The table is serialized as a Markdown pipe table (pandas.DataFrame.to_markdown(index=False, tablefmt="pipe")) with the cell to fill written as [MISSING]. The user message is exactly (see autofill/utils/prompts.py):

markdown
Please fill in the missing value in the input table and provide your confidence level as an integer between 0 (no confidence) and 100 (full confidence). The missing value is denoted by '[MISSING]'. Please return the value filled in JSON format: {"value": "filled_value", "confidence": confidence_level}.

Input Table:
<markdown table>

Expected output: <think> … </think> followed by {"value": "<filled value>", "confidence": <0-100>}.

Usage

With the code repository (recommended) — runs the full ensemble on one table:

bash
python inference/run_specialists.py \
    --table          /path/to/table.csv \
    --knowledge_path lyrain2001/Auto-Fill-Qwen3-8B-Knowledge \
    --reasoning_path lyrain2001/Auto-Fill-Qwen3-8B-Reasoning \
    --coding_path    lyrain2001/Auto-Fill-Qwen3-8B-Coding \
    --calibrators    checkpoints/calibrators.json \
    --gpu_ids        0,1,2

or this specialist alone on the benchmark:

bash
python inference/run_benchmark.py --mode reasoning --model_path lyrain2001/Auto-Fill-Qwen3-8B-Reasoning \
    --dataset Gov-CSV --benchmark Auto-Fill-Benchmark/sample200 --gpu_ids 0

Minimal vLLM example

python
import pandas as pd
from vllm import LLM, SamplingParams

llm = LLM(model="lyrain2001/Auto-Fill-Qwen3-8B-Reasoning", dtype="bfloat16", max_model_len=40960)
table = pd.read_csv("table.csv", dtype=str).to_markdown(index=False, tablefmt="pipe", disable_numparse=True)
prompt = PROMPT + table   # PROMPT = the user message above, up to and including "Input Table:\n"
text = llm.get_tokenizer().apply_chat_template(
    [{"role": "user", "content": prompt}], tokenize=False, add_generation_prompt=True)
out = llm.generate([text], SamplingParams(temperature=0.8, max_tokens=32768))
print(out[0].outputs[0].text)

Results

Recall@Precision=0.9 on the Auto-Fill benchmark (200 cases per dataset; from the paper's specialist ablation):

Table with columns: Pub-XLS, Pub-BI, Pub-Wiki, Gov-CSV, Git-Parquet, Ent-CSV*, Ent-XLS*, Pub-Web, Rel-AR, Rel-FD, Rel-ST, Mean
	Pub-XLS	Pub-BI	Pub-Wiki	Gov-CSV	Git-Parquet	Ent-CSV*	Ent-XLS*	Pub-Web	Rel-AR	Rel-FD	Rel-ST	Mean
Reasoning specialist alone	0.360	0.000	0.000	0.455	0.475	0.115

* Ent-CSV / Ent-XLS are proprietary enterprise datasets that are not part of the public benchmark.

Limitations

Trained and evaluated on English-language tables with one missing cell per table; tables were serialized with at most 40,960 tokens.
The model can be wrong with high confidence on cells that require knowledge outside the table; use the calibrated confidence and abstain below a threshold, as in the paper.
Generated code (coding specialist) should be executed in a sandbox.

Citation

bibtex
@article{liu2026autofill,
  title={Auto-Fill: Learning to Predict Missing Values Accurately with Specialist Language Models},
  author={Liu, Yurong and He, Yeye and Dong, Haoyu and Xing, Junjie and Han, Shi and Zhang, Dongmei and Chaudhuri, Surajit},
  journal={Proceedings of the VLDB Endowment},
  volume={19},
  number={11},
  pages={3160--3173},
  year={2026}
}

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

lyrain2001

Model Tree

Base

Qwen/Qwen3-8B

Fine-tuned

this model

Input Modalities

Text

Output Modalities