AdarshSingh7647

TabRankSingleTableCoTGen

README

License: apache-2.0

What this variant does

The model is trained to first write a chain of thought inside a think block and then output the ranking. It generates its own reasoning at inference time. Use this variant when you want an explanation alongside the ranking.

Training data

This model is trained on the NQ Tables training split only which contains single table retrieval questions. The sibling checkpoint TabRankMultiTableCoTGen uses the other training mix with the same objective.

Input and output format

The user message lists candidate tables as blocks headed by ### Table 1 ### Table 2 and so on. The model returns a JSON object whose value is the ranked list of one based candidate positions best first:

json
{"ranked_tables": [3, 1, 5, 2, 4]}

Map those positions back to your table ids to obtain the reranked list.

Evaluation

Scored as a listwise reranker that reorders a first stage top 25 candidate list on four table question answering benchmarks. SQA and TAT QA use the full test split. HybridQA and TabFact use a fixed shared 500 query sample. acc@10 counts a query correct only when every gold table falls inside the top 10.

Table with columns: Dataset, n, recall@10, ndcg@10, acc@10, MRR
Dataset	n	recall@10	ndcg@10	acc@10	MRR
SQA	148	0.864	0.724	0.791	0.698
TAT QA	362	0.634	0.502	0.398	0.581
HybridQA	500	0.842	0.735	0.738	0.771
TabFact	500	0.716	0.639	0.522

Usage with vLLM

python
from vllm import LLM, SamplingParams
from transformers import AutoTokenizer

repo = "AdarshSingh7647/TabRankSingleTableCoTGen"
tok = AutoTokenizer.from_pretrained(repo)
llm = LLM(model=repo, dtype="bfloat16", max_model_len=32768)

system = ("You are a table relevance expert. Given a question and a set of candidate tables "
          "rank them from most to least useful for answering the question. Reason in a "
          "<think>...</think> block then output exactly JSON with key ranked_tables.")
user = "Question: ...\n\n### Table 1\n...\n\n### Table 2\n...\n"

msgs = [{"role": "system", "content": system}, {"role": "user", "content": user}]
text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
# the model writes a <think> block then the ranking json
out = llm.generate([text], SamplingParams(temperature=0.6, top_p=0.95, max_tokens=8192))
print(out[0].outputs[0].text)

Usage with Transformers

python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "AdarshSingh7647/TabRankSingleTableCoTGen"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype=torch.bfloat16, device_map="auto")

system = ("You are a table relevance expert. Given a question and a set of candidate tables "
          "rank them from most to least useful for answering the question. Reason in a "
          "<think>...</think> block then output exactly JSON with key ranked_tables.")
user = "Question: ...\n\n### Table 1\n...\n\n### Table 2\n...\n"

msgs = [{"role": "system", "content": system}, {"role": "user", "content": user}]
text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
inputs = tok(text, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=8192, temperature=0.6, top_p=0.95, do_sample=True)
print(tok.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

Model details

Base model Qwen3 8B
Method LoRA rank 16 fine tuning merged into the base weights so it loads directly
Precision bfloat16 single file safetensors near 16 GB
Family the six TabRank checkpoints span three objectives (Answer Only, Reasoning Generation, Reasoning Conditioned) across two training mixes (Single Table, Single plus Multi Table)

Citation

The MultiTabQA data comes from RAG over Tables. Please cite it when using the Single plus Multi Table checkpoints:

bibtex
@misc{zou2025ragtableshierarchicalmemory,
      title={RAG over Tables: Hierarchical Memory Index, Multi-Stage Retrieval, and Benchmarking},
      author={Jiaru Zou and Dongqi Fu and Sirui Chen and Xinrui He and Zihao Li and Yada Zhu and Jiawei Han and Jingrui He},
      year={2025},
      eprint={2504.01346},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2504.01346}
}

The accompanying TabRank paper is currently under review. A citation will be added here once it is available on arXiv.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

AdarshSingh7647

Model Tree

Base

Qwen/Qwen3-8B

Fine-tuned

this model

Input Modalities

Text

Output Modalities