ntphuc149

ViLegalQwen3-1.7B-Base

README

License: apache-2.0

How to Use

python

from transformers import AutoModelForCausalLM, AutoTokenizer tokenizer = AutoTokenizer.from_pretrained("ntphuc149/ViLegalQwen3-1.7B-Base")model = AutoModelForCausalLM.from_pretrained("ntphuc149/ViLegalQwen3-1.7B-Base")

Note: This is a base (pretrained) model, not an instruction-tuned model. We do not recommend using base language models for conversations. Instead, you can apply post-training, e.g., SFT, RLHF, continued pretraining, etc., on this model.

Model Summary

Table with columns: Attribute, Value
Attribute	Value
Architecture	Qwen3 (decoder-only, causal LM)
Parameters	1.72B
Base model	Qwen3-1.7B-Base
Max sequence length	4096 tokens
Tokenizer	Qwen3 tokenizer
Training objective	Causal Language Modeling (CLM)
Training domain	Vietnamese legal text
Precision	FP32

Evaluation Results

ViLegalQwen3-1.7B-Base achieves state-of-the-art results among 1.7B-scale models across all evaluated Vietnamese legal benchmarks. Bold = best in the 1.7B parameter group. Italic = closed-source model scores.

True/False — ALQAC-TF

Table with columns: Model, Pre, Rec, F1
Model	Pre	Rec	F1
Qwen3-1.7B-Base	90.27	87.89	89.07
qwen3-1.7b-legal-pretrain	89.62	86.32	87.94
ViLegalQwen3-1.7B-Base	90.62	91.58	91.10
gpt-4o-mini (0-shot)	89.86	97.89

Multiple Choice — ALQAC-MCQ & VLSP-MCQ-LK

Table with columns: Model, Pre_mac, Rec_mac, F1_mac
Model	Pre_mac	Rec_mac	F1_mac
ALQAC-MCQ
Qwen3-1.7B-Base	85.64	84.68	85.03
qwen3-1.7b-legal-pretrain	87.96	88.19	88.00
ViLegalQwen3-1.7B-Base	89.22	88.81

Abstractive QA — ViBidLQA-AQA

Table with columns: Model, ROUGE-L, BLEU-4, BS-F1
Model	ROUGE-L	BLEU-4	BS-F1
Qwen3-1.7B-Base	73.81	51.18	90.24
qwen3-1.7b-legal-pretrain	74.84	51.49	90.32
ViLegalQwen3-1.7B-Base	74.49	52.11	90.43
gpt-4o-mini (0-shot)	67.06	40.46

NLI — VLSP-NLI

Table with columns: Model, Precision, Recall, F1
Model	Precision	Recall	F1
Qwen3-1.7B-Base	94.00	97.33	95.64
qwen3-1.7b-legal-pretrain	97.44	97.22	97.24
ViLegalQwen3-1.7B-Base	95.75	100.00	97.83
gpt-4o-mini (0-shot)	100.00	86.67

Syllogism Reasoning — VLSP-Syllogism

Table with columns: Model, BS-F1, LLM-Judge
Model	BS-F1	LLM-Judge
Qwen3-1.7B-Base	76.69	0.2760
qwen3-1.7b-legal-pretrain	76.80	0.2899
ViLegalQwen3-1.7B-Base	77.50	0.3038
gpt-4o-mini (0-shot)	78.63	0.5069

Also in ViLegalLM

Table with columns: Model, Architecture, Params, Context
Model	Architecture	Params	Context
ViLegalBERT	Encoder-only	135M	256
ViLegalQwen2.5-1.5B-Base	Decoder-only	1.54B	2,048
ViLegalQwen3-1.7B-Base (this model)	Decoder-only	1.72B	4,096

Limitations and Biases

Domain scope: Trained exclusively on Vietnamese legal texts; may not generalize to other legal systems or jurisdictions.
Base model only: Not instruction-tuned; outputs may be incomplete or incoherent without task-specific fine-tuning.
Temporal bias: Legal corpus reflects Vietnamese law as of the collection date; model outputs may not reflect recent legislative changes.
Inherited biases: May reflect biases present in the source legal corpora, including regional variations in legal practice and domain coverage imbalances.
Not a legal authority: Model outputs should never be used as definitive legal interpretations without expert validation.

Intended Use

Intended for:

Further fine-tuning on Vietnamese legal downstream tasks (QA, NLI, reasoning)
Vietnamese legal text generation and completion
Research on Vietnamese legal NLP and continual pretraining

Not intended for:

Direct conversational or instruction-following use without fine-tuning
Replacing professional legal counsel or human judgment in legal decision-making
Providing legal advice without expert validation
Legal systems outside Vietnam without careful domain adaptation

Citation

If you use ViLegalQwen3-1.7B-Base, please cite our paper:

bibtex

@inproceedings{nguyen-etal-2026-vilegallm,    title = "{V}i{L}egal{LM}: Language Models for {V}ietnamese Legal Text",    author = "Nguyen, Truong-Phuc  and      Nguyen, Quy-Nhan  and      Nguyen, Minh-Tien",    editor = "Liakata, Maria  and      Moreira, Viviane P.  and      Zhang, Jiajun  and      Jurgens, David",    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",    month = jul,    year = "2026",    address = "San Diego, California, United States",    publisher = "Association for Computational Linguistics",    url = "https://aclanthology.org/2026.findings-acl.1801/",    pages = "36136--36150",    ISBN = "979-8-89176-395-1",    abstract = "We present **ViLegalLM**, comprising **ViLegalBERT** and **ViLegalQwen**, the first suite of Vietnamese pretrained language models for legal text understanding and generation. It includes one encoder-only model (ViLegalBERT, 135M parameters) and two decoder-only models (ViLegalQwen2.5-1.5B-Base and ViLegalQwen3-1.7B-Base), all continually pretrained on a newly curated 16GB Vietnamese legal corpus, significantly larger than previous work. To mitigate data scarcity, we construct three synthetic datasets using LLM-based generation and hard negative mining for True/False QA, Multiple Choice QA, and Natural Language Inference. We establish state-of-the-art results among open-source models on four main Vietnamese legal downstream tasks spanning ten benchmarks, demonstrating that continual pretraining from base models consistently outperforms instruction-tuned adaptation. Source codes, corpus, datasets, and model checkpoints are publicly available at https://github.com/ntphuc149/ViLegalLM."}

License

Apache-2.0

This model is derived from Qwen3-1.7B-Base which is licensed under Apache-2.0. ViLegalQwen3-1.7B-Base is released under the same Apache-2.0 license.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

ntphuc149

Model Tree

Base

Qwen/Qwen3-1.7B-Base

Fine-tuned

this model

Input Modalities

Text

Output Modalities