GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
huncijr
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
huncijr
Model Tree
unsloth/Qwen3-4B-unsloth-bnb-4bit
Input Modalities
Output Modalities
Supported Functionality
The training examples were derived from official eCFR XML snapshots dated 2026-10-08.
The selected training profile contains:
| Split | Examples |
|---|---|
| Training | 8,323 |
| Validation | 394 |
The source collection covers Titles 1–10, but the selected profile is not exhaustive coverage of those titles or of United States federal law.
Targets were derived from source text rather than from unverified model-generated baseline answers. Automated checks included evidence quotations, support hashes, token lengths, and split overlap.
Many questions were generated using extractive templates. Source-text verification does not establish that every question is naturally phrased or that every example has received independent legal review.
The eCFR is a government-maintained, authoritative but unofficial compilation. The source snapshot is historical and may not reflect subsequent regulatory changes.
Source: https://www.ecfr.gov/
| Setting | Value |
|---|---|
| Quantized base weights | 4-bit |
| LoRA rank | 8 |
| LoRA alpha | 16 |
| LoRA dropout | 0 |
| Learning rate | 0.0001 |
| Gradient accumulation | 4 |
| Maximum sequence length | 512 tokens |
| Completed optimizer steps | 2,081 |
| Seed | 3407 |
LoRA target modules:
q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj,
and down_proj.
Training used non-thinking chat formatting with
enable_thinking=False.
The following example follows the Unsloth-based loading approach used during development. It requires a compatible CUDA environment.
Install PyTorch for your CUDA environment and compatible versions of Unsloth, Transformers, PEFT, Accelerate, and BitsAndBytes.
python
import unslothimport torchfrom peft import PeftModelfrom transformers import AutoTokenizerfrom unsloth import FastLanguageModel base_id = "unsloth/Qwen3-4B-unsloth-bnb-4bit"adapter_id = "huncijr/US-federal-regulations-qlora" model, tokenizer = FastLanguageModel.from_pretrained( model_name=base_id, max_seq_length=2048, load_in_4bit=True,) model = PeftModel.from_pretrained( model, adapter_id, is_trainable=False,)tokenizer = AutoTokenizer.from_pretrained(adapter_id) FastLanguageModel.for_inference(model) messages = [ { "role": "user", "content": ( "Who serves as Secretary of the Administrative " "Committee of the Federal Register?" ), }] prompt = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True, enable_thinking=False,)inputs = tokenizer(prompt, return_tensors="pt").to(model.device) with torch.inference_mode(): output = model.generate( **inputs, max_new_tokens=256, do_sample=True, temperature=0.7, top_p=0.8, top_k=20, ) response = output[0, inputs["input_ids"].shape[1]:]print(tokenizer.decode(response, skip_special_tokens=True))This repository's remote-loading example has not yet been verified end to end. Local adapter inference was tested during development.
A diagnostic evaluation tested the adapter in fresh conversations using simple, paraphrased, citation-free, contextual, and multi-part questions.
Responses were qualitatively compared by an AI assistant against local eCFR source references. This was not an independent expert legal evaluation.
| Assessment | Prompts |
|---|---|
| Correct | 4 |
| Partially correct | 3 |
| Incomplete | 2 |
| Incorrect | 15 |
| Total | 24 |
These prompts were selected for diagnosis and included familiar training-domain facts. The results are not a representative accuracy estimate or a held-out generalization benchmark.
Four additional prompts included relevant regulatory excerpts:
The source-assisted setup used a larger context window, so this was an exploratory comparison rather than a controlled retrieval study.
No controlled base-model comparison or untouched final-test evaluation has been completed for this adapter.
Observed failures include:
Fluent wording should not be interpreted as evidence of correctness. Fine-tuning does not guarantee exact recall of the training examples.
This adapter is intended for experimentation, reproducible fine-tuning research, and evaluation of regulation-domain adaptation.
It is not validated as an authoritative legal reference. Verify substantive answers against the applicable source text and date. For practical applications, consider retrieving relevant regulatory passages and checking the generated answer against them.
adapter_model.safetensors: learned LoRA weights.adapter_config.json: adapter and base-model configuration.chat_template.jinja: input formatting.README.md: this model card.Released under Apache License 2.0. See the upstream base-model license and preserve applicable notices.
import unslothimport torchfrom peft import PeftModelfrom transformers import AutoTokenizerfrom unsloth import FastLanguageModel base_id = "unsloth/Qwen3-4B-unsloth-bnb-4bit"adapter_id = "huncijr/US-federal-regulations-qlora" model, tokenizer = FastLanguageModel.from_pretrained( model_name=base_id, max_seq_length=2048, load_in_4bit=True,) model = PeftModel.from_pretrained( model, adapter_id, is_trainable=False,)tokenizer = AutoTokenizer.from_pretrained(adapter_id) FastLanguageModel.for_inference(model) messages = [ { "role": "user", "content": ( "Who serves as Secretary of the Administrative " "Committee of the Federal Register?" ), }] prompt = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True, enable_thinking=False,)inputs = tokenizer(prompt, return_tensors="pt").to(model.device) with torch.inference_mode(): output = model.generate( **inputs, max_new_tokens=256, do_sample=True, temperature=0.7, top_p=0.8, top_k=20, ) response = output[0, inputs["input_ids"].shape[1]:]print(tokenizer.decode(response, skip_special_tokens=True))