GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
abhishek1995s
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
abhishek1995s
Model Tree
Qwen/Qwen3-8B
Input Modalities
Output Modalities
Supported Functionality
Given a question about RBI/NBFC lending regulation, this model — paired with a retrieval-confidence gate (see Gate Architecture) — either:
Intended use: portfolio/technical demonstration of an end-to-end fine-tuning + retrieval + safety-gating pipeline; a starting point for further work (per-class threshold tuning, larger/better-balanced training data, human-in-the-loop review).
Out-of-scope use: real compliance decision-making without human verification; any use where a fabricated citation could cause financial, legal, or regulatory harm; general-purpose assistant use (this is narrowly trained on RBI/NBFC lending regulation only).
Two separate validation passes, kept distinct because they measure different things:
The gate's two embedding-based thresholds and the nearest-neighbor bank's floor were calibrated against this set. Its "false-block" figure below should be read with one caveat: the nearest-neighbor bank was built from real failures observed within this same set, so its 0.1% figure reflects "known cases fixed," not pure generalization — the independent 100-question sample below is the fairer read on generalization.
| Metric | Deployed (3-signal) gate |
|---|---|
| False-proceed rate on decline/off-topic queries (n=78) | 7.7% (6/78) |
| False-block rate on legitimate queries (n=1,114) | 0.1% (1/1,114) |
| Overall accuracy | 99.4% |
100 fresh questions — none overlapping the calibration set or the nearest-neighbor bank — run against the live respond() endpoint. Full question/answer log: test_100_results.txt in the Space repo.
| Category | Result |
|---|---|
| In-scope questions correctly answered | 77/80 |
| In-scope questions incorrectly blocked | 3/80 (3.75%) |
| Decline/off-topic questions correctly blocked | 20/20 (0% false-proceed) |
| Generation degeneration / garbled citations | 0/100 |
The 3 false-blocks are a known, documented, open issue — see Limitations.
q/k/v/o_proj)decline/web_search examples (6x) relative to rag_local examples, after an earlier version showed the model rarely learned when not to answer.Fine-tuning alone gets a model to reasonable answers; getting it to run reliably in production surfaced a separate set of infrastructure bugs, each diagnosed from live failures (not guessed) and fixed in order:
device_map="auto" invoked a multi-threaded parallel tensor-materialization loader whose peak memory ran well above the model's final resident size. Fixed by loading onto CPU first, then moving to CUDA in one explicit call.repetition_penalty down and removing no_repeat_ngram_size.transformers source: no_repeat_ngram_size blocks n-grams across the entire prompt, not just newly-generated text — and the retrieved-chunk context repeats the same document-title phrase up to 19 times, so the model was blocked from correctly reproducing it in the citation. Fixed by removing that parameter.Because fine-tuning alone doesn't reliably teach a model when to decline (see Limitations), a deterministic gate sits in front of this model — it only generates an answer if at least one of three signals clears its bar.
Signals 2 and 3 combine via OR logic (either passing is enough); signal 1 overrides both.
Honest note on how signal 3 was built: an earlier attempt used a large (2,880-question) generically-generated question bank as this second signal — it was rejected before deployment because it regressed the false-proceed rate on risky queries from 7.7% to 67.9% (generic "sounds like a compliance question" similarity doesn't distinguish in-scope topics from out-of-scope-but-similarly-phrased ones). The deployed version instead builds its bank from real observed gate failures: 263 confirmed cases the regulation-text gate incorrectly blocked, and 47 confirmed cases the rejected generic bank incorrectly let through — each expanded with 2 paraphrase variants. Its floor threshold was calibrated via Youden's J, and — because a bank built from known failures can trivially "pass" a test built from the same failures — it was verified on 5 held-out queries that appear in neither bank before being deployed.
Infrastructure-level issues (fixed this session, 0/100 in live validation): dtype mismatches, OOM, generation degeneration, citation-text corruption. See Production hardening above.
Gate-level issue (open, not yet fixed): the nearest-neighbor bank occasionally false-blocks legitimate questions that are lexically adjacent to an out-of-scope acronym present in its negative bank — e.g. "What is the formula for CRAR?" gets pulled toward a negative-bank example about CRR (Cash Reserve Ratio, a different and out-of-scope ratio), because the positive bank has no CRAR-specific anchor (CRAR questions usually already passed the regulation-text gate before this fix, so they were never captured as "rescue" examples). Same pattern for LCR/SLR. Measured impact: 3/80 (3.75%) false-block rate in the 100-question live test, down from 23.7% before this fix, but not zero.
Model-level issues (predate this session, not addressed by infrastructure fixes — these are about factual/citation reliability, not text corruption):
Source: ... format; the rest embed the citation naturally in prose without the literal prefix.Given these findings, do not use this model's raw output for actual compliance decisions. It is deployed paired with a gate specifically because the model's own judgment about when to answer was found to be unreliable.
python
from transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModelimport torchbase_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-8B", torch_dtype=torch.bfloat16)base_model = base_model.to("cuda")model = PeftModel.from_pretrained(base_model, "abhishek1995s/indian-nbfc-regulatory-assistant-v3")model = model.to(torch.bfloat16) # PEFT loads adapters in fp32 regardless of base dtype -- see Limitationstokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-8B")messages = [{"role": "user", "content": "Your RBI/NBFC lending question here"}]text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=False)inputs = tokenizer(text, return_tensors="pt").to(model.device)outputs = model.generate(**inputs,max_new_tokens=2000,repetition_penalty=1.15,do_sample=False,eos_token_id=tokenizer.eos_token_id,pad_token_id=tokenizer.pad_token_id or tokenizer.eos_token_id,)print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
Strongly recommended: pair with the retrieval-confidence gate (see Gate Architecture) rather than using this adapter's raw output directly — the base model's own judgment about when to decline was found to be unreliable in evaluation. Reference implementation: retrieval/gate.py in the Space repo.
Apache 2.0, matching the base model's license. Training data was generated via distillation from Qwen3.8-27B (Apache 2.0, distillation-permitted license terms).