GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
DeependraVerma
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
DeependraVerma
Model Tree
Input Modalities
Output Modalities
Supported Functionality
Drafting contract clauses and full documents. Ask it for a termination clause, an indemnification provision, or a full NDA and it produces well-structured, correctly-formatted legal prose with sensible numbering, defined terms, and cross-references.
python
import torchfrom transformers import AutoTokenizer, AutoModelForCausalLMm = "DeependraVerma/legal-slm-125m-new-sft"tok = AutoTokenizer.from_pretrained(m)model = AutoModelForCausalLM.from_pretrained(m, torch_dtype=torch.bfloat16).eval()sid = tok.convert_tokens_to_idsSYSTEM = "You are a knowledgeable legal and financial assistant. Answer accurately and concisely."question = "Draft a mutual non-disclosure agreement between two software companies."ids = (tok("<|bos|>", add_special_tokens=False)["input_ids"]+ [sid("<|system|>")] + tok(SYSTEM, add_special_tokens=False)["input_ids"]+ [sid("<|user|>")] + tok(question, add_special_tokens=False)["input_ids"]+ [sid("<|assistant|>")])out = model.generate(torch.tensor([ids]), max_new_tokens=1024, do_sample=True,temperature=0.7, top_p=0.9, repetition_penalty=1.1,eos_token_id=sid("<|eos|>"), pad_token_id=sid("<|pad|>"))print(tok.decode(out[0][len(ids):], skip_special_tokens=True))
The chat format is required — this is not a plain completion model.
max_new_tokens matters: a full NDA needs 600–1200 tokens and the default of
20 will truncate mid-clause.
All numbers from lm-evaluation-harness, 0-shot, run by us on this checkpoint.
| Task | 125m-new-sft | 125m-sft (previous) | 500m-sft | random |
|---|---|---|---|---|
| PIQA | 0.6333 | 0.6104 | 0.5963 | 0.500 |
| ARC-Easy | 0.5025 | 0.4487 | 0.4634 | 0.250 |
| HellaSwag | 0.3209 | 0.2822 | 0.2995 | 0.250 |
It beats the previous 125M on every general benchmark, and also beats this project's 500M model — 4x the parameters — on all three. Extra pretraining data mattered more than parameter count here.
| Task | Score | Baseline | Verdict |
|---|---|---|---|
| LegalBench (13-task mean) | 0.5634 | ~0.5669 majority-class | no signal |
| CaseHOLD | 0.1311 | 0.200 (5-way) | below chance |
| MMLU professional_law | 0.2425 | 0.250 | at chance |
| MMLU jurisprudence | 0.2315 | 0.250 | at chance |
These are not passing scores and we are not presenting them as such.
LegalBench here is mostly binary yes/no tasks with skewed label
distributions; the ~56% mean sits at the majority-class baseline, and on the
base model we verified with --log_samples that it behaves as a near-constant
classifier. A 125M model does not do legal reasoning. It writes legal text.
Per-task LegalBench scores range from 0.4043 (telemarketing_sales_rule) to 0.8165 (contract_nli_explicit_identification); the high ones track label skew, not competence.
Pretraining — 591B tokens, 1,127,242 steps, 2048 context, 8xB200. Warmup-Stable-Decay curriculum: general English first, legal/contract text concentrated into the final 35%. Sources: FineWeb-Edu (bulk), Wikipedia, Gutenberg long-form, Cosmopedia, permissively-licensed code, SEC material contracts, HFforLegal case-law, PleIAs/SEC.
A known defect in the pretraining data: HTML markup was not stripped from
the SEC sources, so roughly 19.3% of the 591B-token budget (~114B tokens) was
spent modelling <font> and  . The base model leaks EDGAR markup in
77.1% of legal completions. This SFT model does not (measured 0.0%),
because the SFT data was clean — but the wasted pretraining capacity is real,
and it is the main motivation for the successor run.
Fine-tuning — 27,060 curated Q&A pairs (teacher-distilled and LLM-judged), covering legal Q&A, contract drafting, full-document generation, and general chit-chat. Final validation loss 0.3974 (perplexity 1.488) over 109,499 supervised tokens.
Decontamination — training documents were filtered against benchmark eval sets with 13-gram overlap matching. A retrospective audit of 60,000 sampled training windows found 0 hits (95% upper bound ≤0.015% by the rule of three).
| Parameters | 125,848,320 (tied embeddings) |
| Layers / hidden / heads | 12 / 768 / 12 (no GQA) |
| Context | 2048 |
| Vocabulary | 16,384 (custom BPE, domain-trained) |
| Precision | bfloat16 |
Llama architecture, so it loads with stock transformers.
legal-slm-125m-new-sft-onnxMIT.
| MMLU international_law | 0.2397 | 0.250 | at chance |