GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Ihteshamstar
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
Ihteshamstar
Model Tree
unsloth/Qwen3-4B-Instruct-2507
Input Modalities
Output Modalities
Supported Functionality
Detection F1: did the model find each clause type present in a contract (and produce a verbatim quote for it)? Scored per (contract, clause type) over 102 test contracts × 41 clause types; predictions merged across contract chunks. Bootstrap 95% CIs over contracts (n=1000).
| model | detection F1 | precision | recall | quote overlap (token F1) | verbatim rate |
|---|---|---|---|---|---|
| this model (seed 42) | 0.900 [0.887–0.913] | 0.901 | 0.899 | 0.938 | 97.2% |
| Qwen3-14B zero-shot + official clause definitions | 0.816 [0.802–0.830] | 0.814 | 0.818 | 0.729 | 85.4% |
Three seeds (42/43/44) under identical config: F1 0.8995 ± 0.0017. The published weights are seed 42.
Strict full-span coverage (a true positive requires the predicted quotes to fully contain every gold span for the clause — our reimplementation of the rule used by ContractEval, arXiv:2508.03080, which benchmarked zero-shot LLMs on this same 102-contract test set):
| model | strict F1 |
|---|---|
| this model (3-seed mean) | 0.678 |
| GPT-4.1 (ContractEval, zero-shot) | 0.641 |
| Qwen3-8B, best open model tested (ContractEval) | 0.540 |
| Claude Sonnet 4 (ContractEval) | 0.523 |
Caveats for that comparison: our matching is a reimplementation of their stated rule (whitespace-normalized, casefolded substring), not their harness; and ContractEval queries one clause type per call with its definition in the prompt, while this model answers all 41 clause types in a single call (~40× fewer calls per contract).
Data curve (seed 42, detection F1 vs number of training contracts): 50 → 0.746 · 150 → 0.862 · 300 → 0.885 · 408 → 0.900. The zero-shot 14B baseline (0.816) is passed somewhere between 50 and 150 annotated contracts.
The model was trained on exactly one prompt shape. Use it verbatim; other phrasings are out of distribution.
System:
markdown
You are a contract review system. Extract the requested clause types from the contract excerpt. Quote the contract text verbatim. A clause type not present in this excerpt gets an empty list. Reply with JSON only.
User:
markdown
Extract all 41 CUAD clause types from this contract excerpt as JSON (keys: clause types, values: lists of verbatim quotes).CONTRACT EXCERPT:<up to ~60k characters of contract text>
Output: one JSON object with all 41 CUAD clause names as keys, each a list of verbatim quotes (empty list = clause absent). Contracts longer than ~60k characters should be chunked with overlap and the per-chunk outputs merged (union per clause).
bash
ollama create cuad-extractor -f Modelfile # FROM ./cuad-4b-s42-q4_K_M.gguf
Call with format: "json", temperature: 0, and num_ctx sized to fit your chunk
plus the output (silent prompt truncation is the classic failure mode). On a small
fraction of dense chunks (~6% in our runs) greedy decoding enters a repetition loop and
hits the output cap; retrying that chunk with repeat_penalty: 1.05 (still temperature
0, deterministic) recovered nearly all cases in our evaluation, escalating to 1.15/1.25
for the rare holdouts.
python
from transformers import AutoModelForCausalLM, AutoTokenizermodel = AutoModelForCausalLM.from_pretrained("Ihteshamstar/qwen3-4b-cuad-extractor",torch_dtype="auto", device_map="auto")tokenizer = AutoTokenizer.from_pretrained("Ihteshamstar/qwen3-4b-cuad-extractor")
Repository layout: fp16 merged weights at the root (from_pretrained works directly),
gguf/ holds the q4_K_M quantization measured above, lora/ holds the LoRA adapter
(r=16) if you prefer to apply it to the base model yourself.