GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Privoid-ai
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
Privoid-ai
Model Tree
unsloth/Qwen3-4B-Instruct-2507
Input Modalities
Output Modalities
Supported Functionality
All three models read the same 8 contracts (157 sections) from the official CUAD test split and were scored the same way.
| Model | Clause F1 | 95% range | Precision | Recall | Contract-level F1 | Valid JSON | Exact quotes |
|---|---|---|---|---|---|---|---|
| Privoid Contract 4B | 0.514 | 0.435 to 0.638 | 0.461 | 0.580 | 0.870 | 0.917 | 0.989 |
| Gemini 3.5 Flash | 0.441 | 0.384 to 0.541 | 0.454 | 0.430 | 0.846 | 0.936 | 1.000 |
| Qwen3-4B untrained | 0.233 | 0.185 to 0.382 | 0.176 | 0.347 | 0.705 | 0.745 | 0.617 |
| Model | Clause F1 | 95% range | Precision | Recall | Contract-level F1 | Valid JSON | Exact quotes |
|---|---|---|---|---|---|---|---|
| Privoid Contract 4B, all 102 test contracts | 0.595 | 0.558 to 0.634 | 0.562 | 0.633 | 0.876 | 0.962 | 0.991 |
This is the fullest result for Privoid. It covers more contracts than the comparison above, so do not compare this row with those rows. The 8-contract group turned out harder for Privoid than the test set as a whole.
Clear single-sentence clauses score best, such as Governing Law (0.91) and Agreement Date (0.85). Weak categories include both Affiliate License categories (0.17 and 0.14), Rofr/Rofo/Rofn (0.26) and Non-Disparagement (0.27): do not rely on the model for these. Categories with few test clauses have noisy scores. Price Restrictions had no clauses in the test set, so it has no score and is left off the chart. Exact numbers are in per_category.csv.
Split a contract into sections of up to 5,000 characters, with some overlap so that no clause is cut in half. Send each section separately. The prompt in prompt.json is required.
bash
pip install transformers peft accelerate huggingface_hubpython
import jsonfrom huggingface_hub import hf_hub_downloadfrom peft import AutoPeftModelForCausalLMfrom transformers import AutoTokenizer repo = "Privoid-ai/Privoid-Contract-4B"settings = json.load(open(hf_hub_download(repo, "prompt.json"), encoding = "utf8"))tokenizer = AutoTokenizer.from_pretrained(repo)model = AutoPeftModelForCausalLM.from_pretrained(repo, torch_dtype = "auto", device_map = "auto") def review(section): messages = [{"role": "system", "content": settings["system_prompt"]}, {"role": "user", "content": f"<contract_section>\n{section}\n</contract_section>"}] prompt = tokenizer.apply_chat_template(messages, tokenize = False, add_generation_prompt = True) + settings["assistant_prefix"] inputs = tokenizer(prompt, return_tensors = "pt", add_special_tokens = False).to(model.device) output = model.generate(**inputs, max_new_tokens = 700, do_sample = False) return json.loads(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens = True)) print(review("This Agreement shall be governed by the laws of the State of New York."))bash
pip install llama-cpp-python huggingface_hubpython
import jsonfrom huggingface_hub import hf_hub_downloadfrom llama_cpp import Llama repo = "Privoid-ai/Privoid-Contract-4B-GGUF"settings = json.load(open(hf_hub_download(repo, "prompt.json"), encoding = "utf8"))llm = Llama(model_path = hf_hub_download(repo, "privoid-contract-4b-Q5_K_M.gguf"), n_ctx = 4096, verbose = False) def review(section): prompt = (f"<|im_start|>system\n{settings['system_prompt']}<|im_end|>\n" f"<|im_start|>user\n<contract_section>\n{section}\n</contract_section><|im_end|>\n" f"<|im_start|>assistant\n{settings['assistant_prefix']}") reply = llm(prompt, max_tokens = 700, temperature = 0, stop = ["<|im_end|>"]) return json.loads(reply["choices"][0]["text"]) print(review("This Agreement shall be governed by the laws of the State of New York."))About 4% of replies across the test set were not valid JSON, for example when a long reply is cut off, so wrap json.loads in try in real use. Always check each quote against the source text and have a person review the result.
privoid-contract-4b-Q5_K_M.gguf is 2.69 GB. Q5_K_M is used because Q4 corrupted digits on Qwen3 in earlier work. It was compared with the full model on 20 contract sections before upload:
| Check | Full model | GGUF |
|---|---|---|
| Exact quotes | 98.4% | 97.8% |
| Valid JSON | 16/20 | 15/20 |
| Identical replies | 14/20 |
This is a small check that catches serious damage, not a full benchmark. The scores above are for the full model. On a Colab CPU, one section took roughly 45 seconds.
google/gemini-3.5-flash, without tuning.| Base model | unsloth/Qwen3-4B-Instruct-2507 (Apache-2.0) |
| Data | CUAD v1: 510 contracts and 39 categories, split by contract |
| Training sections | 10,612 |
| Method | LoRA (r 16, alpha 32) with Unsloth, 16-bit |
| Length | 1 epoch, 885 steps |
| Best validation loss | 0.0320 |
| Testing | The official 102 CUAD test contracts, 1,246 sections |
Trained on the Contract Understanding Atticus Dataset (CUAD) v1 by The Atticus Project, used under CC BY 4.0. Text was normalized and split into sections. The model weights are released under Apache-2.0.
pip install transformers peft accelerate huggingface_hubimport jsonfrom huggingface_hub import hf_hub_downloadfrom peft import AutoPeftModelForCausalLMfrom transformers import AutoTokenizer repo = "Privoid-ai/Privoid-Contract-4B"settings = json.load(open(hf_hub_download(repo, "prompt.json"), encoding = "utf8"))tokenizer = AutoTokenizer.from_pretrained(repo)model = AutoPeftModelForCausalLM.from_pretrained(repo, torch_dtype = "auto", device_map = "auto") def review(section): messages = [{"role": "system", "content": settings["system_prompt"]}, {"role": "user", "content": f"<contract_section>\n{section}\n</contract_section>"}] prompt = tokenizer.apply_chat_template(messages, tokenize = False, add_generation_prompt = True) + settings["assistant_prefix"] inputs = tokenizer(prompt, return_tensors = "pt", add_special_tokens = False).to(model.device) output = model.generate(**inputs, max_new_tokens = 700, do_sample = False) return json.loads(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens = True)) print(review("This Agreement shall be governed by the laws of the State of New York."))import jsonfrom huggingface_hub import hf_hub_downloadfrom llama_cpp import Llama repo = "Privoid-ai/Privoid-Contract-4B-GGUF"settings = json.load(open(hf_hub_download(repo, "prompt.json"), encoding = "utf8"))llm = Llama(model_path = hf_hub_download(repo, "privoid-contract-4b-Q5_K_M.gguf"), n_ctx = 4096, verbose = False) def review(section): prompt = (f"<|im_start|>system\n{settings['system_prompt']}<|im_end|>\n" f"<|im_start|>user\n<contract_section>\n{section}\n</contract_section><|im_end|>\n" f"<|im_start|>assistant\n{settings['assistant_prefix']}") reply = llm(prompt, max_tokens = 700, temperature = 0, stop = ["<|im_end|>"]) return json.loads(reply["choices"][0]["text"]) print(review("This Agreement shall be governed by the laws of the State of New York."))