yuyangchee98

qwen3-14b-jp-us-claims-lora

README

License: apache-2.0

Intended use

Producing a first English draft of US claims from the claims of a Japanese application, for review by a patent professional.

Out-of-scope use

Filing without professional review.
English-to-Japanese translation.
Descriptions, abstracts or other parts of a specification. The model was trained on claims only.

Training data

978 patent families in which the same invention was filed first in Japan and then in the United States.

Source: the Japanese claims as filed, from the Japanese unexamined publication (A).
Target: the US claims as filed, from the US pre-grant publication (A1), using the official USPTO text.
A family was included only if the US application claims priority from the Japanese application, is not a continuation or divisional, and does not combine other Japanese applications. US publications that reflect a preliminary amendment were excluded.
Each example is a complete claim set. Claims were not aligned one to one, because US counsel often restructures the claim set.
The Japanese applications were filed in 2018.
The data was split by family into 978 training, 49 validation and 209 test examples.

Evaluation

209 held-out families, greedy decoding, compared with the untuned base model and with the US claims as filed.

Table with columns: Metric, US claims as filed, Base model, This adapter
Metric	US claims as filed	Base model	This adapter
US claim format	100%	0%	100%
Outputs containing multiple-dependent claims	0%	86.6%	0%
Outputs violating 37 CFR 1.75(c)	0%	67.5%	0%
chrF against the US claims as filed	100	61.5	64.9

Numerical values are the quantities, ranges and ordinals in the Japanese claims; claim references are excluded. Drawing reference signs are normally removed from US claims, so a low value is expected. Claim counts often differ because counsel adds claims that have no Japanese counterpart.

How to use

Use the prompt below exactly as written, with Qwen3's thinking mode turned off.

python

import torchfrom peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-14B")model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-14B", dtype=torch.bfloat16, device_map="auto")model = PeftModel.from_pretrained(model, "yuyangchee98/qwen3-14b-jp-us-claims-lora").merge_and_unload() PROMPT = ("Translate the following Japanese patent claims into English claims for the "          "corresponding US patent application.")claims = "【請求項1】…\n【請求項2】請求項1に記載の…"   # one claim per line, each starting with 【請求項N】 msgs = [{"role": "user", "content": f"{PROMPT}\n\n{claims}"}]text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True, enable_thinking=False)ids = tok(text, return_tensors="pt", add_special_tokens=False).to(model.device)out = model.generate(**ids, max_new_tokens=3000, do_sample=False)print(tok.decode(out[0, ids["input_ids"].shape[1]:], skip_special_tokens=True))

Training procedure

Framework: mlx-lm 0.32 on an Apple M1 Ultra (128 GB). Training took 7 h 47 min.
LoRA rank 16, alpha 320, on the attention and MLP projections of layers 24–39.
2 epochs, batch size 1 with 4 gradient-accumulation steps, learning rate 1e-5 with cosine decay, maximum sequence length 8,192 tokens. Loss was computed on the English claims only.
Validation loss was 0.603 before training and 0.335 after.
The adapter was converted from mlx-lm format to PEFT format without retraining. Validation loss after conversion differs from the original by less than 0.002.

Limitations

Output must be reviewed. Terms, numbers and dependencies can be wrong.
The model cannot add claims that counsel would add, and may word claims differently from counsel.
In 12 of 650 cases, a numerical value that was retained in the US claims as filed was dropped.
About 1% of generations repeat text or keep adding dependent claims until the length limit. Limit the output length.
The training data covers only Japan-first applications filed in 2018.

License

Apache 2.0, the same as the base model.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

yuyangchee98

Model Tree

Base

Qwen/Qwen3-14B

Adapter

this model

Input Modalities

Text

Output Modalities