GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
cometadata
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
cometadata
Model Tree
Qwen/Qwen3.5-4B
Input Modalities
Output Modalities
Supported Functionality
Test split: 574 documents from 171 degree-granting institutions, none of which appear in
training (the dataset's test split). Scoring: per-field normalized matching (case,
diacritics and punctuation folded; order-insensitive name lists with token-F1 partial credit
above 0.5; strict null semantics), aggregate = unweighted mean of the 10 field scores,
exact = fraction of documents with all 10 fields correct. Greedy decoding, max 1,024 tokens.
| model | aggregate | exact record |
|---|---|---|
| Qwen3.5-4B zero-shot | 0.794 | 0.056 |
| Qwen3.5-4B 3-shot | 0.805 | 0.068 |
| Claude Opus 5, 3-shot (strongest non-RL baseline) | 0.831 | 0.063 |
previous release (...-4b-dapo) | 0.852 | 0.099 |
| this model | 0.867 | 0.146 |
Paired bootstrap over documents (10k resamples), this model minus baseline, 95% CI:
Per field (test): title 0.930 | creators 0.928 | advisors 0.888 | date_issued 0.984 | degree_grantor 0.847 | degree_level 0.840 | degree_discipline 0.877 | degree_department 0.761 | language 0.929 | coverage_spatial 0.686
Per language (test, n): en 0.841 (259), fr 0.886 (167), es 0.879 (83), pt 0.925 (58), nl 0.707 (6), de 0.950 (1)
Validation split (138 docs, held-out institutions, used only for checkpoint selection): aggregate 0.897, exact record 0.167.
Where the remaining errors are: coverage_spatial gold mixes null and filled conventions
across annotators and often uses the region rather than the printed city; degree_department
and degree_level misses are mostly a choice between two printed candidates (college vs
department, DOCTORAT vs Docteur); bilingual title pages (e.g. Serbian/English) are labeled from
the English page. About 10% of gold values are not present in the extracted text at all.
This model is the uniform average of three Qwen3.5-4B checkpoints:
cometadata/etd-metadata-parsing-qwen3.5-4b-dapo — the previous release (DAPO on the first 1,259 records; seed 43, step 40).language are never masked) and reward weights raised to 1.5 on degree_department and degree_level, step 20.On the test split the three ingredients score 0.852, 0.859 and 0.863; the average scores 0.867. Continuing the previous release on the new data (warm start) stayed at 0.853, and a prompt with explicit convention rules did not help, so neither is included.
Data. cometadata/etd-parsing-train-test, institution-held-out split (train 2,399 / test 600).
From train: 1,852 RL prompts, 138 validation docs from 59 institutions held out of RL, 180 docs
reserved for few-shot exemplars; 157 PDFs without a usable text layer, 98 near-empty records and
6 records with no determinable language were excluded. Input = PyMuPDF text of the first pages
joined with [PAGE N] markers under a 4,096-token cap (min 3 pages), after NFKC/ligature/accent
repair.
RL. DAPO via SkyRL v0.3.0: full-parameter FSDP2 fine-tune, 8 colocated vLLM engines,
128 prompts × 16 samples per step, mini-batch 32 prompts (4 updates/step), AdamW lr 1e-6 with
40 warmup mini-steps, clip 0.2/0.28 (dual-clip 10), token-mean loss, mean-only advantages
(no std normalization), dynamic sampling that resamples groups with reward std < 0.005,
KL anchor to the init (k3, coef 1e-3), temperature 1.0, max response 1,024 tokens, thinking
disabled, truncated responses scored (not masked). Reward = the same per-field scorer as the
evaluation (scripts/scoring.py), weighted (title 2, creators 2, advisors 1.5, others 1,
language/coverage_spatial 0.5). Checkpoints selected on validation reward; exports every 5 steps.
One run ≈ 30 steps × 9 min on 8×H100.
Soup. Uniform weight-space average of the listed checkpoints (all Qwen3.5-4B full models, fp32 average cast back to bf16). Adding a fourth checkpoint (a second base-init seed) did not change the test score.
The model was trained and evaluated with exactly this system prompt and a user message holding the page text. Use it verbatim.
text
You are an expert at reading the front matter of electronic theses and dissertations (ETDs) and extracting their bibliographic metadata. The user will show you the first pages of a thesis or dissertation — title page, repository cover sheet, approval/signature page, etc. — and your job is to extract the metadata fields in a structured format. Documents are mostly in Portuguese, English, Spanish, French, German, or Dutch. Copy every field value in the document's original language and wording — do NOT translate (e.g. keep 'Doutor em Ciências' as is; do not render it as 'Doctor of Philosophy'; keep 'Doktorgrades' rather than 'Doctorate'). If a field's value is not stated anywhere in the document, emit null (or [] for advisors). Do not guess or infer values that are not printed. ### JSON Schema { "type": "object", "properties": { "title": { "type": [ "string", "null" ], "description": "Full thesis/dissertation title, exactly as printed." }, "language": { "type": [ "string", "null" ], "description": "ISO 639-1 code of the document language, lowercase (e.g. 'pt', 'en', 'es', 'fr', 'de', 'nl')." }, "creators": { "type": "array", "items": { "type": "string" }, "description": "Author name(s), exactly as printed." }, "advisors": { "type": "array", "items": { "type": "string" }, "description": "Advisor / supervisor / director name(s), without titles like Prof. or Dr. Empty list if none are stated." }, "degree_discipline": { "type": [ "string", "null" ], "description": "Field of study of the degree (e.g. 'Statistics', 'Engenharia El\u00e9trica')." }, "degree_grantor": { "type": [ "string", "null" ], "description": "Institution granting the degree, exactly as printed." }, "coverage_spatial": { "type": [ "string", "null" ], "description": "City/state/country of the institution as stated in the document (e.g. 'S\u00e3o Paulo', 'Louisville, Kentucky')." }, "degree_department": { "type": [ "string", "null" ], "description": "Department, school, faculty, or program within the institution." }, "degree_level": { "type": [ "string", "null" ], "description": "Degree sought, in the document's own words (e.g. 'Doctor of Philosophy', 'Mestre', 'DOCTORAT', 'Doktorgrades')." }, "date_issued": { "type": [ "string", "null" ], "description": "4-digit year the thesis was issued/defended (e.g. '2009')." } }, "required": [ "title", "language", "creators", "advisors", "degree_discipline", "degree_grantor", "coverage_spatial", "degree_department", "degree_level", "date_issued" ], "additionalProperties": false} ### Example Output { "title": "Modelagem numérica de escoamentos turbulentos em canais abertos", "language": "pt", "creators": [ "Ana Beatriz Ferreira" ], "advisors": [ "Carlos Eduardo Lima" ], "degree_discipline": "Engenharia Civil", "degree_grantor": "Universidade Federal do Paraná", "coverage_spatial": "Curitiba", "degree_department": "Departamento de Engenharia Civil", "degree_level": "Mestre em Engenharia Civil", "date_issued": "2015"} ### Summary Read the pages carefully, then respond with a single JSON object in the format specified above containing the extracted metadata.User message format:
text
[PAGE 1]<text of page 1> [PAGE 2]<text of page 2>...[... REMAINING PAGES OMITTED ...] (only if pages were dropped by the token cap)The easiest path is the end-to-end script shipped in this repo under inference/: PDF in,
JSON out, with the exact text extraction, cleanup, page budgeting and prompt used in training.
bash
pip install "vllm>=0.23" "transformers>=5" pymupdfhuggingface-cli download cometadata/etd-metadata-parsing-qwen3.5-4b-dapo-v2 --include "inference/*" --local-dir .python inference/etd_extract.py --pdf-dir /path/to/pdfs --out preds.jsonlSee inference/README.md for options and the output format. Calling the model directly:
python
from vllm import LLM, SamplingParamsllm = LLM(model="cometadata/etd-metadata-parsing-qwen3.5-4b-dapo-v2", trust_remote_code=True, enable_prefix_caching=False)msgs = [{"role": "system", "content": SYSTEM_PROMPT}, {"role": "user", "content": "[PAGE 1]\n...first pages of the ETD..."}]out = llm.chat(msgs, SamplingParams(temperature=0, max_tokens=1024, stop_token_ids=[248044, 248046]), chat_template_kwargs={"enable_thinking": False})print(out[0].outputs[0].text) # a single JSON objectThis is the full multimodal-architecture checkpoint used text-only; disable prefix caching with vLLM (Qwen3.5 gated-DeltaNet state). Requires vLLM ≥ 0.23 / transformers ≥ 5.
Trained on one 8×H100 node on SF Compute (24 node-hours, 432)plusevaluationonStanford′sMarlowecluster.FrontierbaselinesviaOpenRouter21).
COMET ETD metadata parsing pilot, 2026. Code: https://github.com/cometadata/etd-affiliation-parsing
You are an expert at reading the front matter of electronic theses and dissertations (ETDs) and extracting their bibliographic metadata. The user will show you the first pages of a thesis or dissertation — title page, repository cover sheet, approval/signature page, etc. — and your job is to extract the metadata fields in a structured format.Documents are mostly in Portuguese, English, Spanish, French, German, or Dutch. Copy every field value in the document's original language and wording — do NOT translate (e.g. keep 'Doutor em Ciências' as is; do not render it as 'Doctor of Philosophy'; keep 'Doktorgrades' rather than 'Doctorate').If a field's value is not stated anywhere in the document, emit null (or [] for advisors). Do not guess or infer values that are not printed.### JSON Schema{ "type": "object", "properties": { "title": { "type": [ "string", "null" ], "description": "Full thesis/dissertation title, exactly as printed." }, "language": { "type": [ "string", "null" ], "description": "ISO 639-1 code of the document language, lowercase (e.g. 'pt', 'en', 'es', 'fr', 'de', 'nl')." }, "creators": { "type": "array", "items": { "type": "string" }, "description": "Author name(s), exactly as printed." }, "advisors": { "type": "array", "items": { "type": "string" }, "description": "Advisor / supervisor / director name(s), without titles like Prof. or Dr. Empty list if none are stated." }, "degree_discipline": { "type": [ "string", "null" ], "description": "Field of study of the degree (e.g. 'Statistics', 'Engenharia El\u00e9trica')." }, "degree_grantor": { "type": [ "string", "null" ], "description": "Institution granting the degree, exactly as printed." }, "coverage_spatial": { "type": [ "string", "null" ], "description": "City/state/country of the institution as stated in the document (e.g. 'S\u00e3o Paulo', 'Louisville, Kentucky')." }, "degree_department": { "type": [ "string", "null" ], "description": "Department, school, faculty, or program within the institution." }, "degree_level": { "type": [ "string", "null" ], "description": "Degree sought, in the document's own words (e.g. 'Doctor of Philosophy', 'Mestre', 'DOCTORAT', 'Doktorgrades')." }, "date_issued": { "type": [ "string", "null" ], "description": "4-digit year the thesis was issued/defended (e.g. '2009')." } }, "required": [ "title", "language", "creators", "advisors", "degree_discipline", "degree_grantor", "coverage_spatial", "degree_department", "degree_level", "date_issued" ], "additionalProperties": false}### Example Output{ "title": "Modelagem numérica de escoamentos turbulentos em canais abertos", "language": "pt", "creators": [ "Ana Beatriz Ferreira" ], "advisors": [ "Carlos Eduardo Lima" ], "degree_discipline": "Engenharia Civil", "degree_grantor": "Universidade Federal do Paraná", "coverage_spatial": "Curitiba", "degree_department": "Departamento de Engenharia Civil", "degree_level": "Mestre em Engenharia Civil", "date_issued": "2015"}### SummaryRead the pages carefully, then respond with a single JSON object in the format specified above containing the extracted metadata.[PAGE 1]<text of page 1>[PAGE 2]<text of page 2>...[... REMAINING PAGES OMITTED ...] (only if pages were dropped by the token cap)pip install "vllm>=0.23" "transformers>=5" pymupdfhuggingface-cli download cometadata/etd-metadata-parsing-qwen3.5-4b-dapo-v2 --include "inference/*" --local-dir .python inference/etd_extract.py --pdf-dir /path/to/pdfs --out preds.jsonl