GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Pucciami
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
Pucciami
Model Tree
Qwen/Qwen3-32B
Input Modalities
Output Modalities
Supported Functionality
The labels describe how an article frames vaccination. They are media framing indicators for monitoring by experts. They do not measure the attitudes or behaviour of any person or population, and they do not support causal statements about vaccination uptake.
Scored once on a held-out evaluation set of 100 articles, double-annotated and adjudicated, that was
never used for adaptation. The same set was used to choose between the two model families; there is no
external confirmatory set. The scores are end to end, with the base relevance classifier of the same
model family (system G0_E1). The 95% confidence intervals are Wilson score intervals for
accuracy and article-level percentile bootstrap intervals (10,000 resamples) for F1.
| Metric | Value (95% CI) |
|---|---|
| Relevance accuracy (relevance classifier) | 0.970 (0.915–0.990) |
| Construct-presence macro-F1 | 0.659 (0.536–0.752) |
| Construct-presence micro-F1 | 0.763 (0.701–0.818) |
Per-construct F1 of the deployed Qwen3-32B pipeline is in Table 2 of the article, and its valence
results in Supplementary Tables S3 and S4. python reproduce.py in the Zenodo record recomputes the
per-construct and valence results of every system, this one included.
Limits. The evaluation rests on one held-out evaluation set of 100 articles from one national media system, with no external set for confirmation. Confidence intervals are wide for the rarer constructs, and Calculation is exploratory. Undermining valence is detected poorly. For the deployed Qwen3-32B pipeline, pooled recall and precision were both 0.32 (6/19), with 95% confidence intervals of 0.08–0.56 and 0.09–0.52 (article-level bootstrap). Check the output before acting on it.
training_config_used.json
in this repository and in pipeline/configs/qwen3_32b_lora_media_clean_v2_gold300_relevance_readj_v1.json in the Zenodo record.Use the extractor system prompt from the Zenodo record unchanged
(prompts/stage2_extractor_system_prompt_v7.txt). The user message is Article:, a line break, the
article title, a blank line and the article body. Decode greedily. The article's runs used max_new_tokens 1024 and retried at 2048 when the output was truncated or not valid JSON. Qwen3-32B ran with its reasoning mode on. Text inside <think> tags is discarded and only the final JSON object is used.
python
from peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer base = "Qwen/Qwen3-32B"adapter = "Pucciami/vaccination-media-5c-qwen3-32b-lora-supervised"tokenizer = AutoTokenizer.from_pretrained(base)model = PeftModel.from_pretrained( AutoModelForCausalLM.from_pretrained(base, torch_dtype="auto", device_map="auto"), adapter) system_prompt = open("stage2_extractor_system_prompt_v7.txt", encoding="utf-8").read()messages = [ {"role": "system", "content": system_prompt}, {"role": "user", "content": "Article:\n" + title + "\n\n" + body},]inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, return_tensors="pt").to(model.device)output = model.generate(inputs, do_sample=False, max_new_tokens=1024)print(tokenizer.decode(output[0][inputs.shape[1]:], skip_special_tokens=True))The script used for the article's runs, with JSON parsing, retries and checkpoints, is
pipeline/src/models/run_qwen_merged_eval_hf_job.py in the Zenodo record.
Fafangel M, Žibert J, Zaletel-Kragelj L. From annotation towards surveillance: adapted open-weight large language models for article-level 5C classification of vaccination-related media discourse, Slovenia, 2010–2024. Manuscript for Eurosurveillance. Code and data: doi:10.5281/zenodo.22912892.
Apache-2.0, the licence of the base model Qwen/Qwen3-32B. This is one of the four adapters released with
the Zenodo record of the article.
from peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer base = "Qwen/Qwen3-32B"adapter = "Pucciami/vaccination-media-5c-qwen3-32b-lora-supervised"tokenizer = AutoTokenizer.from_pretrained(base)model = PeftModel.from_pretrained( AutoModelForCausalLM.from_pretrained(base, torch_dtype="auto", device_map="auto"), adapter) system_prompt = open("stage2_extractor_system_prompt_v7.txt", encoding="utf-8").read()messages = [ {"role": "system", "content": system_prompt}, {"role": "user", "content": "Article:\n" + title + "\n\n" + body},]inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, return_tensors="pt").to(model.device)output = model.generate(inputs, do_sample=False, max_new_tokens=1024)print(tokenizer.decode(output[0][inputs.shape[1]:], skip_special_tokens=True))