Privoid-ai

Privoid-Action-1B

README

License: apache-2.0

What it does

Input: one transcript segment of whole turns with turn IDs and speaker IDs, plus optional speaker names, meeting date and timezone. It has been tested on segments of up to about 1,536 transcript tokens. Longer meetings should be split into segments.
Output (schema v1.0, docs/SCHEMA.md):
- actions: task, owner speaker IDs and owner wording, assignment status, deadline wording, conditions, active or cancelled
- decisions: active or reversed
- unresolved requests: no response, declined, deferred or unclear
- on every record: exact evidence quotes and review flags
Deadlines are converted to dates by deterministic code, and only when unambiguous. Unknown owners and deadlines are left empty rather than guessed.
The validator is part of the model. privoid_action.contract.validate_output(..., ground=True) and inference.extract handle the raw output as follows:
- A quote that differs from the transcript only in tidied wording (fillers, repeats, disfluencies) is snapped to the exact transcript text and flagged.
- A quote that occurs only on a different turn is moved to that turn and flagged.
- Owner wording that cannot be found in the evidence is removed.
- Any record that still fails validation is rejected. It is never repaired with invented content.
All numbers below for real meetings use this validator, the same way the bundled API does. The raw validity of the model output alone is reported next to them.

Out of scope

Audio, speech recognition, diarization and speaker identification; whole-meeting summarisation in one pass; chat and Q&A; autonomous task execution; languages other than English.

How to use

bash

pip install -r requirements.txt   # tested with transformers 5.5.0, peft 0.20.0, torch 2.11.0hf download Privoid-ai/Privoid-Action-1B --local-dir Privoid-Action-1Bcd Privoid-Action-1B

python

import jsonfrom privoid_action import inference model, tok = inference.load_model("ibm-granite/granite-4.0-1b", revision="6a7381ba1f54d684ff508d991aeb7dc580157103",                                  adapter_path=".", local_files_only=False)   # downloads the base model oncepayload = json.load(open("examples/request_accepted_request.json", encoding="utf-8"))out = inference.extract(model, tok, payload, max_prompt_tokens=2600)print(json.dumps(out["result"], indent=2))   # validated records; rejected ones are listed separately

A local FastAPI service bound to 127.0.0.1 is included (privoid_action_api/, see docs/API.md). It loads from local files only and does not log or store transcripts.

Using the adapter with plain transformers and peft skips the validator. Raw outputs then contain unverified quotes and owners, so don't use them that way.

Training

Base: ibm-granite/granite-4.0-1b at revision 6a7381ba1f54d684ff508d991aeb7dc580157103 (1.63 B parameters, bf16).
Adapter: LoRA with r 16, alpha 32 and dropout 0.05, on the q/k/v/o and MLP input/output projections.
Stage 1: a new adapter trained on 6,572 synthetic segments (677 for dev).
- Settings: 2 epochs, learning rate 2e-4 cosine, effective batch 16, 822 steps, 7.9 h.
- The synthetic data was generated from templates covering product, engineering, sales, operations, people and finance scenarios.
- Examples longer than 4,096 tokens were excluded, never truncated.
Stage 2, top-up: the same adapter trained for 79 more steps (37 minutes), at learning rate 1e-4 with effective batch 8.
- Data: 640 examples. 340 are long composites built by joining several synthetic segments, so that they resemble real meeting length; 300 are replays of stage-1 data.
- No real meeting text was used in either stage.
Hardware: a single RTX 3060 (12 GB).

Evaluation

Locked test. It was run once, after the acceptance criteria were recorded at 2026-10-27T18:26:26. Full results are in results/final_test.json.

Synthetic test sets

These contain the same kind of data as the training set. "Held-out templates" are scenario families never seen in training. The 95% interval for held-out action F1 is 0.979–0.993 (cluster bootstrap).

Table with columns: System, Set, n, Action F1, Decision F1, Request F1, Owner acc., Status acc., No-action FP, Fully valid
System	Set	n	Action F1	Decision F1	Request F1	Owner acc.	Status acc.	No-action FP	Fully valid
Privoid Action-1B	familiar templates	401	1.000	0.995	0.989	0.991	0.998	0.000	0.995
Privoid Action-1B	held-out templates	391	0.986	0.981	0.946	0.962	0.977	0.043	0.972
Rules baseline	familiar templates	401	0.761	0.830	0.833	0.996	0.893	0.204	1.000
Rules baseline	held-out templates	391	0.741	0.994	0.543	1.000	0.874	0.174	1.000
Granite 4.0 1B (no adapter)	familiar templates	401	—	—	—	—	—	0.000	0.105
Granite 4.0 1B (no adapter)	held-out templates	391	—	—	—	—	—	0.000	0.130

The base model without the adapter almost never produces valid output in this format, so its F1 is undefined (—).

Real meetings

AMI and ICSI meeting transcripts, scored against "silver" labels: the actions and decisions that the corpora's own human summaries link to specific turns. This measures recall only. It can't measure precision, because the summaries don't list everything said in a meeting. The label sets are small, especially the 28 AMI actions, so treat these numbers as rough.

Table with columns: System, Set, Segments (meetings), Actions found, Decisions found, Fully valid (raw → with quote matching), Rejected records
System	Set	Segments (meetings)	Actions found	Decisions found	Fully valid (raw → with quote matching)	Rejected records
Privoid Action-1B	AMI test	250 (20)	10/28 (0.36)	12/72 (0.17)	0.70 → 0.86	37
Privoid Action-1B	ICSI (unseen style)	250 (68)	no labels	12/167 (0.07)	0.65 → 0.80	78
Rules baseline	AMI test	250 (20)	7/28 (0.25)	0/72 (0.00)	0.99 → 0.99	2
Rules baseline	ICSI (unseen style)	250 (68)	no labels	0/167 (0.00)	0.98 → 0.98	4
Granite 4.0 1B (no adapter)	AMI test	250 (20)	0/28 (0.00)	0/72 (0.00)	0.15 → 0.15	6
Granite 4.0 1B (no adapter)	ICSI (unseen style)	250 (68)	no labels	0/167 (0.00)	0.10 → 0.10	0

Acceptance criteria

The criteria were recorded before the test and checked afterwards.

Table with columns: Criterion, Bar, Measured, Result
Criterion	Bar	Measured	Result
`min_action_f1_synthetic_heldout`	0.8	0.986	pass
`min_action_f1_ami_human_test`	—	—	not measured
`max_invented_owner_rate`	0.02	—	not measured
`max_invented_deadline_rate`	0.02	0.000	pass
`max_no_action_false_positive_rate`	0.05	0.043	pass
`min_assignment_status_acc`	0.85	0.977	pass
`min_fully_valid_rate`	0.95	0.972	pass
`min_summary_action_recall_ami_test`	0.35	0.357	pass
`min_summary_decision_recall_ami_test`	0.1	0.167	pass
`min_fully_valid_rate_ami_test`	0.75	0.860	pass
`max_injection_executed`	0	0	pass

Invented owners: not measured. The synthetic test sets contain no actions without an owner, so this rate could not be computed.
AMI human test: none. No human-labelled AMI test exists.
Instruction-injection probes: 7 probes plant instructions inside a transcript. The adapter produced 0 records from the planted instructions and still found all 14 real actions.

Development history (dev sets, not the test)

Stage-1 adapter on 300 AMI dev segments:
- fully valid output rose from 0.78 to 0.91 once quote matching was added to the validator;
- actions found: 14/38 → 19/38;
- decisions found: 6/63 → 8/63.
Top-up on a fixed 100-segment AMI dev subset:
- decisions found: 8/63 → 13/63;
- actions unchanged at 19/38;
- fully valid output fell from 0.88 to 0.82, because more malformed records were produced and rejected;
- no regression on synthetic dev.

Limitations

The model misses many real items. Recall on real meetings is low, and lower again on meeting styles unlike the synthetic training data (ICSI).
Some outputs are rejected. On real meetings, 14–20% of segment outputs contained at least one rejected record. Without the quote-matching validator, 30–35% of segments would fail validation.
The transcript must be well formed. It needs turn IDs and speaker IDs. Diarization errors upstream carry straight through.
Segments are independent. A request and its acceptance far apart can be missed, and cancellations across segments are flagged but not applied.
Relative deadlines are not converted. Wording like "Friday" or "end of week" is kept as text.
Valid output is not proof of correctness. A valid record with exact quotes can still be the wrong interpretation.

Disclosures about how this release was made

Synthetic labels were not reviewed. The planned human review of synthetic labels was skipped, and only automatic validation was applied.
Acceptance criteria were set late. They were written after training, but before the test was run. They were based on dev results.
The validator was tuned on dev errors. The quote-matching and owner-removal rules were designed while examining dev-set errors, so the dev numbers are optimistic. The test numbers above are the ones to rely on.
The top-up was a separate step. It was a short additional run, chosen on dev results, and no other variants were tried against the test.
Results are single runs. Each was a single training run with fixed seeds, with no repeated runs.

Privacy

Inference is local. Whether a deployment is private also depends on the application around the model, its logs and its network setup.

Attribution and licences

Base model: IBM Granite 4.0 1B, Apache-2.0. This adapter and code are Apache-2.0 (see LICENSE and NOTICE).
Evaluation data: annotations of the AMI Meeting Corpus and ICSI Meeting Corpus, both CC BY 4.0 (https://groups.inf.ed.ac.uk/ami/). No corpus text is included in this repository.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider