ajaxdavis

donto-qwen3.8-27b-predicate-extractor

README

License: apache-2.0

The ambition

Knowledge systems were designed for a world in which producing structured claims was scarce. Generative models invert that constraint: candidate properties and relations can be produced in abundance. The new problem is not merely generating one summary or one canonical graph. It is holding a large, multi-directional, possibly contradictory field of claims without erasing disagreement.

Donto is built for that second problem. It is a contradiction-preserving claim substrate: emit free predicates now; preserve separate claims; defer ontology alignment, entity resolution, joining, standing, and reconciliation to later query-time processes. Donto-Qwen is the extraction front door. Its job is to read a bounded source unit carefully and produce many compact candidate claims that the larger Donto system can hold, compare, cite, align, and re-rank.

The purpose of this adapter is deliberately narrower than general chat:

recover atomic subject–predicate–object relations;
preserve attribution, negation, modality, qualification, and disagreement;
represent direct claims separately from derived or hypothetical views;
use free, concise predicates instead of forcing every source into a fixed ontology;
stop cleanly when a bounded extraction task is exhausted;
return only a validated tool call—never prose around hand-parsed JSON.

What this model is

Table with columns: Property, Value
Property	Value
Release	V15 / `pilot-live-v15-exhaustive-unit-r16-v1`
Base	Qwen3.8-27B, exact revision `1d4bf0f...`
Artifact	PEFT LoRA adapter; base weights are not duplicated here
Primary modality	Text; the Qwen vision tower was neither trained nor evaluated
Intended context	Short document chunks; trained and served at 4,096 tokens
Output	Forced `submit_facts` tool call
Fact fields	`s`, `p`, `o`, ,

This adapter changes the base model's behaviour, not its underlying factual knowledge. It does not intrinsically make Qwen3.8 decode faster. Its efficiency advantages come from short inputs, compact structured outputs, shared prompt prefixes, fewer retries, and greater document concurrency.

Output contract

Every call must produce the submit_facts tool with this shape:

json
{
  "facts": [
    {
      "s": "ex:review-panel",
      "p": "postponed",
      "o": "ex:deployment",
      "c": 0.99,
      "h": false
    },
    {
      "s": "ex:postponement-wasted-money",
      "p": "contrastsWith",
      "o": "ex:postponement-protected-staff",
      "c": 0.92,
      "h": true
    }
  ],
  "more_supported_facts": false
}

s: subject IRI, normally a stable ex:kebab-case identifier.
p: concise camelCase free predicate, with standard RDF predicates allowed.
o: entity IRI or string, numeric, or Boolean literal.
c: model confidence in [0, 1]; this is not calibrated probability.
h: false for direct/source-licensed claims; true for a derived, interpretive, hypothetical, or otherwise non-direct view.
more_supported_facts: continuation signal for the current bounded task.

The exact OpenAI-compatible tool declaration is in schemas/tools.json.

Extraction architecture used for the reported scores

The benchmark did not ask one vague prompt to "extract everything." Each document was traversed through explicit bounded tasks:

Table with columns: Stage, Purpose
Stage	Purpose
Metadata	Document, creator, date, and topic relations
D0	Atomic propositions directly expressed in a source unit
Local residual	Recover a deliberately omitted supported fact, then prove exhaustion
D1 unit	Licensed semantic views and local relational expansion
D2	Discourse-level, contrastive, causal, or interpretive relations (`h:true`)
Structural batches	Relations over the dynamically discovered entity inventory
Global residual	Final bounded coverage audit

This decomposition is important. A single generic extraction prompt is not expected to reproduce the published multi-stage score.

Results

Headline evaluations

Table with columns: Evaluation, Documents, Calls, Gold facts, Exact recall, Exact precision, Mean document recall, 95% bootstrap recall floor, Structured success
Evaluation	Documents	Calls	Gold facts	Exact recall	Exact precision	Mean document recall	95% bootstrap recall floor	Structured success
Source-disjoint V15 validation	40	1,085	2,140	93.32%	93.89%	93.36%	91.74%	100%
Frozen graph-first Gold 100

Gold 100 also produced:

4,762 exact matches from 5,338 targets;
5,324 valid predicted facts;
zero failed calls;
zero malformed fact objects;
zero duplicate exact items;
two reflexive entity errors.

Gold 100 by task

Table with columns: Task, Calls, Targets, Exact recall, Exact precision
Task	Calls	Targets	Exact recall	Exact precision
D0 direct extraction	428	1,751	75.10%	75.14%
D1 unit views	766	645	95.35%	97.77%
D2 discourse views	212	446	93.72%	94.78%

The overall release clears the project's predeclared 80% aggregate-gold target, but the direct D0 slice does not. That distinction is intentional and public. The weakest blind families were long wetlands argument/procurement documents; the strongest were bounded incidents, metadata, residual stopping, and structural relations.

Full row-level outputs—not summaries alone—are published under evaluation/.

What the score does and does not mean

The metric uses one-to-one exact keys over (s, p, o, h). A supported free predicate synonym or stable entity alias receives no credit, which can make the score conservative. Conversely, the gold documents are programmatically generated from explicit graphs and templates. They measure controlled compositional transfer, not the full ambiguity, OCR damage, genre diversity, or factual uncertainty of unseen real-world documents.

The 100-document suite was generated after V15 training, then frozen and hashed before the first V15 inference. It has:

100 unique document hashes;
73 parent templates;
zero source-hash overlap with the 240 V15 training/validation documents;
zero parent-template overlap with the 102 training/validation templates;
45–59 unique gold facts per document (mean 52.88).

The run used one seed. It is not a multi-seed reliability estimate and is not a human-blind real-document benchmark. The prior real llm_nerd fixture was heavily used for development diagnosis and is therefore not presented as blind evidence.

Training data

All distributable training, validation, validator, gold, manifest, and audit files are published at ajaxdavis/donto-qwen3.8-27b-predicate-extraction-data.

Canonical V15 training mixture:

4,255 training rows;
9,013 target facts;
200 source families;
4,123 synthetic task rows from 152 generated documents;
132 reviewed-real rows from 48 source families;
1,085 validation rows with 2,140 facts from 40 source- and template-disjoint synthetic documents;
no training/validation source-family overlap;
no training/validation parent-template overlap;
every rendered row below the 4,096-token contract.

The dataset card separates synthetic and reviewed-real provenance and explains its mixed licensing status.

Training recipe

Table with columns: Parameter, Value
Parameter	Value
Base revision	`1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0`
Seed	38
Epochs / optimizer steps	1 / 532
LoRA rank / alpha / dropout	16 / 32 / 0.05
Trainable parameters	116,727,808
Learning rate	`2e-5`, cosine decay
Warmup	5% / 27 steps
Microbatch / accumulation

Hybrid-layer LoRA coverage

Qwen3.8-27B contains 48 Gated DeltaNet layers and 16 full-attention layers. A generic q_proj/v_proj LoRA recipe would miss most recurrent projections. This adapter explicitly targeted 496 language modules / 992 LoRA tensors and no vision module:

48 each: linear_attn.in_proj_qkv, in_proj_z, in_proj_a, in_proj_b, out_proj;
16 each: self_attn.q_proj, k_proj, v_proj, o_proj;
64 each: mlp.gate_proj, up_proj, down_proj.

The complete name-level audit is artifacts/lora-target-audit.json.

Serving with vLLM on one RTX 3090

The physically tested path used vLLM 0.27.1, Transformers 5.15.0, bitsandbytes 0.50.1, a 4,096-token limit, four concurrent sequences, and prefix caching. vLLM required the tensor-preserving conditional-key remap under runtime/vllm-conditional/; the root adapter remains the canonical native PEFT artifact.

bash
hf download ajaxdavis/donto-qwen3.8-27b-predicate-extractor \
  --local-dir ./donto-qwen

vllm serve Qwen/Qwen3.8-27B \
  --served-model-name qwen3.8-27b \
  --language-model-only \
  --quantization bitsandbytes \
  --load-format bitsandbytes \
  --dtype bfloat16 \
  --max-model-len 4096 \
  --max-num-seqs 4 \
  --max-num-batched-tokens 2048 \
  --enable-prefix-caching \
  --generation-config vllm \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --enable-lora \
  --lora-modules donto-qwen=./donto-qwen/runtime/vllm-conditional \
  --max-loras 1 \
  --max-lora-rank 16

The exact tested launcher includes additional GPU-specific cache and attention flags and is preserved at training/source/start_runpod_quality_canary.sh.

Then call the forced tool and validate its arguments with a real schema. A complete Pydantic/OpenAI-compatible example is in examples/structured_client.py.

Native PEFT use

The root adapter_model.safetensors is the canonical PEFT adapter. Load it over the exact pinned base revision with current Transformers and PEFT. The training script in training/source/train_qwen38_donto_3090.py contains the memory-safe text-only 3090 loader and exact QLoRA configuration.

Do not attach the root native adapter directly to vLLM if its Qwen3.8 key resolver expects the conditional language-model root; use the verified remap supplied in runtime/vllm-conditional.

Efficiency profile

Adapter weights are about 467 MB; the base model remains the dominant memory and compute cost.
The adapter does not improve raw decode tokens/second by itself.
Short chunks reduce prefill and KV-cache costs.
Prefix caching reuses the large common tool/prompt prefix across documents.
The anchor-free five-field format reduces output tokens per useful fact.
Four concurrent sequences fit in the physically tested 24 GB 3090 serving path.
Gold 100 completed 2,718 structured calls in 8,511 seconds at concurrency four, about 19.2 bounded decisions per minute. This is not a generic tokens/second benchmark.

Limitations

Direct extraction is the weakest blind slice. Gold-100 D0 exact recall and precision are both about 75%.
Synthetic-domain bias. The main quantitative evidence comes from generated graph/template documents, not a large human-authored blind corpus.
One training seed. The recipe has not been repeated across multiple seeds.
Two reflexive errors occurred in Gold 100. Role direction and reflexivity still require downstream validation.
No evidence anchors. This release deliberately omits source spans. A separate citer or human review is required when provenance is important.
Free predicates are not ontology alignment. Predicate and entity aliases may vary; alignment belongs downstream.
Confidence is uncalibrated. Do not interpret c as a probability of truth.
h:true is not established fact. It marks derived or interpretive output.
Not a contradiction resolver. It may emit incompatible claims by design; Donto preserves and later ranks them.
Legal, medical, financial, historical, or personal claims require source review.

Development history and negative results

This release is the fifteenth curriculum iteration, trained fresh from the pinned stock base. Rejected adapters were never reused as weight bases. The repository preserves rejection reports for V11–V14 because the failures are informative:

V11 learned structured calls but missed its precision gate and had semantic contamination in one generator family.
V12 completed finitely but was too terse and failed real-document coverage.
V13 reached strong source-disjoint exact metrics but retained role/scope errors.
V14 nearly memorized its supervised contract (about 99% exact) yet transferred poorly to the real development document because its curriculum rewarded short, clean pages.
V15 started fresh and taught dense D0 pages, nonempty recovery passes, explicit exhaustion, larger local D1 views, and broader graph families.

See docs/ for the plan, audits, launch receipt, runbook, and complete negative-result reports.

Reproducibility and integrity

Native final adapter SHA-256: 40d11bcad20061f8291741c4dd8a3701405902f336c27b60592d62d6b2f7a974
vLLM remapped adapter SHA-256: 28df663ffa70bf541cafff2c9bb60df11e4fe000a2ea9124b346a9cc3eb20d8b
Training corpus SHA-256: 1b3630d00d11476c7f1e941a60a442a5eb5eb57dd882b7d188ef9b85d461e738
V15 validation SHA-256: c360e8296e5ec7973f2fab8416d0b30e2aa850fe6b2cc618bb3fa52894d818b2
Gold-100 dataset SHA-256: 6f16c87d06c2a0b850321b8750e54a63364937cfa2823f34f698f6c00f160f1c
Gold-100 evaluation SHA-256: a21e32e6db21f217af4ad7fc66cbcb3836f0607567bdaab9fcc5c11859e2fd55

The repository includes intermediate adapter checkpoints 100–500, trainer states, the exact trainer/evaluator/generator sources, training and serving logs, run contracts, target-module audit, and full row-level evaluation outputs. Unsafe and unnecessary optimizer/RNG pickle states are not part of the public release.

The Hub upload, clean-download hash checks, and unauthenticated browser proof are recorded in PUBLICATION-RECEIPT-20260825.md.

Licence and acknowledgements

The adapter and code are released under Apache-2.0, consistent with the pinned Qwen3.8-27B base. Dataset licensing is documented separately because the corpus contains both generated material and a small reviewed-real research tier.

Built on Qwen3.8-27B, PEFT, Transformers, bitsandbytes, Liger Kernel, and vLLM. The practical RTX 3090 serving work was informed by syv-ai/qwen38-27b-rtx3090.

Citation

bibtex
@misc{donto_qwen38_v15_2026,
  author       = {Davis, Thomas},
  title        = {Donto-Qwen3.8-27B Predicate Extractor V15},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/ajaxdavis/donto-qwen3.8-27b-predicate-extractor}},
  note         = {Experimental LoRA adapter for structured Donto predicate extraction}
}

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider