thurstonpowers

xll-regulations-qwen3.5-9b

README

License: apache-2.0

Base model and loading contract

Base: Qwen/Qwen3.5-9B, revision c202236235762e1c871ad0ccb60c8ee5ba337b9a. This candidate is a full-language model, not a LoRA adapter. Its protocol is raw-text-cfr-source-native-structure-0.2, prompt variant cfr-native-lexical-sparse-v2-regulations.

Use the exact trained system prompt and message encoding illustrated in example.json, a training example, not heldout evidence. User input consists of sparse start:end|JSON-string lexical fragments; unlisted characters are ASCII spaces. Offsets are zero-based, end-exclusive Unicode character positions. Output is {"tree":[tag,start,end,attributes,children]}. Do not replace the trained interface with an arbitrary plain-text prompt. Deterministic assembly, validation and human review remain required.

Training

5,000 sections: 2,792 GPO-derived gold and 2,208 deterministic eCFR silver.
Two epochs, 2,500 optimizer updates, effective batch four; seed 102301.
Learning rate 5e-6; Adafactor; 125-step warmup, cosine decay to 0.1 of peak.
Full language parameters trained; vision frozen. FP32 training parameters, BF16 compute; retained checkpoint is BF16 safetensors.
32,768-token training context, without truncation; longest example 26,455 tokens.

Table with columns: Target XML nodes, Gold, Silver, Total
Target XML nodes	Gold	Silver	Total
1–10	845	661	1,506
11–25	919	693	1,612
26–50	612	373	985
51+	416	481	897

Training includes 15 gold and 82 silver table-bearing sections. Silver is a deterministic conversion, not human-reviewed gold. Source diversity does not establish correctness on all titles. Training report and inventory record the recipe and counts.

Reserved validation — distinct from development

On a score-blind selected, token-eligible 158-section reserved slice, the retained candidate achieved 148/158 (93.7%) whitespace-tolerant exact, 146/158 (92.4%) strict exact, and 158/158 native-valid. Train/development overlap was checked by IDs, groups and substantial passages; remaining limitations are documented with reserved evidence.

This slice has only titles 5, 12 and 27; no eligible tables; node-bin counts 35/50/49/24. It has fewer high-complexity examples than development200. Do not compare the overall percentages directly or infer table/general-title performance. It is now consumed validation, not untouched data for future recipe selection.

Complexity-balanced development evaluation

Frozen 200-section model-selection/development set, not an untouched test estimate. Success requires a native-valid tree, identical hierarchy/tags and supported attributes, and corresponding spans equal after stripping outer whitespace. Interior whitespace is preserved. This is target-aware reconstruction, not complete USLM XSD validation, and differs from the statutes card's schema-gated structural metric. Percentages across the two model series are not interchangeable.

Table with columns: Metric, Count, Rate
Metric	Count	Rate
Whitespace-tolerant joint exact	127/200	63.5%
Strict joint exact	115/200	57.5%
Native tree valid	179/200	89.5%
Hierarchy and labels exact	137/200	68.5%

Table with columns: Target XML nodes, Sections, Exact success, Native-valid
Target XML nodes	Sections	Exact success	Native-valid
1–10	35	33/35 (94.3%)	35/35
11–25	50	40/50 (80.0%)	50/50
26–50	49	28/49 (57.1%)	45/49
51+	66	26/66 (39.4%)	49/66

Success by structural complexity

Complexity counts target XML nodes, including inline markup and XHTML; it is not legal difficulty. Lines connect ordinal-bin rates, not fitted trends. Of the 73 failed development sections, 61 are above 25 nodes and 40 are in the 51+ bin. Table-bearing sections: 13/30 exact (43.3%), 16/30 native-valid (53.3%). Without tables: 114/170 exact (67.1%), 163/170 native-valid (95.9%). Invalid trees and structural errors remain significant deployment risks.

Task comparison and frontier caveats

These are same-cohort original shared-prompt results, not a demonstrated best-prompt frontier comparison. Unlike the statutes series, no clarified-prompt regulation rerun is available here; it would require a separate experiment.

Table with columns: Model, Sections, Whitespace-tolerant exact, Native-valid
Model	Sections	Whitespace-tolerant exact	Native-valid
Retained full fine-tune	200	127/200	179/200
Selected LoRA	200	101/200	179/200
GPT-6 Sol — original shared prompt	200	0/200	183/200

GPT-6 Sol produced 183/200 native-valid trees and 14/200 matching hierarchies, but zero supported-attribute-exact outputs, hence zero joint-exact outputs. Zero exact success does not imply no useful structure or inferior general legal ability. It received the same frozen prompts via OpenRouter, reasoning disabled, provider- default sampling and an 8,192-token output cap. Prompt suitability, tokenization, stopping and runtime differences limit the comparison. This is not an optimized frontier benchmark or a general intelligence ranking. No output repairs are used.

Appendix: partial vanilla, matched IDs only

Vanilla completed only a nonrandom 126-section prefix on RTX 5090; it is not a complete/hardware-matched 200-section baseline. All models below are restricted to those same 126 IDs; do not mix these denominators with the full200 results.

Table with columns: Model, Matched sections, Whitespace-tolerant exact, Native-valid
Model	Matched sections	Whitespace-tolerant exact	Native-valid
Retained full fine-tune	126	80/126	121/126
Selected LoRA	126	61/126	117/126
GPT-6 Sol — original shared prompt	126	0/126	114/126
Vanilla — partial RTX 5090	126	0/126	0/126

Matched partial vanilla comparison

Runtime and reproducibility

Local evaluation uses BF16 without quantization, greedy batch-one generation, 32,768-token context, at most 8,192 new tokens, and JSON-object stopping. Retained and LoRA candidates were evaluated on NVIDIA GB10; partial vanilla on RTX 5090. Evaluation used Transformers 5.15.1 and PyTorch 2.13.0+cu130. Quantized variants and every consumer GPU/context combination are not qualified. BF16 weights are roughly 18 GiB before activation/cache overhead.

Development/complexity evidence includes counts, title/table provenance, report and prediction hashes, and partial-baseline caveats. Reserved validation and training report are separate evidence. Weights status records expected checkpoint hashes, not proof of an uploaded model. SHA256SUMS.json covers the documentation artifacts published in this commit.

Selection, limitations, and license

The retained mixed5000, two-epoch seed-102301 candidate is not automatically replaced by later recipes. Development selection is not an unbiased test estimate. No complete training corpus or development predictions are packaged. Supported attributes are restricted; opaque metadata, full USLM schema compliance and legal accuracy are not claimed. Whole titles, oversized sections, OCR/PDF input, arbitrary state conventions and long-context/quantized deployment are not validated. Use official source text as authoritative; review all model-generated structure.

Apache-2.0; the base-model license is included as LICENSE.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

thurstonpowers

Model Tree

Base

Qwen/Qwen3.5-9B

Fine-tuned

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality