What it is
- Merge: the adapter merged into Qwen/Qwen3-8B (revision
b968826d9c46dd6066d109eabc6255188de91218) with peft's merge_and_unload, in float32, then saved in bfloat16 (peft 0.21.2); 15.26 GiB of weights.
- Quantization: llm-compressor's AWQ (llmcompressor 0.15.0 and compressed-tensors 0.20.1), duo_scaling both, then the W4A16_ASYM scheme on the Linear layers except lm_head: 4-bit int weights with zero points, in groups of 128, activations not quantized; saved in compressed-tensors' pack-quantized format, 5.68 GiB of weights.
- Calibration: 128 examples of the training data (prompt and reply, laid out as in training), at most 16,384 tokens each, drawn with a fixed seed; none from a validation or test split.
- Files: a transformers model folder (config.json with its quantization_config, the weights as safetensors in compressed-tensors' format), hence
library_name: transformers: vLLM serves it as it is, and transformers loads it with compressed-tensors installed.
How to use
Serve it with vLLM 0.32.0 on Linux with an NVIDIA GPU. vLLM 0.32.0's wheels are built for CUDA 13.0: pip installs them whatever the driver, but the server needs an NVIDIA driver for CUDA 13.0 or later.
pip install vllm==0.32.0vllm serve Jinnn2001/qwen3-8b-sec-earnings-int4-awq --max-model-len 40960
Then send each release as the adapter's card does, with this repository as the model (the prompt, extract_v2.txt, is in the adapter's repository; release.md is one press release as Markdown):
import jsonfrom decimal import Decimal import httpxfrom huggingface_hub import hf_hub_download prompt_path = hf_hub_download( repo_id="Jinnn2001/qwen3-8b-sec-earnings-lora", filename="extract_v2.txt")with open(prompt_path, encoding="utf-8") as f: prompt = f.read() # the system message, verbatimwith open("release.md", encoding="utf-8") as f: release = f.read() # one press release as Markdown response = httpx.post( "http://localhost:8000/v1/chat/completions", json={ "model": "Jinnn2001/qwen3-8b-sec-earnings-int4-awq", "messages": [ {"role": "system", "content": prompt}, {"role": "user", "content": release}, ], "temperature": 0.0, "chat_template_kwargs": {"enable_thinking": False}, "max_tokens": 1024, }, timeout=600.0,)response.raise_for_status()content = response.json()["choices"][0]["message"]["content"]reply = json.loads(content, parse_float=Decimal) # Revenue and net income come as printed, with their unit; EPS is a number.UNITS = { "units": 1, "thousands": 1_000, "millions": 1_000_000, "billions": 1_000_000_000,}figures = {"period_end": reply["period_end"]}for period in ("quarter", "year_to_date"): figures[period] = {} for name, figure in reply[period].items(): if isinstance(figure, dict): # {"value": 12345, "unit": "millions"} value, unit = figure["value"], figure.get("unit") figure = None if value is None else Decimal(value) * UNITS[unit] figures[period][name] = figureprint(json.dumps(figures, indent=2, default=str))
Results
From the serving benchmark (results/serving_results.md): one NVIDIA H100 NVL at $3.429 per GPU-hour, vLLM 0.32.0, prefix caching off, --max-model-len 40,960; vLLM's defaults otherwise. The adapter column is the base model in bf16 with the LoRA adapter, as stage 2 served it but with prefix caching off. Accuracy is correct over graded figures, as in the benchmark. Validation is where step-00025 was chosen, so every validation score below may overstate: it compares the variants, and the time test gives the level (descriptive here, as it is in the adapter's card). Stage 2's runs of the adapter scored 287/293 (98.0%) on validation, chosen on validation, and 1699/1740 (97.6%) on the time test. Agreement counts the predicted figures (6 per release; null equals null) equal to another run's.
Table with columns: measure, int4-awq, adapter| measure | int4-awq | adapter |
|---|
| validation (step-00025 chosen on it), correct / graded | 280/293 (95.6%) | 287/293 (98.0%) |
| time test, correct / graded (descriptive) | 1699/1740 (97.6%) | 1700/1740 (97.7%) |
| validation figures equal to the adapter variant's | 353/360 figures, 56/60 releases | (itself) |
| time test figures equal to the adapter variant's | 2034/2076 figures, 328/346 releases | (itself) |
| validation figures equal to stage 2's run of the adapter | 353/360 figures, 56/60 releases | 360/360 figures, 60/60 releases |
| time test figures equal to stage 2's run of the adapter | 2032/2076 figures, 328/346 releases | 2069/2076 figures, 343/346 releases |
| best throughput, releases/h | 4,938 | 4,491 |
| at requests in flight | 32 | 32 |
| latency p95 there (s) | 36.38 | 41.09 |
| /1,000releasesthere,at3.429/h | $0.694 | $0.763 |
| at $1/h | $0.202 | $0.223 |
| weights in GPU memory (GiB) | 5.78 | 15.36 |
| KV cache (GiB) | 76.71 | 66.84 |
| KV cache (tokens) | 558,608 | 486,720 |
| max concurrency at 40,960 tokens | 13.64 | 11.88 |
| launch to healthy (s) | 64 | 172 |
| model on disk (GiB) | 5.68 | – |
int4-awq per level (the time test's prompts, 128 per level after 8 warm-up requests; a cost per 1,000 releases is the GPU's price for the level's makespan):
Table with columns: in flight, ok, releases/h, latency p50 / p95 (s), first token p50 (s), decode tok/s (median), /1,000at3.429/h, at $1/h| in flight | ok | releases/h | latency p50 / p95 (s) | first token p50 (s) | decode tok/s (median) | /1,000at3.429/h | at $1/h |
|---|
| 1 | 128/128 | 2,706 | 1.27 / 1.98 | 0.52 | 180 | $1.267 | $0.370 |
| 2 | 128/128 | 3,578 | 1.92 / 3.17 | 0.58 | 104 | $0.958 | $0.279 |
| 4 | 128/128 | 4,197 | 3.28 / 4.99 | 0.79 | 55 | $0.817 | $0.238 |
| 8 | 128/128 | 4,609 | 6.09 / 8.57 | 0.99 | 27 | $0.744 | $0.217 |
| 16 | 128/128 | 4,822 | 11.65 / 16.16 | 1.96 | 14 | $0.711 | $0.207 |
| 32 | 128/128 | 4,938 | 23.42 / 36.38 | 3.23 | 7 | $0.694 | $0.202 |
| 64 | 128/128 | 4,915 | 44.15 / 70.89 | 17.89 | 5 | $0.698 | $0.203 |
Limitations
- Every limitation in the adapter's card applies to this model too.
- Quantization changes some predictions: the figures equal to the adapter variant's are 353/360 figures, 56/60 releases on validation and 2034/2076 figures, 328/346 releases on the time test.
- One session on one GPU type: throughput, latency and cost depend on the GPU, the vLLM version and its settings.
- Calibrated on training releases only (128 examples).
License
Apache-2.0, like Qwen/Qwen3-8B. The benchmark's code is MIT-licensed (GitHub); the dataset card states its data's terms.
Citation
@misc{sankaran2026secearningsbench, author = {Swaminathan Sankaran}, title = {{sec-earnings-bench}: extracting {GAAP} figures from {SEC} earnings releases, graded against {XBRL}}, year = {2026}, url = {https://github.com/jinnn11/sec-earnings-bench}}
Links