laskar-ks

qwen2.5-0.5b-revenue-estimator

README

License: apache-2.0

The task

Input — sector, industry, headcount, and a business description where every company name, ticker and brand token is replaced with [COMPANY].

Output — a single number: log10(revenue per employee) in USD.

Redaction is the point. Left in, the names let a model recognise the brand and recall the answer; that is memorisation, not skill, and it collapses on unseen companies. Removed, the model has to read the business itself.

Revenue per employee — rather than revenue — is the target because headcount alone explains 87% of revenue (r = 0.931) but is uncorrelated with productivity per employee (r = 0.008). Predicting revenue directly would leave almost nothing for the text to contribute.

Evaluation

356 held-out US-listed companies. Greedy decoding, predictions clamped to [4.0, 7.0], identical scoring for every row in the table.

Table with columns: Model, Reads text, MAE (dex), R²
Model	Reads text	MAE (dex)	R²
Constant (training mean)	no	0.310	-0.001
Sector mean	no	0.294	0.085
Linear (headcount + sector + industry)	no	0.260	0.213
TF-IDF + Ridge	yes	0.254	0.318
Qwen2.5-0.5B zero-shot	yes	1.360	-10.562
Qwen2.5-0.5B 3-shot	yes	0.609	-1.809
This adapter, no description	no	0.274	+0.181
This adapter	yes	0.248	+0.339

MAE is in dex (decimal exponents): 0.248 dex means the typical guess is off by a factor of 1.77.

Standard error of the MAE is 0.012 dex, so gaps smaller than about 0.025 dex should not be read as differences.

Fine-tuning fixed output format completely

Table with columns: out-of-range predictions
	out-of-range predictions
Zero-shot	356 / 356 (100%)
This adapter	0 / 356 (0.0%)

The base model answers in dollars, ignoring the log10 instruction, and repeats round numbers such as $1,000,000 for most companies.

Ablation: does reading actually help?

Two adapters trained with identical hyperparameters, seed, and splits. Only the input column differs.

Table with columns: MAE (dex), R²
	MAE (dex)	R²
With description	0.248	+0.339
Without description	0.274	+0.181
Paired difference	+0.0262	+0.159
95% CI (paired bootstrap, 5000 resamples)	[+0.0099, +0.0436]

The interval excludes zero: reading the description measurably helps.

The gain is concentrated rather than uniform — it improves 51% of companies, and helps most where a sector label is misleading. Real Estate, Basic Materials and Energy benefit most; Industrials and Communication Services show no gain.

Usage

python
import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

BASE = "Qwen/Qwen2.5-0.5B"
tokenizer = AutoTokenizer.from_pretrained(BASE)
base = AutoModelForCausalLM.from_pretrained(BASE, torch_dtype=torch.float16, device_map="cuda")
model = PeftModel.from_pretrained(base, "laskar-ks/qwen2.5-0.5b-revenue-estimator").eval()

prompt = (
    "Sector: Technology\n"
    "Industry: Software - Application\n"
    "Employees: 1,200\n"
    "Description: [COMPANY] provides a cloud-based platform for enterprise workflow "
    "automation, sold on subscription to mid-market and enterprise customers.\n"
    "\n"
    "Revenue per employee, log10 USD:"
)

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=12, do_sample=False,
                     pad_token_id=tokenizer.eos_token_id)
answer = tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)

log_ratio = float(answer.strip().split()[0])
revenue = 10 ** log_ratio * 1200
print(f"{10 ** log_ratio:,.0f} per employee -> {revenue:,.0f} total revenue")

The prompt format must match exactly, including the [COMPANY] placeholder. Sending a real company name gives the model a format it was never trained on.

Training

Table

Method	LoRA (r=16, alpha=32, dropout=0.10)
Target modules	q, k, v, o, gate, up, down
Learning rate	2e-4, cosine schedule, 5% warmup
Effective batch	16 (batch 4 x grad accum 4)
Epochs	4 configured, best at epoch 3 (early stopping, patience 1)
Precision	fp16
Hardware	single Colab T4
Training rows	2,852

Loss is computed on the answer tokens only; the prompt is masked with -100. Without that mask, roughly 99% of the loss would come from reproducing the business description.

Limitations

Trained only on US-listed companies reporting in USD. Private and non-US companies are out of scope.
2,852 training rows is small. Rare industries are weakly represented.
Redaction cannot catch product brands that share no tokens with the parent company name, so some companies remain identifiable from context.
Predictions have a narrower spread than reality (std 0.262 vs 0.419), which is the expected response to genuine uncertainty rather than a defect.
A TF-IDF + ridge regression on the same text reaches comparable overall accuracy. This adapter is not a demonstration that LLMs beat classical methods here.
Not a valuation tool. Estimating scale from text does not replace financial statements.

Attribution

Training data derived from defeatbeta/yahoo-finance-data, licensed ODC-BY, sourced from Yahoo Finance. Released for research and education.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

laskar-ks

Model Tree

Base

Qwen/Qwen2.5-0.5B

Adapter

this model

Input Modalities

Text

Output Modalities