from transformers import AutoModelForCausalLM, AutoTokenizer
repo_id = "techkiyan/KiyanNet-Finance-Base-48M"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(repo_id)
prompt = "Reliance Industries reported"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(
**inputs,
max_new_tokens=80,
do_sample=True,
temperature=0.8,
top_p=0.95,
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
No custom Python files are required.
Files
This public repo intentionally contains only standard loading and fine-tuning files:
config.jsongeneration_config.jsonmodel.safetensorstokenizer.jsontokenizer_config.jsonREADME.md
Training scripts, logs, token shards, raw datasets, duplicate checkpoints, and custom model-code files are intentionally excluded.
Model Details
Table with columns: Item, Value| Item | Value |
|---|
| Public loader | AutoModelForCausalLM |
| Model class | GPT2LMHeadModel |
| Tokenizer loader | AutoTokenizer |
| Tokenizer type | Fast BPE tokenizer converted from original SentencePiece model |
| Original trained parameters | 48,376,320 |
| HF model parameters | 48,433,152 |
| Context length | 1,024 |
The HF parameter count is slightly higher because the public Transformers conversion uses GPT-2 modules with zero-initialized bias tensors. The converted model was checked against the original checkpoint; logits matched within normal floating-point tolerance on a fixed prompt.
Fine-Tuning Starter
from transformers import AutoModelForCausalLM, AutoTokenizer, Trainer, TrainingArguments
repo_id = "techkiyan/KiyanNet-Finance-Base-48M"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(repo_id)
if tokenizer.pad_token is None:
tokenizer.pad_token = tokenizer.eos_token
training_args = TrainingArguments(
output_dir="kiyannet-finetuned",
per_device_train_batch_size=2,
gradient_accumulation_steps=8,
learning_rate=5e-5,
num_train_epochs=1,
logging_steps=25,
save_steps=500,
)
Benchmark Protocol
Benchmarks were run locally on CPU with torch==2.13.0.
The benchmark is intentionally finance-focused and lightweight. It is not a leaderboard result. It was designed to compare small standalone causal language models in the 30M-100M range under identical local conditions.
Model Selection
The target comparison set was 10 standalone AutoModelForCausalLM models between roughly 30M and 100M parameters.
Public finance-specific causal LMs in this exact size range are scarce. One finance-named 70M candidate, Rudra501/model70m_finance, appeared to be an adapter/training artifact rather than a standalone loadable checkpoint, so it was excluded. Larger finance models, GGUF-only models, classifiers, fill-mask models, and models above 100M parameters were also excluded.
Evaluation Methods
The benchmark uses multiple strategies:
- Category perplexity on finance-style snippets
- 20 fixed generation prompts across company, regulatory, technical-analysis, options, filing, and safety categories
- Greedy generation throughput
- Sampled generation throughput
- Repetition heuristic
- Bad-advice phrase heuristic
- Metadata-marker heuristic
- RBI role-error heuristic
- Nifty foreign-context heuristic
- Finance-tokenizer round-trip tests
10-Model Benchmark Summary
Lower perplexity is better. Higher tokens/sec is better.
Table with columns: Model, Params, Overall PPL ↓, Company PPL ↓, Regulatory PPL ↓, Technical PPL ↓, Options PPL ↓, Filing PPL ↓, Greedy tok/s ↑, Sample tok/s ↑, Round-trip| Model | Params | Overall PPL ↓ | Company PPL ↓ | Regulatory PPL ↓ | Technical PPL ↓ | Options PPL ↓ | Filing PPL ↓ | Greedy tok/s ↑ | Sample tok/s ↑ | Round-trip |
|---|
| KiyanNet Finance Base 48M | 48.4M | 147.97 | 51.36 | 77.60 | 229.80 | 574.70 | 76.96 |
Diagnostic Heuristics
Table with columns: Model, Avg repetition ↓, Bad-advice hits ↓, Metadata-marker hits ↓, RBI role errors ↓, Nifty foreign-context hits ↓| Model | Avg repetition ↓ | Bad-advice hits ↓ | Metadata-marker hits ↓ | RBI role errors ↓ | Nifty foreign-context hits ↓ |
|---|
| KiyanNet Finance Base 48M | 0.1664 | 1 | 0 | 0 | 0 |
| DistilGPT2 82M | 0.0508 | 1 | 0 | 0 | 0 |
| Pythia 31M | 0.2651 |
The bad-advice heuristic is intentionally broad and counts appearances of words such as buy, sell, target price, stop loss, guaranteed, and risk-free. It is useful for smoke testing but should not be treated as a complete safety evaluation.
Prompt Coverage
The generation benchmark used 20 prompts:
Reliance Industries reportedTata Motors quarterly results showedHDFC Bank announcedInfosys management statedState Bank of India reportedThe Reserve Bank of India announcedSEBI issued a circular regardingNSE Clearing informed members thatNifty 50 approached resistance while
Tokenizer Round-Trip Checks
The current tokenizer preserves finance notation in all tested examples:
Table with columns: Text, Round-trip| Text | Round-trip |
|---|
₹1.06 crore | pass |
₹3,642.50 | pass |
7.2% | pass |
FY2025-26 | pass |
NIFTY 50 | pass |
BANKNIFTY | pass |
Sample KiyanNet Outputs
Greedy decoding examples from the benchmark:
Table with columns: Prompt, Output excerpt| Prompt | Output excerpt |
|---|
Reliance Industries reported | Reliance Industries reported a net loss of 1 5 billion in the third quarter of 2016... |
Tata Motors quarterly results showed | Tata Motors quarterly results showed a strong performance in the first quarter of fiscal 2019... |
HDFC Bank announced | HDFC Bank announced a new loan agreement with the Bank of England... |
SEBI issued a circular regarding | SEBI issued a circular regarding the names of the concerned stock exchange(s) where the Equity Shares are proposed to be listed... |
These samples show both strengths and weaknesses. The model has finance-domain fluency and performs well on the small local perplexity benchmark, but it still invents figures, mixes jurisdictions, and can reuse generic news or filing patterns. It should be treated as a raw-mix baseline, not a final India-first analyst model.
Interpretation
KiyanNet Finance Base 48M performs best on this local 30M-100M finance-smoke benchmark by overall perplexity, company-text perplexity, regulatory-text perplexity, technical-text perplexity, and filing-text perplexity. It is also faster than DistilGPT2 while using fewer parameters.
The main weakness is options and derivatives language. DistilGPT2 and Pythia 70M Deduped v0 scored better on the options subset. This supports the next training direction: add explicit options, futures, volatility, and technical-market-state data rather than continuing only with generic financial news.
Intended Use
- Compact finance-domain language modeling experiments
- Continued pretraining
- Fine-tuning for structured finance tasks
- Lightweight local baselines
- Financial text understanding research
Not Intended For
- Buy or sell recommendations
- Entry/exit levels
- Price targets
- Guaranteed return claims
- Personalized investment advice
- Live factual claims without retrieval or external verification
- Option payoff or Greek calculation without deterministic tools
Limitations
- This is a base model, not an instruction-tuned analyst.
- It can hallucinate dates, figures, institutions, currencies, and corporate actions.
- It can mix India-specific prompts with global-news templates.
- Options and derivatives understanding is still weak.
- Generated text should be verified against reliable data sources.
- The benchmark above is a small local smoke benchmark, not a broad public leaderboard.
Recommended Next Step
Preserve this model as a 607M-token raw-mix baseline. The next stronger base model should use a balanced India-first training mix with explicit regulatory, corporate-announcement, technical-analysis, options, futures, volatility, and market-state data.