Model Details
- Name: Tara 1.1 1M SFT
- Architecture: GPT-2-style decoder-only causal LM
- Hugging Face class:
GPT2LMHeadModel
- Parameters: 865,344
- Layers: 4
- Hidden size: 96
- Attention heads: 4
- Vocabulary: 4,096 BPE tokens
- Context length: 256 tokens
- Tensor type: float32
- Format:
safetensors
- License: Apache-2.0
Relationship To Tara10M
The reference Tara10M model is a 10.4M-parameter Llama-style Burmese-English SFT model with a 16K SentencePiece vocabulary and 1,024-token context. Tara 1.1 is not the same architecture and is not a drop-in replacement for Tara10M.
Tara 1.1 is the smaller educational branch:
- much smaller parameter count
- GPT-2-compatible architecture
- English-focused project/domain knowledge
- tuned around short assistant-style answers
- designed for quick local experiments and classroom demonstration
Training Summary
Training was staged locally:
- Continued base-model improvement on WikiText.
- Domain base continuation on local Tara project knowledge in
data/base_school.txt.
- SFT on local instruction/chat data in
data/sft_assistant.txt.
- Multi-turn repair pass on
data/multiturn_fix.txt.
- Conservative final SFT refresh from
tara-1m-assistant-v2 to preserve stronger assistant behavior while adding domain knowledge.
The final shipped checkpoint is:
outputs/tara-1m-assistant-v5-packed
Evaluation
Lower is better.
Table with columns: Model, Eval file, Loss, Perplexity| Model | Eval file | Loss | Perplexity |
|---|
| tara-1m-assistant-v2 | data/sft_assistant.txt | 3.0114 | 20.3157 |
| Tara 1.1 1M SFT | data/sft_assistant.txt | 2.1528 | 8.6091 |
| tara-1m-assistant-v2 | data/multiturn_fix.txt | 4.0452 | 57.1240 |
| Tara 1.1 1M SFT |
Intended Use
Best test areas:
- explaining Tara 1M
- explaining tokenizers, base models, SFT, and safetensors
- short school-project assistant answers
- simple multi-turn educational chat
Prompt format:
User: What is SFT?Assistant:
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer repo_id = "aungkomyint/tara1.1" tokenizer = AutoTokenizer.from_pretrained(repo_id)model = AutoModelForCausalLM.from_pretrained(repo_id) prompt = "User: What is SFT?\nAssistant:"inputs = tokenizer(prompt, return_tensors="pt")outputs = model.generate( **inputs, max_new_tokens=80, do_sample=False, pad_token_id=tokenizer.pad_token_id or tokenizer.eos_token_id,) print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Local folder usage:
from transformers import pipeline pipe = pipeline("text-generation", model="./tara1.1-hf")print(pipe("User: What is Tara 1M?\nAssistant:", max_new_tokens=80)[0]["generated_text"])
Example Outputs
Prompt:
User: What is SFT?Assistant:
Output:
SFT means supervised fine tuning. It trains the model on examples of instructions and good answers.
Prompt:
User: How do I improve the model?Assistant:
Output:
Use cleaner data, train longer, test outputs, then add better examples for weak areas.
Limitations
This model is intentionally tiny and should be treated as an experiment.
Known limitations:
- weak factual reliability outside the training domain
- repetition on some prompts
- short context window
- not robust for long-form generation
- not suitable for safety-critical, legal, medical, or financial use
Files
model.safetensors
config.json
generation_config.json
tokenizer.json
tokenizer_config.json
special_tokens_map.json
vocab.json
merges.txt
chat_template.jinja