GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
sraivante
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
sraivante
Model Tree
TinyLlama/TinyLlama-1.1B-Chat-v1.0
Input Modalities
Output Modalities
Supported Functionality
Use a task's bundled system prompt from task_prompts.json, followed by the user's short reply. Examples of task families:
Return formats differ by task: a number, a date such as 10-1986, a boolean, an exact label, RETRY, NONE, NULL or a small JSON object. Preserve the task prompt; the user message alone does not identify the intended mapping.
The financial and healthcare constants in these prompts are fixed application examples, not current tax/Medicare rules, forecasts, eligibility decisions or personalized advice. Validate outputs and current business rules separately before using them in an application. The model has not been evaluated as a general financial or medical assistant.
The release verification used PyTorch 2.12.0, Transformers 4.56.2, PEFT 0.17.1 and Tokenizers 0.22.1, with unquantized BF16 base weights on CPU. The training environment used newer library versions, listed below. PEFT 0.17.1 warns about extra inactive metadata in the original adapter configuration; exact loading of every adapter tensor was verified.
bash
pip install "torch>=2.6" "transformers==4.56.2" "peft==0.17.1" accelerate safetensorspython
import jsonimport torchfrom huggingface_hub import hf_hub_downloadfrom transformers import AutoTokenizer, AutoModelForCausalLMfrom peft import PeftModel adapter_id = "sraivante/TinyLlama-1.1B-Financial-Form-Normalizer-LoRA"base_id = "TinyLlama/TinyLlama-1.1B-Chat-v1.0"base_revision = "fe8a4ea1ffedaf415f4da2f062534de366a451e6" tokenizer = AutoTokenizer.from_pretrained(adapter_id)base = AutoModelForCausalLM.from_pretrained( base_id, revision=base_revision, torch_dtype=torch.bfloat16, low_cpu_mem_usage=True,)model = PeftModel.from_pretrained( base, adapter_id, is_trainable=False, autocast_adapter_dtype=False,).eval() with open(hf_hub_download(adapter_id, "task_prompts.json"), encoding="utf-8") as f: task_prompts = json.load(f) messages = [ {"role": "system", "content": task_prompts["birthdate"]["system"]}, {"role": "user", "content": "My birth month was October in the year 1986."},]# Match the original inference notebook: template first, tokenizer second.prompt = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True,)inputs = tokenizer(prompt, return_tensors="pt")with torch.inference_mode(): outputs = model.generate( **inputs, max_new_tokens=32, do_sample=False, pad_token_id=tokenizer.eos_token_id, eos_token_id=tokenizer.eos_token_id, )print(tokenizer.decode( outputs[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True,).strip())See example_normalize.py for a command-line version with a task selector and context-length check. The base model and saved tokenizer support a 2,048-token context. Keep the task prompt, input and output within that limit. The saved adapter has no GGUF or merged-weight export in this release.
The base revision above is pinned for release verification. The original adapter configuration records the base name but no revision, so the exact base commit used during training is not independently recorded.
| Setting | Recorded value |
|---|---|
| Method | Supervised fine-tuning with QLoRA |
| Base | TinyLlama-1.1B-Chat-v1.0 |
| LoRA | Rank 16; alpha 32; dropout 0.05; no bias |
| Target modules | q_proj and v_proj across all 22 layers |
| Adapter tensors / parameters | 88 / 2,252,800 |
| Training examples / tasks | 3,657 / 38 |
| Epochs / final step | 3 / 1,374 |
| Per-device batch / accumulation | 4 / 2, effective 8 examples on one device |
| Learning rate / schedule | 0.0002 / constant |
| Optimizer | paged_adamw_8bit; weight decay 0.001 |
| Gradient clipping / seed | 0.3 / 42 |
| Base quantization during training | 4-bit NF4; double quantization disabled |
| Saved mixed precision | BF16 true, FP16 false |
| Saved SFT maximum length | 1,024; packing disabled |
| Training-time evaluation | None recorded |
The notebook defines a separate MAX_SEQ_LENGTH=512 variable, but the saved SFT configuration says max_length=1024 and the final tokenizer says 2,048. Re-tokenizing the packaged training messages produced lengths of 64–302 tokens, below all three limits. The notebook calls model-level gradient checkpointing; the saved trainer argument flag itself is false.
Warmup ratio 0.03 appears in the notebook and serialized arguments. The newer serialized warmup_steps field also contains 0.03; no precise effective warmup schedule is reconstructed. The logged learning rate stays at 0.0002.
The saved setup cell reports a Tesla T4 and the later training cell reports BF16 enabled. No independent device profile was archived. The notebook records completion at 1,374/1,374, epoch 3/3, with elapsed time around 1 hour 43 minutes.
The original card reports: PEFT 0.19.1, TRL 1.4.0, Transformers 5.9.0, PyTorch 2.10.0+cu128, Datasets 4.8.5 and Tokenizers 0.22.2. See training/training_config.json, training/trainer_state.json and the source notebook. The notebook's code is retained; outputs and Colab execution metadata were removed from the published copy. Optimizer and pickle training-state files are omitted.
The final logged training loss is 0.108085 at step 1,350. The final checkpoint is step 1,374. This is training loss, not validation loss or field-level accuracy; repeated system-prompt tokens contribute to training metrics.
A paired release diagnostic compared the unchanged base with the adapter on 12 newly authored inputs absent verbatim from the packaged training corpus. Both used the same task system prompts and deterministic generation capped at 32 new tokens.
| Model | Exact requested values |
|---|---|
| TinyLlama chat base | 0 / 12 |
| Published LoRA adapter | 11 / 12 |
The adapter returned RETRY for the jointly filing tax-status example, whose expected label was MARRIED_FILING_JOINTLY. The base generally returned explanations instead of the required bare values. Only surrounding whitespace was removed for scoring; this measures strict value-and-format matching on these twelve inputs.
This is a small convenience sample across 11 tasks, not an independently curated benchmark or evidence of reliability across all 38 tasks. Use evaluate_adapter.py to repeat the diagnostic. Full prompts, expected labels, raw outputs and conditions are in verification.json and the linked dataset. No data was used to tune the adapter during this release.
All 88 adapter tensors loaded exactly; finite values, nonzero LoRA matrices, tokenizer behavior and the base-weight checksum were checked. The checkpoint's hash matches the final saved checkpoint-1374 adapter.
The Financial Form Normalization Instructions dataset contains the training JSONL, existing project diagnostic and new release diagnostic. An identical copy is included in training_data/.
The local combined chat corpus and the May 20 uploaded copy have identical parsed records and order; their whitespace/serialization bytes differ. The notebook's filename and recorded 3,657-example count match this corpus. No training-time dataset checksum was recorded, so historical byte identity remains unproven.
There are 12 conflicting system/user prompt pairs with different target labels. The original 114-example project test file overlaps training in every case, so it is labeled as a diagnostic, not a held-out test set. There is no genuine training-time validation or test split.
The data contains short, example-style field values and typo variants. Its exact authoring/generation process was not archived, and no external dataset attribution accompanies this corpus. No additional cleaning, label correction or deduplication was performed for publication.
Expect errors on ambiguous replies, unsupported task prompts and unfamiliar values. Fixed ZIP-code defaults, narrow gender labels and other application-specific assumptions may be unsuitable elsewhere. The model can emit wrong labels or extra text. Application schema checks, ambiguity handling and user confirmation of consequential values remain necessary.
Copyright (c) 2026 sraivante for original fine-tuning contributions, original dataset contributions and original release material, to the extent copyright subsists. These are released under Apache License 2.0.
TinyLlama's base model is separately provided by the TinyLlama project under Apache-2.0. Preserve upstream attribution; this release does not claim ownership of the base model, its tokenizer or its chat template. See NOTICE, upstream model card and the dataset's copyright statement.
Release prepared 2026-09-25.
pip install "torch>=2.6" "transformers==4.56.2" "peft==0.17.1" accelerate safetensorsimport jsonimport torchfrom huggingface_hub import hf_hub_downloadfrom transformers import AutoTokenizer, AutoModelForCausalLMfrom peft import PeftModel adapter_id = "sraivante/TinyLlama-1.1B-Financial-Form-Normalizer-LoRA"base_id = "TinyLlama/TinyLlama-1.1B-Chat-v1.0"base_revision = "fe8a4ea1ffedaf415f4da2f062534de366a451e6" tokenizer = AutoTokenizer.from_pretrained(adapter_id)base = AutoModelForCausalLM.from_pretrained( base_id, revision=base_revision, torch_dtype=torch.bfloat16, low_cpu_mem_usage=True,)model = PeftModel.from_pretrained( base, adapter_id, is_trainable=False, autocast_adapter_dtype=False,).eval() with open(hf_hub_download(adapter_id, "task_prompts.json"), encoding="utf-8") as f: task_prompts = json.load(f) messages = [ {"role": "system", "content": task_prompts["birthdate"]["system"]}, {"role": "user", "content": "My birth month was October in the year 1986."},]# Match the original inference notebook: template first, tokenizer second.prompt = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True,)inputs = tokenizer(prompt, return_tensors="pt")with torch.inference_mode(): outputs = model.generate( **inputs, max_new_tokens=32, do_sample=False, pad_token_id=tokenizer.eos_token_id, eos_token_id=tokenizer.eos_token_id, )print(tokenizer.decode( outputs[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True,).strip())