GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
sammoftah
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
sammoftah
Model Tree
HuggingFaceTB/SmolLM2-360M-Instruct
Input Modalities
Output Modalities
Supported Functionality
sammoftah)adapter_model.safetensors)Given a free-text match scenario, the model emits a structured record of
factors — the things in the text that should move a forecast. Each factor
carries a type, the team it applies to, a severity and certainty in [0, 1], and
the span of evidence it was drawn from. The schema also has explicit slots for
unsupported_claims and ambiguities, so the model is asked to mark what it
could not ground rather than quietly inventing it.
Research and replication of the evaluation below. Not intended for production use — the base model plus a deterministic fallback outperforms it on the metrics that matter (see below).
Anything requiring calibrated severity or certainty values. The central failure of this adapter is that it emits near-zero severity and certainty, which a downstream zero-weight check correctly flags as unusable.
| Method | LoRA (PEFT 0.19.1) |
Rank r | 16 |
lora_alpha | 32 |
lora_dropout | 0.05 |
| Target modules | q_proj, v_proj |
| Bias | none |
| Task type |
Benchmarked 2026-06-13 against a frozen 98-example test set
(data/scenarios/test.jsonl), base vs. tuned, both quantised to Q8_0 and served
through the same runtime. Thresholds were declared before the run.
| Metric | Base | Tuned | Δ | Threshold | |
|---|---|---|---|---|---|
| Factor micro-F1 | 0.0261 | 0.0270 | +0.0010 | ≥ +0.15 gain | ❌ |
| Team attribution accuracy | 0.0430 | 0.0323 | −0.0108 | no regression | ❌ |
| Fallback rate | 0.0918 |
Result: 3 of 4 checks failed. Decision: NO-SHIP.
The adapter got marginally better at classifying factor type and materially worse at everything else. It emits near-zero severity and certainty, which trips the zero-weight hallucination check, which is why the fallback rate nearly doubled. It is faster, and speed was not one of the gates.
The production path remains base model + deterministic fallback.
A fine-tune that fails a gate is only a wasted run if you throw away the gate. The threshold table, the frozen test set, and the base-vs-tuned comparison are reusable; the weights are the least interesting output. Publishing the adapter alongside the report is what makes the negative result checkable by someone else.
python
from peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizerbase = "HuggingFaceTB/SmolLM2-360M-Instruct"model = AutoModelForCausalLM.from_pretrained(base)model = PeftModel.from_pretrained(model, "sammoftah/underdog-lab-smollm2-360m-lora")tok = AutoTokenizer.from_pretrained(base)
bibtex
@misc{moftah2026underdoglora,author = {Moftah, Osama},title = {underdog-lab-smollm2-360m-lora: a LoRA adapter and the eval gate that blocked it},year = {2026},url = {https://huggingface.co/sammoftah/underdog-lab-smollm2-360m-lora}}
CAUSAL_LM |
| Train / validation | 861 / 56 examples |
| 0.1735 |
| +0.0817 |
| ≤ +0.02 |
| ❌ |
| Schema validity | 1.000 | 1.000 | 0.000 | ≥ 0.99 | ✅ |
| Median latency | 3675.6 ms | 2995.4 ms | −680.2 ms | — | — |