GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
shadowrock-io
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
shadowrock-io
Model Tree
microsoft/harrier-oss-v1-270m
Input Modalities
Output Modalities
Supported Functionality
FP8_DYNAMIC through llm-compressor 0.13.0 / compressed-tensors 0.18.0:
per-channel E4M3 weight scales and dynamic per-token E4M3 activations.See provenance, recipe, and the module inventory.
These are retained measurements from August 18, 2026. We compared FP8 with the same model served as BF16 through vLLM using the same evaluation wrapper. They are our paired baseline, not Microsoft's official leaderboard results. This documentation update rechecks the stored scores and fixture identities; it does not report a new inference run.
| Task | BF16 via vLLM | FP8 via vLLM | FP8 − BF16 |
|---|---|---|---|
| STSBenchmark | 0.814828 | 0.815145 | +0.000318 |
| STS17 | 0.809866 | 0.810461 | +0.000595 |
| SciFact | 0.696000 | 0.691260 | -0.004740 |
| NFCorpus | 0.306750 | 0.307030 | +0.000280 |
STSBenchmark and STS17 use their MTEB main scores; STS17 averages 11 subsets. SciFact and NFCorpus use nDCG@10. Mean absolute difference across the four tasks: 0.001483, below the existing 0.005 threshold. Every absolute task difference is below 0.01. BF16 results · FP8 results.
The fixture comparison covers 190 cases: 189 normal and one long-input stress case.
| Fixture metric | Measured | Existing threshold |
|---|---|---|
| Normal-case cosine, 5th percentile | 0.997234 | ≥ 0.995 |
| Normal-case cosine, minimum | 0.992033 | ≥ 0.99 |
| Long-input stress cosine | 0.977244 | ≥ 0.95 |
The retained results pass these thresholds. The 270M fixture thresholds were revised after the initial FP8 observations; they were not an independent preregistered acceptance test. This update preserves the historical thresholds. Per-case results · fixture identities · thresholds.
From a downloaded copy of this repository:
bash
python scripts/validate_evidence.py
The verifier checks evidence hashes, the exact case-ID set, stress-case separation, per-subset score means, and acceptance thresholds. It uses Python's standard library. Fixture source-text hashes and the original golden-embedding hash appear in the fixture manifest. Dataset text, binary embeddings, and full inference/quantization harnesses are not included in this release.
Tests used an RTX 5070 Ti (16 GB), vLLM 0.26.0, llm-compressor 0.13.0, compressed-tensors 0.18.0, and torch 2.11.0+cu130. The fixture engine used BF16 compute, eager execution, and an 8192-token limit. These are the original recorded conditions, not a compatibility guarantee for newer runtimes.
The MTEB summaries retain scores and subsets, but not a complete environment lock or dataset snapshot record. The historical wrapper passes text to vLLM without explicit per-task prompt selection. These measurements should not be treated as full reproduction of Microsoft's evaluation protocol. They establish neither latency nor throughput improvements.
Use a vLLM installation with support for this compressed-tensors FP8 format and compatible GPU kernels. The usage below follows the tested 0.26.0 serving path.
python
from vllm import LLMllm = LLM(model="shadowrock-io/Harrier-OSS-v1-270M-Community-FP8",runner="pooling",trust_remote_code=True,dtype="bfloat16",max_model_len=8192,)documents = ["The observatory opens its roof after sunset."]query = ("Instruct: Given a web search query, retrieve relevant passages that answer the query\n""Query: when does the observatory open its roof")document_vectors = llm.embed(documents)query_vectors = llm.embed([query])
Apply the upstream instruction prefix to queries once; passages need no prefix. vLLM reads last-token pooling and L2 normalization from the supplied configuration. Trailing whitespace can change the last token and therefore the embedding.
Use the embeddings for retrieval, semantic similarity, clustering, and related applications covered by the upstream model card. Microsoft's intended-use and safety guidance still applies. Our measurements cover only the cases and tasks disclosed above.
Quantization, validation, and packaging by Matt Busi at ShadowRock. Cite Microsoft's original model for the architecture and training.
MIT. See LICENSE and the upstream model's license declaration.
ShadowRock is an AI-specialized systems integrator and Zendesk Premier Partner. We help businesses get real value from their go-to-market technology, from CRM and support platforms to applied AI like the models in this collection. Find us at shadowrock.io or on LinkedIn.