GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
farbodtavakkoli
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
farbodtavakkoli
Model Tree
Input Modalities
Output Modalities
Supported Functionality
This repository contains the OTel 2.0 QLoRA checkpoint. Its weights may be updated as model development continues. Use a pinned model revision, checkpoint hash, or release tag for reproducible evaluation and production deployment.
| Attribute | Value |
|---|---|
| Base model | Gemma 4 31B-IT (trained on its 4-bit NF4 quant, unsloth/gemma-4-31b-it-unsloth-bnb-4bit) |
| Parameters | 31B |
| Model family | OTel 2.0 |
| Training method | Telecom domain post-training via QLoRA (LoRA adapter on a 4-bit base, Unsloth) |
| Adapter config | LoRA rank r=256, alpha=512, dropout 0.0, targets all-linear |
| Released format | Merged bf16 HuggingFace checkpoint (Gemma4ForConditionalGeneration), ~62.5 GB |
| Raw telecom corpus | ~15B tokens from GSMA through Open Telco AI |
| Processed data volume | >1T tokens processed using Red Hat's open-source Synthetic Data Generation Hub (SDG Hub) |
| Training tokens | ~440B |
| Compute for data processing | ~530 GPUs through Microsoft Azure Managed Compute, primarily AMD MI300X |
| Model training infrastructure | On-premises AMD MI355X GPUs with Dell Technologies infrastructure and servers |
| Day 0 inference availability | Microsoft Foundry, Featherless AI, and Red Hat |
| Supported interaction | English-language text (see Supported Language and Modality) |
| Architecture modality | Multimodal (text + image), inherited from Gemma 4; only the text path was trained |
Gemma 4 31B-IT -> 4-bit NF4 quantization -> OTel 2.0 telecom data processing and QLoRA post-training -> merge (dequantize NF4→bf16 + fold LoRA) -> OTel-2.0-LLM-31B-IT-QLoRA
| Category | Metric | OTel 1.0 | OTel 2.0 |
|---|---|---|---|
| Tokens | Raw documents | ~600M | ~15B |
| Tokens | Training | ~1B | ~440B |
| Data preparation | General-purpose agentic tool calling | No | Yes |
| Data preparation | Knowledge / fact direct QnA | No | Yes |
Relative to OTel 1.0, OTel 2.0 increases raw source coverage by roughly 25x and training-token volume by roughly 440x. More importantly, the data mixture is broader: OTel 1.0 focused primarily on context-grounded RAG and abstention, while OTel 2.0 adds direct telecom knowledge QnA, general-purpose instruction-following and tool-use-oriented examples, and base-model-style telecom training data. Telecommunications-specific MCP, tool-calling, and instruction-following examples are not part of the current training mixture.
The starting corpus contains approximately 15 billion raw tokens provided by GSMA through Open Telco AI. The corpus includes telecom standards and technical material from seven standards development and industry organizations:
| Source family | Examples |
|---|---|
| Cellular standards | 3GPP |
| Telecom standards and specifications | ETSI, ITU |
| Industry and operator materials | GSMA |
| Network API specifications | CAMARA |
| Open RAN specifications | O-RAN |
| Telecom business and operations frameworks | TM Forum |
Dense technical specifications from these sources were converted into material suitable for model training. The raw corpus was processed into over 1 trillion tokens using Red Hat's open-source Synthetic Data Generation Hub (SDG Hub) on Microsoft Azure Managed Compute, using approximately 530 GPUs, primarily AMD MI300X. From this processed pool, approximately 440 billion training tokens were generated for OTel 2.0 post-training. Model training ran on on-premises AMD MI355X GPUs with Dell Technologies infrastructure and servers.
OTel 2.0 includes a broader supervised and synthetic-data mixture than OTel 1.0:
| Capability | Description |
|---|---|
| RAG | Context-grounded telecom answer generation from retrieved standards and technical documents |
| Abstention | Training examples that teach the model to avoid answering when context is missing, irrelevant, or insufficient |
| Direct telecom QnA | Knowledge and factual question-answer pairs for standards, protocols, services, and network concepts |
| Instruction following | General-purpose instruction-following examples; the current mixture does not include telecommunications-specific instruction-following examples |
| Agentic tool calling | General-purpose tool-calling examples; the current mixture does not include telecommunications-specific MCP or tool-calling examples |
OTel-2.0-LLM-31B-IT-QLoRA is intended for telecom-focused applications where domain knowledge, standards familiarity, and deployment control matter. Suitable use cases include:
For high-impact operational use, the model should be deployed with retrieval, source attribution, validation checks, and human review appropriate to the system risk.
This model was produced with parameter-efficient fine-tuning (QLoRA) and released as a merged bf16 checkpoint.
r=256, alpha=512, dropout 0.0, all-linear targets) trained on the 4-bit NF4 base unsloth/gemma-4-31b-it-unsloth-bnb-4bit with Unsloth. During training the base is dequantized from NF4 in the forward pass, so the effective base is dequant(nf4(W)).dequant(nf4(W)) + B·A), not the pristine bf16 base. The adapter learned to correct the quantized base, so merging onto the pristine bf16 base degrades quality; the correct procedure dequantizes each Linear4bit to bf16 first and then folds the deltas (note that calling merge_and_unload() directly on a 4-bit base re-quantizes the result).Gemma4ForConditionalGeneration, , ~62.5 GB) loadable with — no bitsandbytes, PEFT, or Unsloth needed at inference.Requirements
Attempted to load weight (512) into parameter (256). If you hit this, upgrade to a vLLM release that lists Gemma 4 support. vLLM pins its own transformers/torch, so a matching stack is installed for you.Serve
bash
# 2-GPU tensor parallelism; adjust --max-model-len / --gpu-memory-utilization to your hardwareCUDA_VISIBLE_DEVICES=0,1 vllm serve <model_dir_or_hf_repo> \--served-model-name otel-2.0-llm-31b-it-qlora \--host 0.0.0.0 --port 8000 \--tensor-parallel-size 2 \--max-model-len 8192 \--gpu-memory-utilization 0.90
Verify
bash
curl http://localhost:8000/v1/models # -> otel-2.0-llm-31b-it-qlora
Inference
Query with /v1/completions using a prompt built from the chat template (see Prompt Format). Use add_special_tokens: false (the chat template already prepends <bos>; a second one corrupts the prompt), and stop token ids [1, 106] (<eos> and end-of-turn). For deterministic output use greedy decoding (temperature: 0).
bash
curl http://localhost:8000/v1/completions -H 'Content-Type: application/json' -d '{"model": "otel-2.0-llm-31b-it-qlora","prompt": "<bos><|turn>user\nWhat model are you?<turn|>\n<|turn>model\n","max_tokens": 256,"temperature": 0,"add_special_tokens": false,"stop_token_ids": [1, 106]}'
OTel 2.0 uses the Gemma 4 chat template. Build prompts with apply_chat_template(messages, add_generation_prompt=True); when sending a raw prompt to /v1/completions, trim it to end exactly at the <|turn>model\n marker so the model generates the reply directly.
No system prompt is required. The model works directly from the user turn, including knowing its own identity:
markdown
Input:What model are you?Output:I'm OTel 2.0, a large language model trained by AT&T Chief Data Office.
You can pass your own system prompt as the first message when your application needs one; it is used as-is with no special handling.
OTel 2.0 is post-trained and supported for English-language text interactions.
The released checkpoint is architecturally multimodal: it is a Gemma4ForConditionalGeneration model and retains the full Gemma 4 vision encoder inherited from the base model, so it will structurally accept image inputs alongside text. However, only the text/language pathway was trained. OTel 2.0 post-training used a LoRA adapter applied exclusively to the text/language layers on a text-only telecom corpus; the vision encoder was frozen and carries the untrained, stock Gemma 4 weights with no OTel/telecom adaptation.
As a result:
For production use, drive the model with English text; do not rely on image inputs without your own task-specific evaluation.
The training data includes general-purpose tool-calling and instruction-following examples, but it does not currently include:
General-purpose tool-use training should not be interpreted as readiness to operate telecommunications tools autonomously. This includes multi-step tasks where a scorer measures the resulting network state rather than the text of a response; general-purpose tool-use training does not establish competence at such closed-loop operational workflows (see Autonomous Operational Task Completion Not Benchmarked). Agentic deployment requires an external tool runtime, validated tool schemas, permission controls, audit logging, safeguards, and human review appropriate to the risk. The model alone does not guarantee correct or safe tool execution.
The current training mixture does not include the following classes of operational telecommunications data:
Standards knowledge should not be interpreted as experience with live operator telemetry, private network records, vendor-specific CLI/NOS behavior, or these excluded operational datasets. Applications involving these data types require separate task-specific evaluation, grounding, and validation.
The model has not been validated against Methods of Procedure (MOPs) — step-by-step operational runbooks (e.g., maintenance activities, configuration changes, turn-ups, failovers, troubleshooting sequences) — executed or emulated against live or digital-twin network device instances. No benchmarking has been performed to confirm that the model's step-by-step procedural reasoning (command sequencing, expected outputs, checkpoints, rollback steps) is correct or safe when applied to real or emulated network operating system instances. Standards and configuration-assistance knowledge demonstrated elsewhere in this model card should not be interpreted as validated readiness for autonomous or human-in-the-loop execution of operational procedures. Any such use requires dedicated MOP-based evaluation (e.g., against emulated NOS instances in a digital twin environment) before deployment.
The model has not been evaluated for autonomous, closed-loop completion of operational telecom tasks whose success is measured by the resulting state of a live or emulated network, rather than by the text of a response. Task families of this kind — extracting quantitative SLA/QoS parameters from a natural-language service request; bringing up and configuring a private 5G standalone core and RAN to meet measured latency and throughput targets; diagnosing and repairing a live routing fault through an alarm-to-closure workflow; and standing up a network defense that must withstand adversarial traffic — exercise capabilities the current training mixture targets but has not validated. Demonstrated standards knowledge and configuration assistance should not be read as readiness for autonomous completion of these tasks, which requires task-specific evaluation with outcome-based scoring before any non-supervised use.
The model has not been trained or evaluated for defensive network-security operations — traffic and anomaly analysis, firewall/packet-filter or eBPF program construction, DDoS detection and mitigation, or deployment of long-running protective processes on network appliances. Security-operations use requires dedicated evaluation and human oversight.
OTel-2.0-LLM-31B-IT.The following work is planned and should not be interpreted as a capability claim for the current checkpoint:
Contributors to the OTel 2.0 release and supporting infrastructure include:
| Organization | Contributors |
|---|---|
| AT&T | Farbod Tavakkoli, Jorden Terrazas, Roderic Paulk, Sharath Japa, Tzvi Chumash, Miguel Armenta, Pavan Tagirisa, Kostikey Mustakas, Mark Austin, Andy Markus |
| MLCommons | Gregory Diamos, David Kanter, Kenneth Church |
| Microsoft | Chunyu Li, Gulsimo Osimi, Rick Lievano, Inayat Wali, Manoj Bableshwar, Marie-Louise Onga Nana, Naomi Moneypenny, Osi Otugo, Rahul Kumar, SeokJin Han, Steve Sweetman, Trinidad Salazar, Ven Kumar, Vivek Ramaswamy |
| AMD | Alexander Finn, Andy Allred, Andrey Ivannikov, Antti-Ville Suni, Mark van Heeswijk, Kumaran Siva, Curt Wortman, Mehrvash Poole, Eric Lynn |
| Dell | Brian Sweeney, Suzanne Randall, Patrick Allen, Matt Currie, Justin Wilson, Jason Kane, Keith Napoleon, Sarah Lake, Mike Hess, Randy Tornes |
| GSMA | Louis Powell, Zeinab Nezami, Enrique Molero |
| Red Hat | Aditi Saluja, William Caban, Shivchander Sudalairaj, Kai Xu, Ravi Sharma, Hanen Garcia, Joe Crispo, Eshwar Sivaramakrishnan |
Organizations involved in the OTel 2.0 release and supporting infrastructure include AT&T, MLCommons, Microsoft, AMD, Dell, GSMA, Red Hat, and Pleias.
bibtex
@misc{otel_2_models_2026,title = {OTel 2.0: Open Telco AI Datasets, Benchmarks, and Models},author = {Tavakkoli, Farbod and others},year = {2026},note = {Open Telco (OTel 2.0) model release},url = {https://huggingface.co/farbodtavakkoli},organization = {AT\&T, MLCommons, Microsoft, AMD, Dell, GSMA, Red Hat, Pleias}}
For technical questions, contact farbod.tavakkoli@att.com or farbodtavakoli@gmail.com.
| Data preparation | RAG | Yes | Yes |
| Data preparation | Abstention | Yes | Yes |
| Data preparation | General-purpose instruction following | No | Yes |
| Data preparation | Base model training set | No | Yes |
quantization_config: nullAutoModelForCausalLM.from_pretrained| Pleias | Anastasia Stasenko, Pierre-Carl Langlais, Mohamed Chenene, Carlos Rosas, Yannick Detrois |