GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
tinyopsec
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
tinyopsec
Model Tree
ibm-granite/granite-4.2-3b
Input Modalities
Output Modalities
Supported Functionality
ibm-granite/granite-4.2-3bGraniteForCausalLMtinyopsec/granite-4.2-3b-HereticThe weights may be split into multiple .safetensors shard files. This is
expected and is handled automatically by Transformers through the model index
file.
bash
pip install -U transformers accelerate safetensors torch
python
import torchfrom transformers import AutoTokenizer, AutoModelForCausalLMmodel_id = "tinyopsec/granite-4.2-3b-Heretic"tokenizer = AutoTokenizer.from_pretrained(model_id,trust_remote_code=True,)model = AutoModelForCausalLM.from_pretrained(model_id,dtype=torch.float16,device_map="auto",trust_remote_code=True,)messages = [{"role": "user","content": "Explain the benefits and limitations of renewable energy.",}]inputs = tokenizer.apply_chat_template(messages,add_generation_prompt=True,return_tensors="pt",).to(model.device)with torch.inference_mode():outputs = model.generate(inputs,max_new_tokens=256,do_sample=True,temperature=0.7,top_p=0.9,repetition_penalty=1.05,pad_token_id=tokenizer.eos_token_id,)new_tokens = outputs[0][inputs.shape[-1]:]print(tokenizer.decode(new_tokens,skip_special_tokens=True,))
For hardware where float16 is not appropriate, use automatic dtype selection:
python
from transformers import AutoTokenizer, AutoModelForCausalLMmodel_id = "tinyopsec/granite-4.2-3b-Heretic"tokenizer = AutoTokenizer.from_pretrained(model_id,trust_remote_code=True,)model = AutoModelForCausalLM.from_pretrained(model_id,dtype="auto",device_map="auto",trust_remote_code=True,)
CPU inference is possible but considerably slower and may require substantial system RAM:
python
from transformers import AutoTokenizer, AutoModelForCausalLMmodel_id = "tinyopsec/granite-4.2-3b-Heretic"tokenizer = AutoTokenizer.from_pretrained(model_id,trust_remote_code=True,)model = AutoModelForCausalLM.from_pretrained(model_id,dtype="auto",device_map="cpu",trust_remote_code=True,)
This model is intended for:
This model has not been comprehensively evaluated after editing. In particular, it may differ from the base model in:
The model may produce incorrect, unsafe, biased, or misleading content. Its outputs must be reviewed by a human before being used in applications, published, or relied upon for consequential decisions.
The model should not be used as an autonomous authority, safety filter, medical advisor, legal advisor, financial advisor, or substitute for professional judgment.
When evaluating this model, compare it with the original base model using the same:
Recommended checks include:
A useful baseline is:
python
BASE_MODEL_ID = "ibm-granite/granite-4.2-3b"EDITED_MODEL_ID = "tinyopsec/granite-4.2-3b-Heretic"
The edited model was generated from the Granite 4.2 3B base model using a Heretic-style optimization workflow.
For exact reproduction, record and publish:
The final model should be treated as an experimental derivative until those details and a broader evaluation are published.
This repository contains modified weights. It is not the original IBM Granite release.
Please read the original model card before using this model:
The base model's license, attribution requirements, acceptable-use rules, and other conditions remain relevant.
This repository is released under the Apache License 2.0, subject to the license and usage conditions of the base model.
See LICENSE and the original
Granite model card for additional information.