GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
HiTZ
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
HiTZ
Model Tree
meta-llama/Llama-3.1-8B
Input Modalities
Output Modalities
Supported Functionality
meta-llama/Llama-3.1-8BThe labels are represented as Python classes, and the guidelines or instructions are introduced as docstrings. The model start generating after the result = [ line.
Python
# Entity definitions@dataclassclass Launcher(Template): """Refers to a vehicle designed primarily to transport payloads from the Earth's surface to space. Launchers can carry various payloads, including satellites, crewed spacecraft, and cargo, into various orbits or even beyond Earth's orbit. They are usually multi-stage vehicles that use rocket engines for propulsion.""" mention: str """ The name of the launcher vehicle. Such as: "Sturn V", "Atlas V", "Soyuz", "Ariane 5" """ space_company: str # The company that operates the launcher. Such as: "Blue origin", "ESA", "Boeing", "ISRO", "Northrop Grumman", "Arianespace" crew: List[str] # Names of the crew members boarding the Launcher. Such as: "Neil Armstrong", "Michael Collins", "Buzz Aldrin" @dataclassclass Mission(Template): """Any planned or accomplished journey beyond Earth's atmosphere with specific objectives, either crewed or uncrewed. It includes missions to satellites, the International Space Station (ISS), other celestial bodies, and deep space.""" mention: str """ The name of the mission. Such as: "Apollo 11", "Artemis", "Mercury" """ date: str # The start date of the mission departure: str # The place from which the vehicle will be launched. Such as: "Florida", "Houston", "French Guiana" destination: str # The place or planet to which the launcher will be sent. Such as "Moon", "low-orbit", "Saturn" # This is the text to analyzetext = ( "The Ares 3 mission to Mars is scheduled for 2032. The Starship rocket build by SpaceX will take off from Boca Chica," "carrying the astronauts Max Rutherford, Elena Soto, and Jake Martinez.") # The annotation instances that take place in the text above are listed hereresult = [ Mission(mention='Ares 3', date='2032', departure='Boca Chica', destination='Mars'), Launcher(mention='Starship', space_company='SpaceX', crew=['Max Rutherford', 'Elena Soto', 'Jake Martinez'])]Please read our 🚀 Example Jupyter Notebooks to get started with GuideX.
The best way to load the model is using our custom load_model fuction. However, you can also load them using the AutoModelForCausalLM class.
Important: Our flash attention implementation has small numerical differences compared to the attention implementation in Huggingface.
You must use the flag trust_remote_code=True or you will get inferior results. Flash attention requires an available CUDA GPU. Running GuideX
pre-trained models on a CPU is not supported. We plan to address this in future releases. First, install flash attention 2:
bash
pip install flash-attn --no-build-isolationpip install git+https://github.com/HazyResearch/flash-attention.git#subdirectory=csrc/rotaryThen you can load the model using
python
import torchfrom transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("HiTZ/Llama-3.1-GuideX-8B")model = AutoModelForCausalLM.from_pretrained("HiTZ/Llama-3.1-GuideX-8B", trust_remote_code=True, torch_dtype=torch.bfloat16)model.to("cuda")Read our 🚀 Example Jupyter Notebooks to learn how to easily define guidelines, generate model inputs and parse the output!
text
## Citation@misc{delafuente2025guidexguidedsyntheticdata, title={GuideX: Guided Synthetic Data Generation for Zero-Shot Information Extraction}, author={Neil De La Fuente and Oscar Sainz and Iker García-Ferrero and Eneko Agirre}, year={2025}, eprint={2506.00649}, archivePrefix={arXiv}, primaryClass={cs.CL}, url={https://arxiv.org/abs/2506.00649}, }# Entity definitions@dataclassclass Launcher(Template): """Refers to a vehicle designed primarily to transport payloads from the Earth's surface to space. Launchers can carry various payloads, including satellites, crewed spacecraft, and cargo, into various orbits or even beyond Earth's orbit. They are usually multi-stage vehicles that use rocket engines for propulsion.""" mention: str """ The name of the launcher vehicle. Such as: "Sturn V", "Atlas V", "Soyuz", "Ariane 5" """ space_company: str # The company that operates the launcher. Such as: "Blue origin", "ESA", "Boeing", "ISRO", "Northrop Grumman", "Arianespace" crew: List[str] # Names of the crew members boarding the Launcher. Such as: "Neil Armstrong", "Michael Collins", "Buzz Aldrin" @dataclassclass Mission(Template): """Any planned or accomplished journey beyond Earth's atmosphere with specific objectives, either crewed or uncrewed. It includes missions to satellites, the International Space Station (ISS), other celestial bodies, and deep space.""" mention: str """ The name of the mission. Such as: "Apollo 11", "Artemis", "Mercury" """ date: str # The start date of the mission departure: str # The place from which the vehicle will be launched. Such as: "Florida", "Houston", "French Guiana" destination: str # The place or planet to which the launcher will be sent. Such as "Moon", "low-orbit", "Saturn" # This is the text to analyzetext = ( "The Ares 3 mission to Mars is scheduled for 2032. The Starship rocket build by SpaceX will take off from Boca Chica," "carrying the astronauts Max Rutherford, Elena Soto, and Jake Martinez.") # The annotation instances that take place in the text above are listed hereresult = [ Mission(mention='Ares 3', date='2032', departure='Boca Chica', destination='Mars'), Launcher(mention='Starship', space_company='SpaceX', crew=['Max Rutherford', 'Elena Soto', 'Jake Martinez'])]pip install flash-attn --no-build-isolationpip install git+https://github.com/HazyResearch/flash-attention.git#subdirectory=csrc/rotaryimport torchfrom transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("HiTZ/Llama-3.1-GuideX-8B")model = AutoModelForCausalLM.from_pretrained("HiTZ/Llama-3.1-GuideX-8B", trust_remote_code=True, torch_dtype=torch.bfloat16)model.to("cuda")## Citation@misc{delafuente2025guidexguidedsyntheticdata, title={GuideX: Guided Synthetic Data Generation for Zero-Shot Information Extraction}, author={Neil De La Fuente and Oscar Sainz and Iker García-Ferrero and Eneko Agirre}, year={2025}, eprint={2506.00649}, archivePrefix={arXiv}, primaryClass={cs.CL}, url={https://arxiv.org/abs/2506.00649}, }