GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
lukashm
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
lukashm
Model Tree
unsloth/llama-3-8b-bnb-4bit
Input Modalities
Output Modalities
Supported Functionality
unsloth/llama-3-8b-bnb-4bit (Quantized)You can run this model using the unsloth library for faster inference, or standard transformers.
bash
pip install unsloth torch transformerspython
from unsloth import FastLanguageModelimport torch # Load the model and tokenizermodel, tokenizer = FastLanguageModel.from_pretrained( model_name = "lukashm/LitLex-Llama-LT-v1", max_seq_length = 2048, dtype = None, load_in_4bit = True,)FastLanguageModel.for_inference(model) # Define the prompt templatealpaca_prompt = """Below is an instruction that describes a task. Write a response that appropriately completes the request. ### Instruction:{} ### Response:""" # Ask a questionquestion = "Kokia bauda gresia už greičio viršijimą daugiau kaip 50 km/h?"inputs = tokenizer([alpaca_prompt.format(question, "")], return_tensors="pt").to("cuda") # Generate answeroutputs = model.generate(**inputs, max_new_tokens=256, use_cache=True)response = tokenizer.batch_decode(outputs)[0]print(response.split("### Response:\n")[1].replace("<|end_of_text|>", ""))The model was fine-tuned on a custom dataset (ank_dataset.json) derived from the Official Administrative Code of Lithuania (ANK) via e-seimas.lrs.lt.
This model is a Proof of Concept (MVP) intended for educational and research purposes.
from unsloth import FastLanguageModelimport torch # Load the model and tokenizermodel, tokenizer = FastLanguageModel.from_pretrained( model_name = "lukashm/LitLex-Llama-LT-v1", max_seq_length = 2048, dtype = None, load_in_4bit = True,)FastLanguageModel.for_inference(model) # Define the prompt templatealpaca_prompt = """Below is an instruction that describes a task. Write a response that appropriately completes the request. ### Instruction:{} ### Response:""" # Ask a questionquestion = "Kokia bauda gresia už greičio viršijimą daugiau kaip 50 km/h?"inputs = tokenizer([alpaca_prompt.format(question, "")], return_tensors="pt").to("cuda") # Generate answeroutputs = model.generate(**inputs, max_new_tokens=256, use_cache=True)response = tokenizer.batch_decode(outputs)[0]print(response.split("### Response:\n")[1].replace("<|end_of_text|>", ""))