GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
secemp9
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
secemp9
Model Tree
unsloth/Mistral-Nemo-Instruct-2407-bnb-4bit
Input Modalities
Output Modalities
Supported Functionality
TraceBack is what I came up with when I thought, "how can we scale reasoning trace data generation effectively?"
Turn out you do not need to depend on just reasoning models (r1, o1, o3, etc) to create reasoning trace!
It has many goals in mind, but mainly:
So far, current proof of concept managed to check the boxes for 1 and 3, and I plan on scaling this more as:
So there are still much room for improvement
This was trained using both instruction and solution as input, and the output being a plausible/possible/matching reasoning trace based on that.
I believe this is the future of reasoning data generation. Stay tuned for an eval release
Here some inference example, using chatgpt instruction + solution as input:
Here I use a simple example from chatgpt, passing both the instruction and the solution as input to the model:
Here the format for the dataset follow instruction + solution: reasoning trace pairs Sample conversation:
text
{ "messages": [ { "role": "user", "content": "Instruction: text_here Solution: text_here }, { "role": "assistant", "content": "text_here" } ]}which look like:
For the prompt format, I was really trying to not overengineer, but I'm sure there is a better way to format this.
For now it's just: Instruction:
Solution:
the output of the model doesn't have (for now) any formatting, it's just reasoning as output
python
from transformers import AutoModelForCausalLM, AutoTokenizerimport torch # Load the tokenizer and modelmodel_name = "secemp9/TraceBack-12b"tokenizer = AutoTokenizer.from_pretrained(model_name)model = AutoModelForCausalLM.from_pretrained(model_name) # Move the model to the desired devicedevice = 'cuda' if torch.cuda.is_available() else 'cpu'model.to(device) # Define the messagesmessages = [ {"role": "user", "content": """Instruction:how many r in strawberry Solution:There are **three** "r"s in "strawberry.""""}] # Step 1: Apply chat template to get formatted text as a stringformatted_text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) # Step 2: Tokenize the formatted text into a dictionary of tensorsinputs = tokenizer(formatted_text, return_tensors="pt").to(device) # Generate the responseoutputs = model.generate(**inputs, max_new_tokens=32000) # Decode and print the outputgenerated_text = tokenizer.decode(outputs[0])print(generated_text)python
from unsloth import FastLanguageModel # Load the model and tokenizermodel, tokenizer = FastLanguageModel.from_pretrained("secemp9/TraceBack-12b") # Define the messages (replace "stuff_here" with your actual input)messages = [ {"role": "user", "content": """Instruction:how many r in strawberry Solution:There are **three** "r"s in "strawberry.""""}] # Step 1: Apply chat template to get formatted text as a stringformatted_text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) # Step 2: Tokenize the formatted text into a dictionary of tensorsinputs = tokenizer(formatted_text, return_tensors="pt").to(model.device) # Generate the responseoutputs = model.generate(**inputs, max_new_tokens=32000) # Decode and print the outputgenerated_text = tokenizer.decode(outputs[0])print(generated_text)For this, I basically tried to convert my unsloth code to an axolotl config file. I also used deepspeed. Configuration below:
config.yml
text
# Base model configurationbase_model: unsloth/Mistral-Nemo-Instruct-2407-bnb-4bitload_in_4bit: true # Dataset configurationdatasets: - path: instruction_solution_to_thought_dataset.jsonl type: chat_template # Chat templatechat_template: chatml # LoRA adapter configurationadapter: loralora_r: 16lora_alpha: 16lora_dropout: 0lora_target_modules: - q_proj - k_proj - v_proj - o_proj - gate_proj - up_proj - down_proj # Training hyperparametersmax_seq_length: 128000micro_batch_size: 2gradient_accumulation_steps: 8learning_rate: 3e-5num_epochs: 3warmup_steps: 100optimizer: adamw_8bitweight_decay: 0.01lr_scheduler_type: cosinemax_grad_norm: 1.0output_dir: ./outputs_solution_to_thoughtseed: 3407merge_lora: truehf_upload: truehf_repo: secemp9/TraceBack-12bxformers_attention:flash_attention: Truebf16: true # Enable BF16 mixed precision# Multi-GPU training with DeepSpeeddeepspeed: deepspeed_configs/zero2.json # Optional: Enable gradient checkpointinggradient_checkpointing: truedeepspeed_configs/zero2.json
text
{ "zero_optimization": { "stage": 2, "allgather_partitions": true, "allgather_bucket_size": 2e8, "overlap_comm": true, "reduce_scatter": true, "reduce_bucket_size": 2e8, "contiguous_gradients": true }, "bf16": { "enabled": true }, "optimizer": { "type": "AdamW", "params": { "lr": "auto", "weight_decay": "auto", "betas": [0.9, 0.999], "eps": 1e-8 } }, "scheduler": { "type": "WarmupLR", "params": { "warmup_min_lr": 0, "warmup_max_lr": "auto", "warmup_num_steps": "auto" } }, "train_micro_batch_size_per_gpu": "auto", "gradient_accumulation_steps": "auto", "steps_per_print": 10, "wandb": { "enabled": true }}{ "messages": [ { "role": "user", "content": "Instruction: text_here Solution: text_here }, { "role": "assistant", "content": "text_here" } ]}# Base model configurationbase_model: unsloth/Mistral-Nemo-Instruct-2407-bnb-4bitload_in_4bit: true# Dataset configurationdatasets: - path: instruction_solution_to_thought_dataset.jsonl type: chat_template# Chat templatechat_template: chatml# LoRA adapter configurationadapter: loralora_r: 16lora_alpha: 16lora_dropout: 0lora_target_modules: - q_proj - k_proj - v_proj - o_proj - gate_proj - up_proj - down_proj# Training hyperparametersmax_seq_length: 128000micro_batch_size: 2gradient_accumulation_steps: 8learning_rate: 3e-5num_epochs: 3warmup_steps: 100optimizer: adamw_8bitweight_decay: 0.01lr_scheduler_type: cosinemax_grad_norm: 1.0output_dir: ./outputs_solution_to_thoughtseed: 3407merge_lora: truehf_upload: truehf_repo: secemp9/TraceBack-12bxformers_attention:flash_attention: Truebf16: true # Enable BF16 mixed precision# Multi-GPU training with DeepSpeeddeepspeed: deepspeed_configs/zero2.json# Optional: Enable gradient checkpointinggradient_checkpointing: true{ "zero_optimization": { "stage": 2, "allgather_partitions": true, "allgather_bucket_size": 2e8, "overlap_comm": true, "reduce_scatter": true, "reduce_bucket_size": 2e8, "contiguous_gradients": true }, "bf16": { "enabled": true }, "optimizer": { "type": "AdamW", "params": { "lr": "auto", "weight_decay": "auto", "betas": [0.9, 0.999], "eps": 1e-8 } }, "scheduler": { "type": "WarmupLR", "params": { "warmup_min_lr": 0, "warmup_max_lr": "auto", "warmup_num_steps": "auto" } }, "train_micro_batch_size_per_gpu": "auto", "gradient_accumulation_steps": "auto", "steps_per_print": 10, "wandb": { "enabled": true }}from transformers import AutoModelForCausalLM, AutoTokenizerimport torch # Load the tokenizer and modelmodel_name = "secemp9/TraceBack-12b"tokenizer = AutoTokenizer.from_pretrained(model_name)model = AutoModelForCausalLM.from_pretrained(model_name) # Move the model to the desired devicedevice = 'cuda' if torch.cuda.is_available() else 'cpu'model.to(device) # Define the messagesmessages = [ {"role": "user", "content": """Instruction:how many r in strawberry Solution:There are **three** "r"s in "strawberry.""""}] # Step 1: Apply chat template to get formatted text as a stringformatted_text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) # Step 2: Tokenize the formatted text into a dictionary of tensorsinputs = tokenizer(formatted_text, return_tensors="pt").to(device) # Generate the responseoutputs = model.generate(**inputs, max_new_tokens=32000) # Decode and print the outputgenerated_text = tokenizer.decode(outputs[0])print(generated_text)from unsloth import FastLanguageModel # Load the model and tokenizermodel, tokenizer = FastLanguageModel.from_pretrained("secemp9/TraceBack-12b") # Define the messages (replace "stuff_here" with your actual input)messages = [ {"role": "user", "content": """Instruction:how many r in strawberry Solution:There are **three** "r"s in "strawberry.""""}] # Step 1: Apply chat template to get formatted text as a stringformatted_text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) # Step 2: Tokenize the formatted text into a dictionary of tensorsinputs = tokenizer(formatted_text, return_tensors="pt").to(model.device) # Generate the responseoutputs = model.generate(**inputs, max_new_tokens=32000) # Decode and print the outputgenerated_text = tokenizer.decode(outputs[0])print(generated_text)