GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
putheng
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
putheng
Model Tree
Qwen/Qwen2.5-Coder-1.5B-Instruct
Input Modalities
Output Modalities
Supported Functionality
This is a complete, standalone 4-bit quantized model (LoRA weights merged into Qwen/Qwen2.5-Coder-1.5B-Instruct) that handles 2.0 MILLION token contexts through curriculum learning.
Part of the Ellora project - Recipe #4: Progressive Long Context Extension.
The model was trained progressively through these stages:
python
from unsloth import FastLanguageModelfrom transformers import TextStreamer # Load model with Unsloth (automatically handles 2M context!)model, tokenizer = FastLanguageModel.from_pretrained( model_name="putheng/qwen2-5-coder-1-5b-instruct-progressive-2000k", max_seq_length=2000000, dtype=None, # Auto-detect load_in_4bit=True,) # Enable native fast generationFastLanguageModel.for_inference(model) # Example: Analyze a large codebase (uses the model's chat template)messages = [ {"role": "system", "content": "You are a helpful coding assistant that understands large code repositories."}, {"role": "user", "content": "Repository Context: [Your repository content up to 2000K tokens] Question: Analyze the overall architecture and provide improvement suggestions."},]prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) inputs = tokenizer(prompt, return_tensors="pt", truncation=True, max_length=2000000)streamer = TextStreamer(tokenizer) outputs = model.generate( **inputs, streamer=streamer, max_new_tokens=1024, temperature=0.7, do_sample=True)python
from transformers import AutoModelForCausalLM, AutoTokenizerimport torch # Load the merged 4-bit model directly - no PEFT adapter neededmodel = AutoModelForCausalLM.from_pretrained( "putheng/qwen2-5-coder-1-5b-instruct-progressive-2000k", load_in_4bit=True, # Saved as a 4-bit quantized model (requires bitsandbytes) torch_dtype=torch.bfloat16, device_map="auto", trust_remote_code=True, attn_implementation="flash_attention_2") # Load tokenizertokenizer = AutoTokenizer.from_pretrained("putheng/qwen2-5-coder-1-5b-instruct-progressive-2000k") # Now you can use contexts up to 2000K tokens!This model was trained using a novel progressive curriculum approach with hybrid optimizations:
Each stage included data from all previous stages, allowing the model to maintain and build upon its learned capabilities.
yaml
Progressive Stages: 32K → 128K → 512K → 2000KFinal Context: 2000K tokensBase Model: Qwen/Qwen2.5-Coder-1.5B-InstructData Generation: vLLM (fast batch inference)Training: Unsloth (memory-efficient training)LoRA Rank: 64LoRA Alpha: 128Learning Rate: 0.0002Batch Size: 1Gradient Accumulation: 4The model excels at:
This model is part of the Ellora project - standardized recipes for enhancing LLM capabilities.
from unsloth import FastLanguageModelfrom transformers import TextStreamer # Load model with Unsloth (automatically handles 2M context!)model, tokenizer = FastLanguageModel.from_pretrained( model_name="putheng/qwen2-5-coder-1-5b-instruct-progressive-2000k", max_seq_length=2000000, dtype=None, # Auto-detect load_in_4bit=True,) # Enable native fast generationFastLanguageModel.for_inference(model) # Example: Analyze a large codebase (uses the model's chat template)messages = [ {"role": "system", "content": "You are a helpful coding assistant that understands large code repositories."}, {"role": "user", "content": "Repository Context: [Your repository content up to 2000K tokens] Question: Analyze the overall architecture and provide improvement suggestions."},]prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) inputs = tokenizer(prompt, return_tensors="pt", truncation=True, max_length=2000000)streamer = TextStreamer(tokenizer) outputs = model.generate( **inputs, streamer=streamer, max_new_tokens=1024, temperature=0.7, do_sample=True)from transformers import AutoModelForCausalLM, AutoTokenizerimport torch # Load the merged 4-bit model directly - no PEFT adapter neededmodel = AutoModelForCausalLM.from_pretrained( "putheng/qwen2-5-coder-1-5b-instruct-progressive-2000k", load_in_4bit=True, # Saved as a 4-bit quantized model (requires bitsandbytes) torch_dtype=torch.bfloat16, device_map="auto", trust_remote_code=True, attn_implementation="flash_attention_2") # Load tokenizertokenizer = AutoTokenizer.from_pretrained("putheng/qwen2-5-coder-1-5b-instruct-progressive-2000k") # Now you can use contexts up to 2000K tokens!Progressive Stages: 32K → 128K → 512K → 2000KFinal Context: 2000K tokensBase Model: Qwen/Qwen2.5-Coder-1.5B-InstructData Generation: vLLM (fast batch inference)Training: Unsloth (memory-efficient training)LoRA Rank: 64LoRA Alpha: 128Learning Rate: 0.0002Batch Size: 1Gradient Accumulation: 4