Sandeep4235

qwen2.5-7b-adapter

README

License: apache-2.0

🚀 Model Summary

Qwen2.5-7B-Instruct LoRA Adapter is a 7.61-billion parameter causal language model fine-tuned (via Unsloth QLoRA) from unsloth/Qwen2.5-7B-Instruct-bnb-4bit to enhance instruction-following capabilities on the Alpaca dataset. The adapter targets all linear projection layers (attention + MLP) with rank-16 LoRA updates, producing sharper, more compliant responses to structured instructions while preserving the base model's broad knowledge and reasoning ability.

Mission. BluePatterns AI built this adapter to demonstrate that efficient, single-GPU fine-tuning (NVIDIA T4, 16 GB VRAM) can produce meaningful instruction-following gains over a strong base model — lowering the barrier for developers who want to customize LLMs without enterprise compute budgets.

💡 Model Architecture & Specifications

This adapter inherits the dense Transformer architecture of Qwen2.5-7B-Instruct and applies a LoRA adapter trained via Unsloth's 4-bit QLoRA pipeline.

Table with columns: Specification, Detail
Specification	Detail
Developed by	Sandeep Hipparagi (BluePatterns AI)
Model type	Causal Language Model (Causal-LM)
Base model	`unsloth/Qwen2.5-7B-Instruct-bnb-4bit` (Qwen2.5-7B-Instruct, 4-bit NF4)
Total parameters (base)	7.61B (6.53B non-embedding)
Architecture	Dense Transformer, decoder-only (auto-regressive)
Attention	Grouped-Query Attention (GQA) — 28 Q heads, 4 KV heads
Layers	28
Hidden size	3,584
Intermediate (FFN) size	18,944
Head dimension	128
Positional encoding	RoPE (base frequency 1,000,000)
Normalization	RMSNorm (pre-normalization)
Activation	SwiGLU
QKV bias	Yes (attention QKV bias retained from Qwen2)
Vocabulary size	151,646
Embedding tying	No
Native context length	32,768 tokens (up to 131,072 with YaRN)
Max generation length	8,192 tokens
Fine-tuning method	Unsloth QLoRA (4-bit NF4 quantization)
LoRA rank (r)	16
LoRA alpha	16
LoRA dropout	0
Bias	none
Adapter target modules	`q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj`
Training dataset	`yahma/alpaca-cleaned`
Optimizer	AdamW 8-bit
Learning rate	2e-4
Batch size (per device)	2
Gradient accumulation steps	4 (effective batch size = 8)
Max sequence length (training)	2,048 tokens
Training precision	FP16 (T4 does not natively support bf16)
Gradient checkpointing	Enabled (Unsloth optimized)
Hardware	NVIDIA T4 GPU (16 GB VRAM)
Language(s)	English
License	Apache 2.0

🛠 Quickstart & Usage

Option A — Load and infer with `transformers` + `peft` (recommended)

This approach loads the 4-bit quantized base model and applies the LoRA adapter from this repository.

python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model_id = "unsloth/Qwen2.5-7B-Instruct-bnb-4bit"
adapter_id = "Sandeep4235/qwen2.5-7b-adapter"

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained(base_model_id)

# Load base model in 4-bit with automatic device placement
model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    device_map="auto",
    torch_dtype=torch.float16,
)

# Apply the LoRA adapter
model = PeftModel.from_pretrained(model, adapter_id)
model.eval()

# Build the prompt using Qwen2.5's ChatML chat template
messages = [
    {"role": "system", "content": "You are a helpful, expert AI assistant."},
    {"role": "user", "content": "Explain the concept of quantum entanglement in simple terms."},
]

input_ids = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt",
).to(model.device)

# Generate
with torch.no_grad():
    output_ids = model.generate(
        input_ids,
        max_new_tokens=512,
        do_sample=True,
        temperature=0.7,
        top_p=0.8,
        pad_token_id=tokenizer.eos_token_id,
    )

generated = output_ids[0][input_ids.shape[-1]:]
response = tokenizer.decode(generated, skip_special_tokens=True)
print(response)

Option B — Load and infer with Unsloth (fastest inference)

python
import torch
from unsloth import FastLanguageModel
from transformers import TextStreamer

base_model_id = "unsloth/Qwen2.5-7B-Instruct-bnb-4bit"
adapter_id = "Sandeep4235/qwen2.5-7b-adapter"

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name=base_model_id,
    max_seq_length=2048,
    dtype=None,        # auto-detects FP16 on T4
    load_in_4bit=True,
)

# Load the fine-tuned adapter weights
model.load_adapter(adapter_id)
FastLanguageModel.for_inference(model)  # enable Unsloth's faster inference

messages = [
    {"role": "system", "content": "You are a helpful, expert AI assistant."},
    {"role": "user", "content": "Write a Python function to reverse a linked list."},
]

input_ids = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt",
).to("cuda")

text_streamer = TextStreamer(tokenizer, skip_prompt=True)

_ = model.generate(
    input_ids=input_ids,
    streamer=text_streamer,
    max_new_tokens=512,
    use_cache=True,
    temperature=0.7,
    min_p=0.1,
)

Option C — Merge adapter into base model for standalone deployment

If you want a single standalone model (no adapter loading at runtime), merge the LoRA weights into the base model and save:

python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model_id = "unsloth/Qwen2.5-7B-Instruct-bnb-4bit"
adapter_id = "Sandeep4235/qwen2.5-7b-adapter"
save_path = "./qwen2.5-7b-merged"

tokenizer = AutoTokenizer.from_pretrained(base_model_id)
model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    device_map="auto",
    torch_dtype=torch.float16,
)

# Apply adapter, then merge into base weights
model = PeftModel.from_pretrained(model, adapter_id)
model = model.merge_and_unload()

# Save the merged model
model.save_pretrained(save_path)
tokenizer.save_pretrained(save_path)
print(f"Merged model saved to {save_path}")

📊 Evaluation and Benchmarks

Formal quantitative benchmarks for this adapter (e.g., instruction-following accuracy, MT-Bench, or AlpacaEval scores against the unadapted base) have not yet been published. Performance was assessed through qualitative evaluation during fine-tuning, focusing on response quality, instruction adherence, and format compliance on held-out Alpaca samples.

General Academic Benchmarks (inherited from base)

The following scores reflect the Qwen2.5-7B-Instruct base model as published by the Qwen team and are provided for reference only — they are not measurements of this fine-tuned adapter.

Table with columns: Benchmark, Qwen2.5-7B-Instruct (base), Metric
Benchmark	Qwen2.5-7B-Instruct (base)	Metric
MMLU-Pro	56.3	Pass@1 accuracy
MMLU-redux	75.4	Pass@1 accuracy
GSM8K	91.6	Pass@1 accuracy
HumanEval	84.8	Pass@1
MBPP	79.2	Pass@1
IFeval (strict-prompt)	71.2

These figures are the base model's published results and do not represent this adapter's fine-tuned performance. Independent benchmarking is encouraged.

⚠️ Intended Uses & Limitations

Primary Use Cases

Instruction following — executing structured prompts in the Alpaca format (### Instruction: / ### Response:) with improved compliance over the base model.
Task-oriented text generation — answering questions, writing code, summarizing, and explaining concepts in response to natural-language instructions.
Rapid prototyping — serving as a starting point for further domain-specific fine-tuning on top of a strong, efficient base.
Single-GPU customization — demonstrating viable QLoRA fine-tuning on a single T4 GPU for developers with limited compute access.

Out-of-Scope Uses

Safety-critical decision-making. Do not use for medical diagnosis, legal advice, financial trading, or any automated system where outputs directly affect individual rights or safety.
High-stakes factual lookup. The model may hallucinate facts, citations, or code. Always verify critical information against authoritative sources.
Multilingual tasks beyond English. Training data (Alpaca-cleaned) is English-only; performance in other languages is untested.
Long-context reasoning beyond 2,048 tokens (training limit). While the base model supports up to 128K context, the adapter was trained at 2,048 tokens. Quality may degrade on inputs significantly longer than the training sequence length.

Limitations & Biases

Alpaca dataset biases. The Alpaca dataset reflects the biases of its generation pipeline (originally distilled from text-davinci-003). Responses may inherit stylistic or topical biases from this distribution.
Hallucination risk. Like all LLMs, the model may generate plausible but incorrect information. Always verify factual claims.
Limited training scope. The adapter was trained for a modest number of steps on a general-purpose instruction dataset. It is not a domain expert and should not be expected to match specialized models on specific tasks.
FP16 precision. Training was done in FP16 (T4 hardware limitation) rather than bf16, which may introduce minor numerical differences compared to bf16-trained adapters.
Quantization loss. The 4-bit NF4 base quantization introduces minor quality degradation compared to full-precision weights. Use the merged full-precision model for maximum quality.

🔒 Responsible AI & Safety Alignment

Alignment Techniques

Supervised Fine-Tuning (SFT). The adapter was trained on the Alpaca-cleaned dataset using trl.SFTTrainer. No RLHF or DPO was applied in this release.
Inherited safety alignment. The base model (Qwen2.5-7B-Instruct) already undergoes extensive safety alignment during its post-training stage. This adapter preserves that alignment while adjusting instruction-following behavior.
ChatML template compliance. The adapter is designed to work with Qwen2.5's native ChatML chat template, ensuring consistent formatting and predictable behavior.

Deployment Recommendations

Input validation. Sanitize all user inputs. Reject prompt-injection attempts and malicious instruction patterns.
Output audit. Review generated content for accuracy and appropriateness before use in any public-facing context.
Content filtering. Pair the model with an input/output content filter (e.g., Llama Guard, NeMo Guardrails) for production deployments.
Rate limiting. Deploy behind an API gateway with authentication and rate limiting.
Context length management. While the base model supports 128K context, keep inference inputs within the 2,048-token training window for best quality, or fine-tune with longer sequences for extended-context use cases.

🏋️ Training Details

Training Data

The adapter was fine-tuned on yahma/alpaca-cleaned, a cleaned and deduplicated version of the Stanford Alpaca dataset. The dataset contains ~52,000 instruction-response pairs covering a wide range of tasks including question answering, code generation, creative writing, and summarization.

Training Environment

Table with columns: Parameter, Value
Parameter	Value
Hardware	NVIDIA T4 GPU (16 GB VRAM)
Cloud provider	Google Colab
Fine-tuning framework	Unsloth + `trl.SFTTrainer`
Deep learning framework	PyTorch (FP16)
Gradient checkpointing	Enabled (Unsloth optimized)

🤝 Citation & Acknowledgements

If you use this adapter in your research or product, please cite it as follows:

bibtex
@misc{hipparagi2025qwen25adapter,
  title        = {Qwen2.5-7B-Instruct LoRA Adapter: Efficient Instruction-Following Fine-tuning via QLoRA on a Single T4 GPU},
  author       = {Sandeep Hipparagi},
  organization = {BluePatterns AI},
  year         = {2025},
  url          = {https://huggingface.co/Sandeep4235/qwen2.5-7b-adapter},
  note         = {Fine-tuned from unsloth/Qwen2.5-7B-Instruct-bnb-4bit on the Alpaca-cleaned dataset under Apache 2.0}
}

Acknowledgements

Qwen Team — for open-sourcing the Qwen2.5-7B-Instruct model under Apache 2.0.
Unsloth — for the efficient QLoRA fine-tuning framework that made single-GPU training viable.
Stanford Alpaca / yahma — for the cleaned Alpaca instruction dataset.
Hugging Face — for model hosting and the open ML ecosystem.

📬 Contact

Sandeep Hipparagi — AI Developer, Co-Founder of BluePatterns AI

Hugging Face: Sandeep4235
Organization: BluePatterns AI

BluePatterns AI is focused on bridging the gap between frontier and open-source models — democratizing access to reliable, accessible intelligence for diverse communities.

Built by Sandeep Hipparagi · BluePatterns AI · Apache 2.0

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

Sandeep4235

Model Tree

Base

unsloth/Qwen2.5-7B-Instruct-bnb-4bit

Adapter

this model

Input Modalities

Text

Output Modalities

Text

Supported Functionality

Dedicated EndpointsContainer

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

🚀 Model Summary

💡 Model Architecture & Specifications

This adapter inherits the dense Transformer architecture of Qwen2.5-7B-Instruct and applies a LoRA adapter trained via Unsloth's 4-bit QLoRA pipeline.

Table with columns: Specification, Detail
Specification	Detail
Developed by	Sandeep Hipparagi (BluePatterns AI)
Model type	Causal Language Model (Causal-LM)
Base model	`unsloth/Qwen2.5-7B-Instruct-bnb-4bit` (Qwen2.5-7B-Instruct, 4-bit NF4)
Total parameters (base)	7.61B (6.53B non-embedding)
Architecture	Dense Transformer, decoder-only (auto-regressive)
Attention	Grouped-Query Attention (GQA) — 28 Q heads, 4 KV heads
Layers	28
Hidden size	3,584
Intermediate (FFN) size	18,944
Head dimension	128
Positional encoding	RoPE (base frequency 1,000,000)
Normalization	RMSNorm (pre-normalization)
Activation	SwiGLU
QKV bias	Yes (attention QKV bias retained from Qwen2)
Vocabulary size	151,646
Embedding tying	No
Native context length	32,768 tokens (up to 131,072 with YaRN)
Max generation length	8,192 tokens
Fine-tuning method	Unsloth QLoRA (4-bit NF4 quantization)
LoRA rank (r)	16
LoRA alpha	16
LoRA dropout	0
Bias	none
Adapter target modules	`q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj`
Training dataset	`yahma/alpaca-cleaned`
Optimizer	AdamW 8-bit
Learning rate	2e-4
Batch size (per device)	2
Gradient accumulation steps	4 (effective batch size = 8)
Max sequence length (training)	2,048 tokens
Training precision	FP16 (T4 does not natively support bf16)
Gradient checkpointing	Enabled (Unsloth optimized)
Hardware	NVIDIA T4 GPU (16 GB VRAM)
Language(s)	English
License	Apache 2.0

🛠 Quickstart & Usage

Option A — Load and infer with `transformers` + `peft` (recommended)

This approach loads the 4-bit quantized base model and applies the LoRA adapter from this repository.

python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model_id = "unsloth/Qwen2.5-7B-Instruct-bnb-4bit"
adapter_id = "Sandeep4235/qwen2.5-7b-adapter"

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained(base_model_id)

# Load base model in 4-bit with automatic device placement
model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    device_map="auto",
    torch_dtype=torch.float16,
)

# Apply the LoRA adapter
model = PeftModel.from_pretrained(model, adapter_id)
model.eval()

# Build the prompt using Qwen2.5's ChatML chat template
messages = [
    {"role": "system", "content": "You are a helpful, expert AI assistant."},
    {"role": "user", "content": "Explain the concept of quantum entanglement in simple terms."},
]

input_ids = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt",
).to(model.device)

# Generate
with torch.no_grad():
    output_ids = model.generate(
        input_ids,
        max_new_tokens=512,
        do_sample=True,
        temperature=0.7,
        top_p=0.8,
        pad_token_id=tokenizer.eos_token_id,
    )

generated = output_ids[0][input_ids.shape[-1]:]
response = tokenizer.decode(generated, skip_special_tokens=True)
print(response)

Option B — Load and infer with Unsloth (fastest inference)

python
import torch
from unsloth import FastLanguageModel
from transformers import TextStreamer

base_model_id = "unsloth/Qwen2.5-7B-Instruct-bnb-4bit"
adapter_id = "Sandeep4235/qwen2.5-7b-adapter"

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name=base_model_id,
    max_seq_length=2048,
    dtype=None,        # auto-detects FP16 on T4
    load_in_4bit=True,
)

# Load the fine-tuned adapter weights
model.load_adapter(adapter_id)
FastLanguageModel.for_inference(model)  # enable Unsloth's faster inference

messages = [
    {"role": "system", "content": "You are a helpful, expert AI assistant."},
    {"role": "user", "content": "Write a Python function to reverse a linked list."},
]

input_ids = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt",
).to("cuda")

text_streamer = TextStreamer(tokenizer, skip_prompt=True)

_ = model.generate(
    input_ids=input_ids,
    streamer=text_streamer,
    max_new_tokens=512,
    use_cache=True,
    temperature=0.7,
    min_p=0.1,
)

Option C — Merge adapter into base model for standalone deployment

If you want a single standalone model (no adapter loading at runtime), merge the LoRA weights into the base model and save:

python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model_id = "unsloth/Qwen2.5-7B-Instruct-bnb-4bit"
adapter_id = "Sandeep4235/qwen2.5-7b-adapter"
save_path = "./qwen2.5-7b-merged"

tokenizer = AutoTokenizer.from_pretrained(base_model_id)
model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    device_map="auto",
    torch_dtype=torch.float16,
)

# Apply adapter, then merge into base weights
model = PeftModel.from_pretrained(model, adapter_id)
model = model.merge_and_unload()

# Save the merged model
model.save_pretrained(save_path)
tokenizer.save_pretrained(save_path)
print(f"Merged model saved to {save_path}")

📊 Evaluation and Benchmarks

General Academic Benchmarks (inherited from base)

The following scores reflect the Qwen2.5-7B-Instruct base model as published by the Qwen team and are provided for reference only — they are not measurements of this fine-tuned adapter.

Table with columns: Benchmark, Qwen2.5-7B-Instruct (base), Metric
Benchmark	Qwen2.5-7B-Instruct (base)	Metric
MMLU-Pro	56.3	Pass@1 accuracy
MMLU-redux	75.4	Pass@1 accuracy
GSM8K	91.6	Pass@1 accuracy
HumanEval	84.8	Pass@1
MBPP	79.2	Pass@1
IFeval (strict-prompt)	71.2

These figures are the base model's published results and do not represent this adapter's fine-tuned performance. Independent benchmarking is encouraged.

⚠️ Intended Uses & Limitations

Primary Use Cases

Instruction following — executing structured prompts in the Alpaca format (### Instruction: / ### Response:) with improved compliance over the base model.
Task-oriented text generation — answering questions, writing code, summarizing, and explaining concepts in response to natural-language instructions.
Rapid prototyping — serving as a starting point for further domain-specific fine-tuning on top of a strong, efficient base.
Single-GPU customization — demonstrating viable QLoRA fine-tuning on a single T4 GPU for developers with limited compute access.

Out-of-Scope Uses

Safety-critical decision-making. Do not use for medical diagnosis, legal advice, financial trading, or any automated system where outputs directly affect individual rights or safety.
High-stakes factual lookup. The model may hallucinate facts, citations, or code. Always verify critical information against authoritative sources.
Multilingual tasks beyond English. Training data (Alpaca-cleaned) is English-only; performance in other languages is untested.
Long-context reasoning beyond 2,048 tokens (training limit). While the base model supports up to 128K context, the adapter was trained at 2,048 tokens. Quality may degrade on inputs significantly longer than the training sequence length.

Limitations & Biases

Alpaca dataset biases. The Alpaca dataset reflects the biases of its generation pipeline (originally distilled from text-davinci-003). Responses may inherit stylistic or topical biases from this distribution.
Hallucination risk. Like all LLMs, the model may generate plausible but incorrect information. Always verify factual claims.
Limited training scope. The adapter was trained for a modest number of steps on a general-purpose instruction dataset. It is not a domain expert and should not be expected to match specialized models on specific tasks.
FP16 precision. Training was done in FP16 (T4 hardware limitation) rather than bf16, which may introduce minor numerical differences compared to bf16-trained adapters.
Quantization loss. The 4-bit NF4 base quantization introduces minor quality degradation compared to full-precision weights. Use the merged full-precision model for maximum quality.

🔒 Responsible AI & Safety Alignment

Alignment Techniques

Supervised Fine-Tuning (SFT). The adapter was trained on the Alpaca-cleaned dataset using trl.SFTTrainer. No RLHF or DPO was applied in this release.
Inherited safety alignment. The base model (Qwen2.5-7B-Instruct) already undergoes extensive safety alignment during its post-training stage. This adapter preserves that alignment while adjusting instruction-following behavior.
ChatML template compliance. The adapter is designed to work with Qwen2.5's native ChatML chat template, ensuring consistent formatting and predictable behavior.

Deployment Recommendations

Input validation. Sanitize all user inputs. Reject prompt-injection attempts and malicious instruction patterns.
Output audit. Review generated content for accuracy and appropriateness before use in any public-facing context.
Content filtering. Pair the model with an input/output content filter (e.g., Llama Guard, NeMo Guardrails) for production deployments.
Rate limiting. Deploy behind an API gateway with authentication and rate limiting.
Context length management. While the base model supports 128K context, keep inference inputs within the 2,048-token training window for best quality, or fine-tune with longer sequences for extended-context use cases.

🏋️ Training Details

Training Data

Training Environment

Table with columns: Parameter, Value
Parameter	Value
Hardware	NVIDIA T4 GPU (16 GB VRAM)
Cloud provider	Google Colab
Fine-tuning framework	Unsloth + `trl.SFTTrainer`
Deep learning framework	PyTorch (FP16)
Gradient checkpointing	Enabled (Unsloth optimized)

🤝 Citation & Acknowledgements

If you use this adapter in your research or product, please cite it as follows:

bibtex
@misc{hipparagi2025qwen25adapter,
  title        = {Qwen2.5-7B-Instruct LoRA Adapter: Efficient Instruction-Following Fine-tuning via QLoRA on a Single T4 GPU},
  author       = {Sandeep Hipparagi},
  organization = {BluePatterns AI},
  year         = {2025},
  url          = {https://huggingface.co/Sandeep4235/qwen2.5-7b-adapter},
  note         = {Fine-tuned from unsloth/Qwen2.5-7B-Instruct-bnb-4bit on the Alpaca-cleaned dataset under Apache 2.0}
}

Acknowledgements

Qwen Team — for open-sourcing the Qwen2.5-7B-Instruct model under Apache 2.0.
Unsloth — for the efficient QLoRA fine-tuning framework that made single-GPU training viable.
Stanford Alpaca / yahma — for the cleaned Alpaca instruction dataset.
Hugging Face — for model hosting and the open ML ecosystem.

📬 Contact

Sandeep Hipparagi — AI Developer, Co-Founder of BluePatterns AI

Hugging Face: Sandeep4235
Organization: BluePatterns AI

BluePatterns AI is focused on bridging the gap between frontier and open-source models — democratizing access to reliable, accessible intelligence for diverse communities.

Built by Sandeep Hipparagi · BluePatterns AI · Apache 2.0

qwen2.5-7b-adapter

README

🚀 Model Summary

💡 Model Architecture & Specifications

🛠 Quickstart & Usage

Option A — Load and infer with transformers + peft (recommended)

Option B — Load and infer with Unsloth (fastest inference)

Option C — Merge adapter into base model for standalone deployment

📊 Evaluation and Benchmarks

General Academic Benchmarks (inherited from base)

⚠️ Intended Uses & Limitations

Primary Use Cases

Out-of-Scope Uses

Limitations & Biases

🔒 Responsible AI & Safety Alignment

Alignment Techniques

Deployment Recommendations

🏋️ Training Details

Training Data

Training Environment

🤝 Citation & Acknowledgements

Acknowledgements

📬 Contact

Explore FriendliAI today

README

🚀 Model Summary

💡 Model Architecture & Specifications

🛠 Quickstart & Usage

Option A — Load and infer with transformers + peft (recommended)

Option B — Load and infer with Unsloth (fastest inference)

Option C — Merge adapter into base model for standalone deployment

📊 Evaluation and Benchmarks

General Academic Benchmarks (inherited from base)

⚠️ Intended Uses & Limitations

Primary Use Cases

Out-of-Scope Uses

Limitations & Biases

🔒 Responsible AI & Safety Alignment

Alignment Techniques

Deployment Recommendations

🏋️ Training Details

Training Data

Training Environment

🤝 Citation & Acknowledgements

Acknowledgements

📬 Contact

Option A — Load and infer with `transformers` + `peft` (recommended)

Option A — Load and infer with `transformers` + `peft` (recommended)