GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Debarun12
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
Debarun12
Model Tree
Qwen/Qwen2.5-7B-Instruct
Input Modalities
Output Modalities
Supported Functionality
This model is a QLoRA fine-tune of Qwen2.5-7B-Instruct, trained on 407 instruction-following Q&A pairs generated from official AWS documentation. It is designed to answer questions about AWS cloud services accurately, with built-in domain refusal for non-AWS questions. The full system includes FAISS-based retrieval (RAG) and a FastAPI backend.
Load the adapter on top of Qwen2.5-7B-Instruct to answer AWS-related questions about EC2, S3, Lambda, IAM, RDS, VPC, CloudWatch, and DynamoDB.
Plug into a RAG pipeline with a FAISS vector store built from AWS documentation chunks for grounded, source-cited answers. A FastAPI backend with streaming support is included in the project repository.
This model is not intended for general-purpose conversation, code generation, or non-AWS topics. It will refuse questions outside the AWS domain by design.
python
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfigfrom peft import PeftModelimport torch base_model_name = "Qwen/Qwen2.5-7B-Instruct"adapter_name = "Debarun12/aws-enterprise-assistant-qwen2.5-7b-qlora" # Load tokenizertokenizer = AutoTokenizer.from_pretrained(adapter_name, trust_remote_code=True) # Load base model in 4-bitbnb_config = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.bfloat16, bnb_4bit_use_double_quant=True,)base_model = AutoModelForCausalLM.from_pretrained( base_model_name, quantization_config=bnb_config, device_map="auto", trust_remote_code=True, torch_dtype=torch.bfloat16,) # Load LoRA adaptermodel = PeftModel.from_pretrained(base_model, adapter_name)model.eval() # Inferencemessages = [ {"role": "system", "content": "You are an expert AWS cloud assistant."}, {"role": "user", "content": "What is the difference between IAM roles and IAM users?"},]prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)inputs = tokenizer(prompt, return_tensors="pt").to(model.device) with torch.inference_mode(): outputs = model.generate(**inputs, max_new_tokens=300, temperature=0.3, do_sample=True) answer = tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)print(answer)Raw HTML pages were scraped from AWS documentation, cleaned with BeautifulSoup, and chunked into 512-word overlapping segments. Each chunk was passed to a local Qwen2.5-7B model to generate factual and procedural Q&A pairs in Alpaca instruction format. Pairs were formatted using Qwen2.5's chat template via apply_chat_template.
9 held-out questions covering all 8 AWS services, plus 3 out-of-scope questions for refusal testing.
Confidence scores are derived from cosine similarity between query embeddings and retrieved documentation chunks (FAISS IndexFlatIP with normalized vectors).
| Question Type | Avg Confidence | Refused Correctly |
|---|---|---|
| S3 (encryption, storage) | 0.724 | N/A |
| IAM (roles vs users) | 0.686 | N/A |
| Lambda (concurrency) | 0.680 | N/A |
| CloudWatch (monitoring) | 0.783 | N/A |
| DynamoDB (components) | 0.694 | N/A |
| DynamoDB (high availability) | 0.739 |
The model achieves consistent confidence scores of 0.68–0.78 on AWS domain questions and correctly refuses 100% of out-of-scope questions. Answers are grounded in retrieved AWS documentation via FAISS RAG.
Always verify critical AWS configuration decisions against the official AWS documentation. Use the confidence score as a signal — responses below 0.5 should be treated with caution.
Debarun Das — GitHub | HuggingFace
| N/A |
| DynamoDB (auto-scaling) | 0.688 | N/A |
| Out-of-scope (3 questions) | 0.000 | ✅ 100% |