🎯 What This Model Does
Given a user message and conversation history, the classifier outputs a structured
JSON object with 11 psychological signal fields:
{ "emotion": "sadness", "emotion_intensity": 0.8, "distortion": "catastrophizing", "distortion_confidence": 0.7, "crisis_risk": "medium", "crisis_confidence": 0.8, "venting": true, "motivation_level": "low", "conversation_intent": "emotional_support", "tool_recommendation": "breathing_exercise", "tool_confidence": 0.7}
Crisis risk is the safety-critical field. The model is trained to detect passive
suicidal ideation, including indirect language ("everyone would be better off without me"),
denial+ideation patterns ("I'm not going to hurt myself, I just don't see the point"),
and culturally-framed distress.
🚀 Usage
from transformers import AutoTokenizer, AutoModelForCausalLMfrom peft import PeftModelimport torch, json adapter_name = "cogni-x/MOYO-ClassifierModel-AfriqueLlama-8B-Lora"base_model_id = "McGill-NLP/AfriqueLlama-8B" tokenizer = AutoTokenizer.from_pretrained(adapter_name)base = AutoModelForCausalLM.from_pretrained( base_model_id, device_map="auto", load_in_4bit=True,)model = PeftModel.from_pretrained(base, adapter_name)model.eval() system_prompt = "You are a clinical signal classifier. Analyse the user message and output ONLY valid JSON."user_content = """CONVERSATION HISTORY: None — this is the first user message. CURRENT USER MESSAGE TO LABEL:I don't know what the point is anymore. Everyone would be better off without me. Analyse the current user message in the context of the conversation history above.Output the classifier JSON for this user message only.""" messages = [ {"role": "system", "content": system_prompt}, {"role": "user", "content": user_content},]inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, return_tensors="pt").to(model.device) with torch.no_grad(): output = model.generate(inputs, max_new_tokens=256, do_sample=False)new_tokens = output[0][inputs.shape[1]:]result = json.loads(tokenizer.decode(new_tokens, skip_special_tokens=True))print(result)# {'crisis_risk': 'medium', 'emotion': 'sadness', ...}
📊 Evaluation Results (test split)
Safety Gate
Table with columns: Metric, Value, Target, Status| Metric | Value | Target | Status |
|---|
| Crisis recall (medium+high) | 0.5526 | ≥ 0.95 | ❌ FAIL |
| Crisis precision (medium+high) | 0.4286 | ≥ 0.60 | ❌ FAIL |
| Schema-valid output | 100.0% | — | — |
All Fields
Table with columns: Field, Metric, Score, Target| Field | Metric | Score | Target |
|---|
| emotion | Macro F1 | 0.7522 | ≥ 0.80 |
| distortion | Macro F1 | 0.5028 | ≥ 0.70 |
| conversation_intent | Macro F1 | 0.6323 | ≥ 0.80 |
| motivation_level | Macro F1 | 0.7577 | ≥ 0.77 |
| tool_recommendation | Macro F1 | N/A | ≥ 0.75 |
| venting | F1 | 0.9048 | ≥ 0.85 |
Crisis recall is the only hard gate — the model must not miss medium/high crisis signals.
All other fields are soft targets.
🔧 Training Details
Table with columns: Parameter, Value| Parameter | Value |
|---|
| Base Model | McGill-NLP/AfriqueLlama-8B |
| Method | LoRA (Low-Rank Adaptation) |
| LoRA Rank (r) | 16 |
| LoRA Alpha | 16 |
| Max Sequence Length | 1024 tokens |
| Learning Rate | 0.0002 |
| Training Framework | Unsloth + Hugging Face TRL |
Training data: multi-turn mental health conversations with per-turn clinical annotations
produced by Claude Sonnet 4 (Bedrock batch) and corrected via SageMaker Ground Truth
human review. Medium/high crisis examples oversampled with tiered weights based on
human-review provenance (CHANGE×20, KEEP×14, unreviewed×10).
⚖️ Ethical Considerations
⚠️ This classifier is not a replacement for clinical assessment.
It is an assistive signal for a mental health support platform.
Always route detected high-risk users to qualified professionals or emergency services.
Organisation: CogniX Ltd | Project: MOYO-AI-Model
Repository: GitHub
Last updated: 2026-07-18