Kodjaoglanian

Athenas-Symbiote-9B

README

License: apache-2.0

Model Details

Athenas-Symbiote-9B is a large language model (LLM) fine-tuned for advanced reasoning, complex text interpretation, and legal/academic domain expertise, primarily optimized for Brazilian Portuguese (PT-BR).

Built upon the Qwen3.5-9B architecture, this model leverages Parameter-Efficient Fine-Tuning (PEFT) to adapt its robust multilingual baseline specifically to the nuances, terminology, and logical structures required by the Brazilian legal and educational systems.

📄 Technical Report: Available on Zenodo — full methodology, evaluation protocol, and per-edition analysis.

Model Description

Developed by: Bruno Kodjaoglanian Cardoso Tulux (Independent AI Researcher)
Model Type: Causal Language Model (Transformer)
Base Model: unsloth/Qwen3.5-9B-Base
Parameter Count: 9 Billion
Context Length: 32,768 tokens
Language(s) (NLP): Portuguese (Primary), English
License: Apache 2.0
Finetuning Approach: LoRA (Low-Rank Adaptation)

Evaluation & Benchmarks

The model was comprehensively evaluated using the lm-evaluation-harness framework. Inference was conducted on high-performance infrastructure (1x NVIDIA H100 80GB) utilizing native bfloat16 precision to ensure metric fidelity and prevent quantization degradation.

[Insert your consolidated performance bar/radar chart here]

National Benchmarks (Brazil)

Evaluations targeting the Brazilian cultural, legal, and educational framework.

Table with columns: Benchmark, Metric, Score (Zero/Few-Shot)
Benchmark	Metric	Score (Zero/Few-Shot)
OAB Exams (Brazilian Bar Association)	Accuracy	63.23% (3-shot)
ENEM Challenge (National High School Exam)	Accuracy	80.90% (3-shot)

Benchmark Datasets:

OAB Exams: maritaca-ai/oab-bench

ENEM Challenge: maritaca-ai/enem

ENEM Challenge — Detailed Breakdown by Exam

Table with columns: Exam, Accuracy, Std. Error
Exam	Accuracy	Std. Error
2009	84.35%	±1.95%
2010	83.76%	±1.97%
2011	85.47%	±1.88%
2012	83.62%	±1.98%
2013	79.63%	±2.24%
2014	81.65%	±2.15%

OAB Exams — Detailed Breakdown by Exam

Table with columns: Exam, Accuracy, Std. Error
Exam	Accuracy	Std. Error
2010-01	50.59%	±3.13%
2010-02	68.00%	±2.69%
2011-03	56.57%	±2.89%
2011-04	56.25%	±3.21%
2011-05	65.00%	±3.08%
2012-06	65.00%	±3.08%

Global Benchmarks (Translated / Multilingual)

Frontier evaluations assessing general intelligence and mathematical reasoning.

Table with columns: Benchmark, Description, Score
Benchmark	Description	Score
MMLU	Massive Multitask Language Understanding (57 subjects)	—
GSM8K	Grade School Math (Chain-of-Thought Reasoning)	—
HellaSwag	Commonsense NLI	—
ARC Challenge	Advanced Scientific Reasoning	—

Note: Global benchmark evaluations are pending. Scores will be updated upon completion.

How to Get Started with the Model

Use the code below to get started with the model. As this is a base model fine-tuned with LoRA, there is no built-in chat template. The ChatML format is recommended for optimal inference with instruction-style prompts.

Requirements

bash
pip install transformers accelerate bitsandbytes

Python Inference

python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "Kodjaoglanian/Athenas-Symbiote-9B"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    torch_dtype=torch.bfloat16
)

messages = [
    {"role": "system", "content": "Você é a Athenas, uma assistente virtual altamente inteligente e precisa."},
    {"role": "user", "content": "Explique a diferença entre dolo e culpa no código penal brasileiro."}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([text], return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs, 
    max_new_tokens=512, 
    temperature=0.3,
    do_sample=True
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Details

Training Data

The model was fine-tuned on the ClassiCC-Corpus/ClassiCC-PT dataset, consumed in streaming mode with a strict quality filter (edu_score > 0.8). This heuristic ensured the model ingested only high-rigor educational, literary, and academic content. The training corpus includes:

Brazilian jurisprudence, legislation, and legal proceedings.
Academic literature and technical documentation in Portuguese.
Chain-of-Thought (CoT) reasoning structures.

Data Pipeline: Streaming → edu_score > 0.8 filter → EOS concatenation → Sequence packing (packing=True)

Training Pipeline

mermaid
graph TD
    A["Load Qwen3.5-9B-Base<br/>(4-bit, Unsloth)"] --> B["Configure PEFT<br/>LoRA r=32, α=16, rsLoRA"]
    B --> C["Dataset: ClassiCC-PT<br/>(streaming, edu_score > 0.8)"]
    C --> D["SFTTrainer<br/>1,200 steps, LR 5e-5<br/>adamw_8bit, packing=True"]
    D --> E{"Training OK?"}
    E -- No --> F["Auto-retry<br/>(5x, resume checkpoint)"]
    F --> D
    E -- Yes --> G["Push to HF Hub<br/>(Safetensors + Tokenizer)"]
    G --> H["Evaluation<br/>lm-eval-harness<br/>H100, BF16, 3-shot"]

    style A fill:#1B3A5C,color:#fff
    style B fill:#1B3A5C,color:#fff
    style C fill:#1B3A5C,color:#fff
    style D fill:#1B3A5C,color:#fff
    style F fill:#C5A059,color:#fff
    style G fill:#C5A059,color:#fff
    style H fill:#C5A059,color:#fff

Training: Kaggle T4 (16GB) | Evaluation: Lightning AI H100 (80GB)

Training Procedure

The fine-tuning process prioritized structural knowledge retention and computational efficiency.

Hardware: 1x NVIDIA T4 (16GB VRAM) via Kaggle
Quantization: 4-bit (NF4 via bitsandbytes)
Precision: FP16 (T4 does not support BF16)
Framework: Unsloth + TRL + PEFT + bitsandbytes
Optimization: LoRA adapters applied to all linear projections
LoRA Rank: 32
LoRA Alpha: 16
Scaling Strategy: rsLoRA (use_rslora=True)
LoRA Dropout: 0.0
Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, ,

Evaluation Hardware

Infrastructure: Lightning AI platform
GPU: 1x NVIDIA H100 (80GB VRAM)
Precision: bfloat16 (native, no quantization)
Framework: lm-evaluation-harness (EleutherAI)
Few-Shot: 3-shot
Bootstrap Iterations: 100,000
Execution: Sequential evaluation to isolate memory overhead.

Intended Use & Limitations

Intended Use

Primary Use Case: Research and application in Natural Language Processing (NLP) specifically for the Portuguese language.
Secondary Use Case: Automation of complex text analysis, summarization, and logical reasoning within legal and academic contexts (assisted by human review).

Out-of-Scope Use

The model is not designed to operate autonomously in high-stakes environments. It should not be used for:

Automated legal counsel or medical diagnosis without human-in-the-loop validation.
Generation of deceptive, malicious, or highly biased content.

Bias, Risks, and Limitations

Like all statistical language models, Athenas-Symbiote-9B is susceptible to generating hallucinations—plausible-sounding but factually incorrect information. Furthermore, the model may inherit and amplify biases present in its pre-training data and fine-tuning corpus. Users must exercise critical judgment and implement rigorous verification protocols before utilizing the model's outputs in production environments.

Citation

If you use this model in your research, please cite it as follows:

bibtex
@misc{athenas_symbiote_9b,
  author       = {Bruno Kodjaoglanian Cardoso Tulux},
  title        = {Athenas-Symbiote-9B: A Parameter-Efficient Fine-Tuned LLM for Brazilian Portuguese Legal and Academic Domains},
  year         = {2026},
  publisher    = {Zenodo},
  doi          = {10.5281/zenodo.21418168},
  url          = {https://zenodo.org/records/21418168},
  note         = {Model weights: https://huggingface.co/Kodjaoglanian/Athenas-Symbiote-9B}
}

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

Kodjaoglanian

Model Tree

Base

unsloth/Qwen3.5-9B-Base

Adapter

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

Model Details

📄 Technical Report: Available on Zenodo — full methodology, evaluation protocol, and per-edition analysis.

Model Description

Developed by: Bruno Kodjaoglanian Cardoso Tulux (Independent AI Researcher)
Model Type: Causal Language Model (Transformer)
Base Model: unsloth/Qwen3.5-9B-Base
Parameter Count: 9 Billion
Context Length: 32,768 tokens
Language(s) (NLP): Portuguese (Primary), English
License: Apache 2.0
Finetuning Approach: LoRA (Low-Rank Adaptation)

Evaluation & Benchmarks

[Insert your consolidated performance bar/radar chart here]

National Benchmarks (Brazil)

Evaluations targeting the Brazilian cultural, legal, and educational framework.

Table with columns: Benchmark, Metric, Score (Zero/Few-Shot)
Benchmark	Metric	Score (Zero/Few-Shot)
OAB Exams (Brazilian Bar Association)	Accuracy	63.23% (3-shot)
ENEM Challenge (National High School Exam)	Accuracy	80.90% (3-shot)

Benchmark Datasets:

OAB Exams: maritaca-ai/oab-bench

ENEM Challenge: maritaca-ai/enem

ENEM Challenge — Detailed Breakdown by Exam

Table with columns: Exam, Accuracy, Std. Error
Exam	Accuracy	Std. Error
2009	84.35%	±1.95%
2010	83.76%	±1.97%
2011	85.47%	±1.88%
2012	83.62%	±1.98%
2013	79.63%	±2.24%
2014	81.65%	±2.15%

OAB Exams — Detailed Breakdown by Exam

Table with columns: Exam, Accuracy, Std. Error
Exam	Accuracy	Std. Error
2010-01	50.59%	±3.13%
2010-02	68.00%	±2.69%
2011-03	56.57%	±2.89%
2011-04	56.25%	±3.21%
2011-05	65.00%	±3.08%
2012-06	65.00%	±3.08%

Global Benchmarks (Translated / Multilingual)

Frontier evaluations assessing general intelligence and mathematical reasoning.

Table with columns: Benchmark, Description, Score
Benchmark	Description	Score
MMLU	Massive Multitask Language Understanding (57 subjects)	—
GSM8K	Grade School Math (Chain-of-Thought Reasoning)	—
HellaSwag	Commonsense NLI	—
ARC Challenge	Advanced Scientific Reasoning	—

Note: Global benchmark evaluations are pending. Scores will be updated upon completion.

How to Get Started with the Model

Requirements

bash
pip install transformers accelerate bitsandbytes

Python Inference

python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "Kodjaoglanian/Athenas-Symbiote-9B"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    torch_dtype=torch.bfloat16
)

messages = [
    {"role": "system", "content": "Você é a Athenas, uma assistente virtual altamente inteligente e precisa."},
    {"role": "user", "content": "Explique a diferença entre dolo e culpa no código penal brasileiro."}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([text], return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs, 
    max_new_tokens=512, 
    temperature=0.3,
    do_sample=True
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Details

Training Data

Brazilian jurisprudence, legislation, and legal proceedings.
Academic literature and technical documentation in Portuguese.
Chain-of-Thought (CoT) reasoning structures.

Data Pipeline: Streaming → edu_score > 0.8 filter → EOS concatenation → Sequence packing (packing=True)

Training Pipeline

mermaid
graph TD
    A["Load Qwen3.5-9B-Base<br/>(4-bit, Unsloth)"] --> B["Configure PEFT<br/>LoRA r=32, α=16, rsLoRA"]
    B --> C["Dataset: ClassiCC-PT<br/>(streaming, edu_score > 0.8)"]
    C --> D["SFTTrainer<br/>1,200 steps, LR 5e-5<br/>adamw_8bit, packing=True"]
    D --> E{"Training OK?"}
    E -- No --> F["Auto-retry<br/>(5x, resume checkpoint)"]
    F --> D
    E -- Yes --> G["Push to HF Hub<br/>(Safetensors + Tokenizer)"]
    G --> H["Evaluation<br/>lm-eval-harness<br/>H100, BF16, 3-shot"]

    style A fill:#1B3A5C,color:#fff
    style B fill:#1B3A5C,color:#fff
    style C fill:#1B3A5C,color:#fff
    style D fill:#1B3A5C,color:#fff
    style F fill:#C5A059,color:#fff
    style G fill:#C5A059,color:#fff
    style H fill:#C5A059,color:#fff

Training: Kaggle T4 (16GB) | Evaluation: Lightning AI H100 (80GB)

Training Procedure

The fine-tuning process prioritized structural knowledge retention and computational efficiency.

Hardware: 1x NVIDIA T4 (16GB VRAM) via Kaggle
Quantization: 4-bit (NF4 via bitsandbytes)
Precision: FP16 (T4 does not support BF16)
Framework: Unsloth + TRL + PEFT + bitsandbytes
Optimization: LoRA adapters applied to all linear projections
LoRA Rank: 32
LoRA Alpha: 16
Scaling Strategy: rsLoRA (use_rslora=True)
LoRA Dropout: 0.0
Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, ,

Evaluation Hardware

Infrastructure: Lightning AI platform
GPU: 1x NVIDIA H100 (80GB VRAM)
Precision: bfloat16 (native, no quantization)
Framework: lm-evaluation-harness (EleutherAI)
Few-Shot: 3-shot
Bootstrap Iterations: 100,000
Execution: Sequential evaluation to isolate memory overhead.

Intended Use & Limitations

Intended Use

Primary Use Case: Research and application in Natural Language Processing (NLP) specifically for the Portuguese language.
Secondary Use Case: Automation of complex text analysis, summarization, and logical reasoning within legal and academic contexts (assisted by human review).

Out-of-Scope Use

The model is not designed to operate autonomously in high-stakes environments. It should not be used for:

Automated legal counsel or medical diagnosis without human-in-the-loop validation.
Generation of deceptive, malicious, or highly biased content.

Bias, Risks, and Limitations

Citation

If you use this model in your research, please cite it as follows:

bibtex
@misc{athenas_symbiote_9b,
  author       = {Bruno Kodjaoglanian Cardoso Tulux},
  title        = {Athenas-Symbiote-9B: A Parameter-Efficient Fine-Tuned LLM for Brazilian Portuguese Legal and Academic Domains},
  year         = {2026},
  publisher    = {Zenodo},
  doi          = {10.5281/zenodo.21418168},
  url          = {https://zenodo.org/records/21418168},
  note         = {Model weights: https://huggingface.co/Kodjaoglanian/Athenas-Symbiote-9B}
}