Model Details

Model Description

• Developed by: Orange
• Funded by [optional]: [More Information Needed]
• Shared by [optional]: [More Information Needed]
• Model type: [More Information Needed]
• Language(s) (NLP): Wolof
• License: [More Information Needed]
• Finetuned from model [optional]: Orange/Qwen3-8B
• Date [optional]: 2026-06-13 12:10:53

Model Sources [optional]

• Repository: [More Information Needed]
• Paper [optional]: [More Information Needed]
• Demo [optional]: [More Information Needed]

Uses

Direct Use

This model can be used with the transformers library using pipeline abstraction as follows:

python
Copy code
import torch
from transformers import pipeline

model_id = "Orange/Wolof-Qwen3-8B-it-v2-fc-v2-conv-v1"
pipe = pipeline(
    "text-generation",
    model=model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
messages = [
    {"role": "system", "content": "You are chatbot specialized on Wolof language."},
    {"role": "user", "content": "Can you give a sample of your specialized knowledge?"},
]
outputs = pipe(
    messages,
    max_new_tokens=256,
)
print(outputs[0]["generated_text"][-1])

Downstream Use [optional]

[More Information Needed]

Out-of-Scope Use

[More Information Needed]

Bias, Risks, and Limitations

[More Information Needed]

Recommendations

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.

How to Get Started with the Model

Use the code below to get started with the model.

[More Information Needed]

Training Details

This model was finetuned with Orange internal fine tuning tools with the Docker Image tagged 0.2.6t5alpha in the registry and the following configuration file:

yaml
Copy code
data:
    add_dataset_name: false
    dataset_name:
        train:
        -   path: wolof-lm/en_function_calling-instructions
            revision: v10
            split: train
        -   path: wolof-lm/alpaca_translated_into_wolof-instructions
            revision: 2025.02.28
            split: train
        -   path: wolof-lm/argilla_databricks-dolly-15k-curated-multilingual_en_translated_into_wolof-instructions
            revision: 2025.02.28
            split: train
        -   path: wolof-lm/samsung_samsum_instruct_dialogues_translated_in_wolof-instructions
            revision: 2025.02.28
            split: train
        -   path: wolof-lm/alwaly_french_wolof-instructions
            split: train
        -   path: wolof-lm/alwaly_wolof_to_french-instructions
            split: train
        -   path: wolof-lm/claire_masking_multilogue-instructions
            split: train
        -   path: wolof-lm/huggingfaceh4_helpful-anthropic-raw_train_translated_in_wolof-instructions
            split: train
        -   path: wolof-lm/lodrick-the-lafted-hermes-217k-instructions
            split: train
        -   path: wolof-lm/wisenut-nlp-team_translated_in_wolof-instructions
            split: train
        -   path: wolof-lm/claire_one_to_one_dialogue-instructions
            revision: 2026.01.16
            split: train
        -   path: wolof-lm/wol-sonatel-2025-conversations
            revision: 2026.02.18
            split: train
        validation_conversational_eng:
        -   path: wolof-lm/oasst1-en-instructions
            revision: 2026.03.19
            split: validation
        validation_conversational_fra:
        -   path: wolof-lm/oasst1-fr-instructions
            revision: 2026.03.19
            split: validation
        -   path: wolof-lm/synthetic-fr-instructions
            revision: 2026.03.19
            split: validation
        validation_conversational_wol:
        -   path: wolof-lm/claire_one_to_one_dialogue-instructions
            revision: 2026.01.16
            split: validation
        -   path: wolof-lm/wol-sonatel-2025-conversations
            revision: 2026.02.18
            split: validation
        validation_fc:
        -   path: wolof-lm/en_function_calling-instructions
            revision: v10
            split: validation
        validation_if:
        -   path: wolof-lm/tulu-3-sft-personas-instruction-following
            split: validation
        validation_it-v2:
        -   path: wolof-lm/alpaca_translated_into_wolof-instructions
            revision: 2025.02.28
            split: validation
        -   path: wolof-lm/argilla_databricks-dolly-15k-curated-multilingual_en_translated_into_wolof-instructions
            revision: 2026.03.02
            split: validation
        -   path: wolof-lm/samsung_samsum_instruct_dialogues_translated_in_wolof-instructions
            revision: 2025.02.28
            split: validation
        -   path: wolof-lm/alwaly_french_wolof-instructions
            split: validation
        -   path: wolof-lm/alwaly_wolof_to_french-instructions
            split: validation
        -   path: wolof-lm/claire_masking_multilogue-instructions
            split: validation
        -   path: wolof-lm/huggingfaceh4_helpful-anthropic-raw_train_translated_in_wolof-instructions
            split: validation
        -   path: wolof-lm/lodrick-the-lafted-hermes-217k-instructions
            split: validation
        -   path: wolof-lm/wisenut-nlp-team_translated_in_wolof-instructions
            split: validation
    debug: false
    implementation_name: conversations
description:
    contributors:
    -   email: claire.perroux@orange.com
        first_name: Claire
        last_name: Perroux
    -   email: pierre.adam@orange.com
        first_name: Pierre
        last_name: Adam
    domain: Wolof
    languages:
    - wo
    model_name: Orange/Wolof-Qwen3-8B-it-v2-fc-v2-conv-v1
image:
    name: registry.gitlab.tech.orange/nepal/knowledge/orangelm/lm-adaptation/lm-adaptation
    version: 0.2.6t5alpha
model:
    attn_implementation: flash_attention_2
    chat_template_tokenizer: Orange/Qwen3-8B
    model_name_or_path: Orange/Qwen3-8B
    trust_remote_code: true
software_info:
    git:
        branch: 119-cuda-13-and-transformers-5
        commit_date: '2026-06-10 15:41:20+02:00'
        commit_hash: 314a8efb580bf36117958e80dc33d7e221aa7d30
        commit_message: 'build: set same version for kernels library in pyproject
            as in requirements.txt'
        is_dirty: false
        short_hash: 314a8ef
    parameters:
        accelerate_file: null
        clean_output: false
        cluster: marcel
        codecarbon_log_level: warning
        collator_stats: false
        config_file: resources/configs/Wolof-Qwen3-8B-it-v2-fc-v2-conv-v1.yml
        constraints: queue:name=gpu_mem_80,gpu:2,partition:name=multigpu,partition:name=All
        data_seed: 42
        debug: false
        deepspeed_file: resources/deepspeed/zero3.json
        dry_run: false
        environment_variable:
            TOKENIZERS_PARALLELISM: 'false'
        load_only: null
        log_level: INFO
        log_output: null
        merge_peft: false
        no_requeue: false
        output_dir: Wolof-Qwen3-8B-it-v2-fc-v2-conv-v1-t5-torch28-2gpu
        profiler_config: null
        push_to_hub: false
        qat_config: null
        requeue_n_epochs: 1
        time_out: 72h
        time_requeue: false
        use_qat: false
    version: 0.21.2.dev33+gb8ccade7f.d20260610
training:
    assistant_only_loss: true
    bf16: true
    dataloader_num_workers: 4
    dataloader_persistent_workers: false
    dataloader_pin_memory: false
    dataloader_prefetch_factor: 2
    deepspeed: /config/zero3.json
    disable_tqdm: true
    eval_accumulation_steps: 1
    eval_steps: 10
    eval_strategy: steps
    fp16: false
    gradient_accumulation_steps: 32
    gradient_checkpointing: true
    learning_rate: 2.0e-05
    log_level: warning
    logging_dir: /outputs/Wolof-Qwen3-8B-it-v2-fc-v2-conv-v1-t5-torch28-2gpu/logs
    logging_steps: 1
    lr_scheduler_type: cosine
    max_grad_norm: 1.0
    max_length: 2048
    max_steps: -1
    num_train_epochs: 3
    optim: paged_adamw_32bit
    output_dir: /outputs/Wolof-Qwen3-8B-it-v2-fc-v2-conv-v1-t5-torch28-2gpu
    per_device_eval_batch_size: 16
    per_device_train_batch_size: 16
    push_to_hub: false
    report_to: tensorboard
    save_steps: 0
    save_strategy: epoch
    save_total_limit: 1
    seed: 42
    torch_compile: false
    training_type: instruct-tuning
    use_liger_kernel: true
    warmup_ratio: 0.05
    weight_decay: 0.1

The model was trained on 2 gpus with at least 80GB on each gpu.

The model was trained using deepspeed with the following configuration file:

json
Copy code
{
    "fp16": {
        "enabled": "auto",
        "loss_scale": 0,
        "loss_scale_window": 1000,
        "initial_scale_power": 16,
        "hysteresis": 2,
        "min_loss_scale": 1
    },
    "bf16": {
        "enabled": "auto"
    },
    "zero_optimization": {
        "stage": 3,
        "offload_optimizer": {
            "device": "cpu",
            "pin_memory": false
        },
        "overlap_comm": true,
        "contiguous_gradients": true,
        "sub_group_size": "1e9",
        "reduce_bucket_size": "auto",
        "stage3_prefetch_bucket_size": "auto",
        "stage3_param_persistence_threshold": "auto",
        "stage3_max_live_parameters": "1e9",
        "stage3_max_reuse_distance": "1e9",
        "stage3_gather_16bit_weights_on_model_save": true
    },
    "gradient_accumulation_steps": "auto",
    "gradient_clipping": "auto",
    "steps_per_print": 2000,
    "train_batch_size": "auto",
    "train_micro_batch_size_per_gpu": "auto",
    "wall_clock_breakdown": false
}

Training Data

This model was trained on the following datasets:

yaml
Copy code
-   path: wolof-lm/en_function_calling-instructions
    revision: v10
    split: train
-   path: wolof-lm/alpaca_translated_into_wolof-instructions
    revision: 2025.02.28
    split: train
-   path: wolof-lm/argilla_databricks-dolly-15k-curated-multilingual_en_translated_into_wolof-instructions
    revision: 2025.02.28
    split: train
-   path: wolof-lm/samsung_samsum_instruct_dialogues_translated_in_wolof-instructions
    revision: 2025.02.28
    split: train
-   path: wolof-lm/alwaly_french_wolof-instructions
    split: train
-   path: wolof-lm/alwaly_wolof_to_french-instructions
    split: train
-   path: wolof-lm/claire_masking_multilogue-instructions
    split: train
-   path: wolof-lm/huggingfaceh4_helpful-anthropic-raw_train_translated_in_wolof-instructions
    split: train
-   path: wolof-lm/lodrick-the-lafted-hermes-217k-instructions
    split: train
-   path: wolof-lm/wisenut-nlp-team_translated_in_wolof-instructions
    split: train
-   path: wolof-lm/claire_one_to_one_dialogue-instructions
    revision: 2026.01.16
    split: train
-   path: wolof-lm/wol-sonatel-2025-conversations
    revision: 2026.02.18
    split: train

Training Procedure

Preprocessing [optional]

[More Information Needed]

Training Hyperparameters

• Training regime: This model was trained with the following hyperparameters for SFTTrainer,other parameters were set as default:

yaml
Copy code
assistant_only_loss: true
bf16: true
dataloader_num_workers: 4
dataloader_persistent_workers: false
dataloader_pin_memory: false
dataloader_prefetch_factor: 2
deepspeed: /config/zero3.json
disable_tqdm: true
eval_accumulation_steps: 1
eval_steps: 10
eval_strategy: steps
fp16: false
gradient_accumulation_steps: 32
gradient_checkpointing: true
learning_rate: 2.0e-05
log_level: warning
logging_dir: /outputs/Wolof-Qwen3-8B-it-v2-fc-v2-conv-v1-t5-torch28-2gpu/logs
logging_steps: 1
lr_scheduler_type: cosine
max_grad_norm: 1.0
max_length: 2048
max_steps: -1
num_train_epochs: 3
optim: paged_adamw_32bit
output_dir: /outputs/Wolof-Qwen3-8B-it-v2-fc-v2-conv-v1-t5-torch28-2gpu
per_device_eval_batch_size: 16
per_device_train_batch_size: 16
push_to_hub: false
report_to: tensorboard
save_steps: 0
save_strategy: epoch
save_total_limit: 1
seed: 42
torch_compile: false
use_liger_kernel: true
warmup_ratio: 0.05
weight_decay: 0.1

Number of Tokens vs Steps

Learning Rate Curve

Training Loss

Validation Loss

Speeds, Sizes, Times [optional]

[More Information Needed]

Evaluation

Testing Data, Factors & Metrics

Testing Data

[More Information Needed]

Factors

[More Information Needed]

Metrics

[More Information Needed]

Results

[More Information Needed]

Summary

Model Examination [optional]

[More Information Needed]

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

• Hardware Type: CPUs: AMD EPYC 9124 16-Core Processor; GPUs: 2 x NVIDIA H100 NVL
• Hours used: 74:02:30
• Cloud Provider: [More Information Needed]
• Compute Region: [More Information Needed]
• Carbon Emitted: 2.6 kg CO2eq, detailed emissions can be found in emissions.csv (emissions were computed using codecarbon)

Technical Specifications [optional]

Model Architecture and Objective

[More Information Needed]

Compute Infrastructure

[More Information Needed]

Hardware

[More Information Needed]

Software

[More Information Needed]

Citation [optional]

BibTeX:

[More Information Needed]

APA:

[More Information Needed]

Glossary [optional]

[More Information Needed]

More Information [optional]

[More Information Needed]

Model Card Authors [optional]

[More Information Needed]

Model Card Contact

Thanks to Claire Perroux, Pierre Adam for adding this model.