Climate-1000-Qwen-4b-Adapter API & Inference Endpoint

Provenance and selection

Base: Qwen/Qwen3.5-4B, pinned revision 851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a.
Training: unquantized frozen BF16 base, BF16 mixed precision, FP32 adapter storage.
LoRA: all supported linear layers, rank 32, alpha 64, dropout 0.05, bias none.
Dataset: 3,665 training and 406 validation examples, completion-only loss.
Learning rate 1e-4; two epochs; microbatch 1; accumulation 16; seed 3407.
Maximum sequence length 8,192; no packing; thinking disabled.
Completed 460 optimizer steps on 22 October 2026. Selected checkpoint 230, epoch 1, validation loss 0.37746524810791016.
Final step-460 validation loss: 0.38062456250190735.
Training resumed from checkpoint 120 after a memory safety stop. Periodic CUDA cache clearing and memory telemetry enabled; optimization settings unchanged.
The 100-paper benchmark remains provisional (98 authored/reviewed cases, 78 approved). Training validation does not establish downstream scientific accuracy.

Loading

Use transformers.AutoModelForCausalLM to load the text-generation base, followed by peft.PeftModel.from_pretrained. This matches the training architecture Qwen3_5ForCausalLM; adapter tensor names target the text model. Use compatible Transformers/PEFT versions (training used Transformers 5.15.1 and PEFT 0.19.1). Apply the included chat template with enable_thinking=False.

No merged-model inference evaluation is claimed. See checkpoint_manifest.json for pinned identities, data hashes and file checksums.

Provenance and selection

Base: Qwen/Qwen3.5-4B, pinned revision 851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a.
Training: unquantized frozen BF16 base, BF16 mixed precision, FP32 adapter storage.
LoRA: all supported linear layers, rank 32, alpha 64, dropout 0.05, bias none.
Dataset: 3,665 training and 406 validation examples, completion-only loss.
Learning rate 1e-4; two epochs; microbatch 1; accumulation 16; seed 3407.
Maximum sequence length 8,192; no packing; thinking disabled.
Completed 460 optimizer steps on 22 October 2026. Selected checkpoint 230, epoch 1, validation loss 0.37746524810791016.
Final step-460 validation loss: 0.38062456250190735.
Training resumed from checkpoint 120 after a memory safety stop. Periodic CUDA cache clearing and memory telemetry enabled; optimization settings unchanged.
The 100-paper benchmark remains provisional (98 authored/reviewed cases, 78 approved). Training validation does not establish downstream scientific accuracy.

Loading

No merged-model inference evaluation is claimed. See checkpoint_manifest.json for pinned identities, data hashes and file checksums.

Climate-1000-Qwen-4b-Adapter

README

Provenance and selection

Loading

Explore FriendliAI today

README

Provenance and selection

Loading