guicybercode

br-sovereign-llm-smoke

README

License: apache-2.0

Status: engineering artifact only

This is a tiny, randomly initialized Llama checkpoint trained for six CPU optimizer steps on eight original synthetic Brazilian Portuguese documents. It exists only to validate the local from-scratch training, in-process interruption/resume simulation, integrity, export, and loading paths. It does not establish recovery across a process, node, or scheduler restart.

Do not use this model for language generation, evaluation, downstream tasks, or any scientific conclusion. Its loss values are path-validation evidence, not a model-quality result. It was not trained on a supercomputer.

Measured configuration

Table with columns: Field, Value
Field	Value
parameters	149,696
decoder layers	2
hidden size	64
attention heads	4
key/value heads	2
context length	64
observed vocabulary	447
numeric type	float32
initialization seed	1701
optimizer steps	6
interruption after step	3
execution device	CPU

The default implementation used PyTorch 2.13.0, Transformers 5.15.1, and Tokenizers 0.22.2 under Python 3.13.12.

Recovery evidence

Table with columns: Measurement, Value
Measurement	Value
initial loss	`6.131303787231445`
uninterrupted final loss	`6.114025115966797`
resumed final loss	`6.114025115966797`
exact recovery	`true`
gradients finite and nonzero	`true`
model weights changed

The full loss trajectories and final tensor hashes matched between the uninterrupted and interrupted-then-resumed executions. The published folder contains safetensors weights, model configuration, tokenizer, and the smoke training report. Optimizer state and the internal .pt recovery state are not published.

Loading

python
from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "guicybercode/br-sovereign-llm-smoke"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(repo_id)

Loading success does not make the generated text meaningful.

Data and license

The eight synthetic documents were drafted with AI assistance for BR Sovereign LLM, reviewed and adopted by the project owner, and released under Apache-2.0 to the extent of his rights. The smoke weights are also released under Apache-2.0. This choice does not preselect a license for future scientific weights trained on another corpus.

Contamination, memorization, and safety review

Training input was limited to the public eight-document fixture. Its source JSONL SHA-256 is 6896439e772f212c2848846f1f908410a606d81cd8845583024c16496a07979f; the derived Parquet SHA-256 is e2b5d4c01739e92240cfe99e6dccebce3903729d083e61e86afa37f6dd86c8a2.
No benchmark examples or third-party corpus documents are present, so there is no benchmark-contamination result to report for this engineering run.
No formal extraction or memorization evaluation was run. With eight short training documents, reproduction of fixture phrases must be assumed possible.
No safety evaluation was run. The model is randomly initialized and trained for only six steps; its output is not suitable for use.
Hosted inference is disabled in the card. Loading remains documented only to verify artifact compatibility.

The scoped approval and weight-license decision are in PUBLICATION_REVIEW.md; the machine-readable removal history is takedown-ledger.json.

Author

Guilherme Monteiro, FIAP
ORCID: https://orcid.org/0009-0008-5294-224X

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

guicybercode

Model Tree

Base

this model

Input Modalities

Text

Output Modalities