GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
U4RASD
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
U4RASD
Model Tree
Navid-AI/Yehia-7B-preview
Input Modalities
Output Modalities
Supported Functionality
| Item | Selected value |
|---|---|
| Task | Directed Arabic relation extraction |
| Direction | [الأول] → [الثاني] |
| Positive relations | 40 |
| Negative class | لا توجد علاقة / metadata label no_relation |
| Base model | Navid-AI/Yehia-7B-preview |
| Prompt version | yehia_re_ar_v3_one_token_choice |
| Output contract | One displayed Arabic option code |
| SFT dataset | U4RASD/Yehia-RE-SFT |
| Adapter repository | U4RASD/DRU-RE-Yehia |
| Quantization | 4-bit NF4, double quantization, BF16 compute |
| PEFT targeting | target_modules="all-linear" |
| LoRA | rank 16, alpha 32, dropout 0.05, RS-LoRA |
| Training length | 3 epochs / 2,943 optimizer steps on this dataset |
| Effective batch | 16 = physical batch 4 × gradient accumulation 4 |
| Best-checkpoint metric | Validation positive micro F1 |
| Scheduled evaluation | All 1,687 validation rows every 250 optimizer steps |
| Official submission | 2,074 rows; root ZIP member predictions.txt |
text
DRU-RE-Yehia/├── README.md├── CHANGELOG.md # grounded selected-system changes├── model.py # final QLoRA trainer and evaluator├── predict.py # constrained one-token batch inference├── create_submission.py # strict official inference + Codabench ZIP├── prepare_input.py # raw sentence/entity pair to model prompt├── build_dataset.py # deterministic source-to-SFT builder├── download_dataset.py # exact transformed-revision downloader├── audit_dataset.py # exhaustive local dataset audit├── re_sft_common.py # prompt, markers, ontology, transformation├── requirements.txt # validated pinned Python stack├── setup_env.sh # environment and local self-test├── train.sh # foreground training/resume├── train_tmux.sh # detached training/resume├── build_dataset.sh # build followed by independent audit├── build_submission.sh # pinned 2,074-row inference and packaging├── submission.zip # ready-to-upload deterministic artifact├── .env.example # safe configuration without credentials├── .gitattributes # full-release model/checkpoint LFS patterns├── .gitignore├── resources/│ ├── wojood_ontology.json│ ├── entity_type_ar_mapper.json│ ├── relation_mapper.json│ ├── relation_template_ar_mapper.json│ ├── relation_template_ar_reverse_mapper.json│ └── resource_manifest.json├── examples/│ ├── example_record.json│ ├── ALL_41_CASES.md # full real prompt/output for every class│ └── all_41_class_examples.jsonl # same 41 examples, machine-readable├── evidence/ # small correctness evidence, no weights│ ├── README.md│ ├── dataset_summary.json│ ├── dataset_remote_verification.json│ ├── base_model_remote_verification.json│ ├── selected_run_config.json│ ├── all_linear_eligible_modules.json│ ├── lora_wrapped_modules.json│ ├── decision_token_inventory.json│ ├── decision_token_masking_audit.json│ ├── sampling_strategy.json│ ├── smoke_test.json│ └── validation_history.json├── submission/│ ├── README.md # exact competition contract and rebuild│ ├── official_predictions.jsonl # auditable row-local model decisions│ ├── predictions.txt # 2,074 tab-delimited competition lines│ └── run_summary.json # source/model/artifact hashes and counts├── release/ # final metrics, config, manifest, remote proof├── latest_checkpoint/ # complete step-2,943 state; Hugging Face only└── tools/├── build_examples_appendix.py├── plot_losses.py├── monitor_resources.sh├── stage_azure_release.py # build the weight-free Azure mirror├── verify_azure_release.py # reject weights/checkpoints in Azure├── restore_latest_checkpoint.py├── stage_release.py├── verify_release.py└── publish_huggingface.py
After a successful full release-staging operation, the Hugging Face model root
additionally contains PEFT adapter/tokenizer files and inference_config.json;
final run evidence is placed under release/, and the complete latest Trainer
state is placed under latest_checkpoint/. The Azure copy intentionally omits
the root adapter tensor and the entire checkpoint directory. Full Yehia base
weights are never copied to either destination.
Each example contains an Arabic sentence and two entities. The first entity is surrounded by:
text
[الأول] ... [/الأول]
The second entity is surrounded by:
text
[الثاني] ... [/الثاني]
The question is always directional: what relation holds from the first marked
entity to the second marked entity? A reverse relation is not interchangeable.
If no compatible displayed relation is expressed in that direction, the correct
choice is لا توجد علاقة.
The model sees only:
لا توجد علاقة as the final option.Canonical dataset labels such as PERS, ORG, employee_of, or
Affiliation.employee_of remain metadata. They are never rendered inside the
system, user, or assistant message.
text
أنت مصنّف علاقات. حدّد العلاقة الموجّهة من الكيان [الأول] إلى الكيان [الثاني].اختر علاقة واحدة من القائمة تناسب معنى الجملة ونوعي الكيانين.إن لم تنطبق أي علاقة فاختر «لا توجد علاقة».أجب برمز الخيار فقط: حرف عربي واحد دون رقم أو شرح.
text
الجملة:{الجملة العربية مع [الأول]...[/الأول] و[الثاني]...[/الثاني]}نوع الأول: {النوع العربي للأول}نوع الثاني: {النوع العربي للثاني}الخيارات:أ) {علاقة عربية متوافقة}ب) {علاقة عربية متوافقة أخرى}...ي) لا توجد علاقةرمز الإجابة:
The number and letters vary by row. The option-code inventory is:
text
أ، ب، ج، د، ه، و، ز، ح، ط، ي
All ten codes were verified to be exactly one token under Yehia's native tokenizer when preceded by the assistant-template space. A row uses only the prefix required by its number of displayed options.
The assistant content is one code and nothing else, for example:
text
ز
The code is interpreted through that row's displayed list. It is not a global
relation ID. Because لا توجد علاقة is always last, its code changes with the
number of compatible positive options in the row.
A complete real transformed example is available in
examples/example_record.json.
No relation-class tokens were added to Yehia's vocabulary. The letters
أ ب ج د ه و ز ح ط ي already exist in the native Yehia tokenizer and each was
verified to occupy exactly one token in the assistant position. They are
row-local addresses, not global class IDs.
Three arrays on every transformed row have the same length and order:
text
option_codes[i]allowed_options_ar[i]allowed_relation_full_labels[i]
The first array is shown to and emitted by the model. The second array is the
Arabic meaning shown in the prompt. The third array is metadata used to recover
the canonical dataset class. Index i ties them together.
For example, two different rows may contain:
| Row | Letter أ means on that row | Gold output | Decoded class |
|---|---|---|---|
train_5482 | الأول هو قائد الثاني | أ | Administration.leader_of |
train_908 | الأول لغتها الرسمية الثاني | أ |
There is no contradiction: the prompt itself defines the meaning of أ for
that row. Likewise, the same relation can be assigned different letters on
different rows because the compatible positive options are deterministically
shuffled. The decoder must therefore retain each row's mapping until after the
model makes its choice.
لا توجد علاقة is always the final displayed option, but it does not always
have the same letter:
أ;ج;ي.This is why a downstream application must never interpret a bare letter without the originating row's option arrays.
This is the exact real transformed row train_13110. The model receives the
following system message:
text
أنت مصنّف علاقات. حدّد العلاقة الموجّهة من الكيان [الأول] إلى الكيان [الثاني].اختر علاقة واحدة من القائمة تناسب معنى الجملة ونوعي الكيانين.إن لم تنطبق أي علاقة فاختر «لا توجد علاقة».أجب برمز الخيار فقط: حرف عربي واحد دون رقم أو شرح.
It then receives this exact user message:
text
الجملة:[الأول] سوق [الثاني] مدينة يافا [/الثاني] [/الأول] .نوع الأول: منشأةنوع الثاني: كيان جيوسياسيالخيارات:أ) الأول يقع بالقرب من الثانيب) الأول يقع في الثانيج) لا توجد علاقةرمز الإجابة:
The exact assistant target/output is:
text
ب
For this row only, ب is index 1, whose Arabic phrase is
الأول يقع في الثاني and whose metadata class is Location.located_in.
This is the exact real transformed row train_neg-62666-62670AwLOC. The system
message is identical. Its exact user message is:
text
الجملة:[الأول] جبال نابلس [/الأول] [الثاني] والكرمل [/الثاني] .نوع الأول: موقع جغرافينوع الثاني: شخصالخيارات:أ) لا توجد علاقةرمز الإجابة:
The exact assistant target/output is:
text
أ
Here أ decodes to no_relation because the negative choice is the only
ontology-compatible option for that direction and type pair.
examples/ALL_41_CASES.md contains one real,
fully filled system prompt, user prompt, assistant output, and row-local mapping
table for all 40 positive relations plus no_relation. Its coverage index
makes the examples easy to scan. The identical selected records are preserved
in machine-readable form at
examples/all_41_class_examples.jsonl.
Regenerate the appendix deterministically from the pinned transformed training split with:
bash
python tools/build_examples_appendix.py
The generator requires all 15,686 training rows, validates every selected
mapping and output, requires exactly 41 classes, and deliberately documents
no_relation last.
The original experiment asked Yehia to generate the complete Arabic relation phrase. Interactive testing exposed a distinction between semantic mistakes and surface-format mistakes: the base model often copied list punctuation, prefixed the answer with words such as “الخيار”, or produced an explanation around an otherwise recognizable choice.
Prompt development followed this sequence:
لا توجد علاقة visibly and structurally last.The selected explicit prompt achieved calibrated base-model positive micro F1
0.3616; the shorter compact alternative achieved 0.3563. The selected prompt
was therefore slightly stronger and more explicit about direction and output
format. There is no demonstration in the final prompt.
The validation split was used to choose the prompt and calibrate the no-relation decision bias. It is a development set, not an untouched test set. Validation-tuned numbers must not be presented as final held-out test performance. Official/test inference must freeze:
No official labels are used during training or intermediate evaluation.
U4RASD/WojoodRelationsAnnotatedU4RASD/Yehia-RE-SFTa060e47f56025778b97344d4d3de60a8fd53be7cThe transformed revision was independently downloaded after publication and matched local row counts, byte counts, and SHA-256 hashes.
| Split | Rows | Positive | No relation | Role |
|---|---|---|---|---|
train | 15,686 | 9,009 | 6,677 | QLoRA training only |
validation | 1,687 | 867 | 820 | Prompt development and checkpoint evaluation only |
official | 2,074 |
Training never merges validation or official rows. Official examples contain system and user messages only; they have no assistant target or gold relation.
| File | SHA-256 |
|---|---|
train.jsonl | 69efbfccfc6e810b603b17e1b33c0f087590a4357ea52b49f4564f962cc1daad |
validation.jsonl | c5aadb24df10626f047223885968de6404f54504d256414c1223bc8ca6619e11 |
official.jsonl | e6178b498411bdd4782aa13faad81d930700e09361263183fb01ef2bd4477e98 |
The corresponding source-file hashes are retained in
evidence/dataset_summary.json.
build_dataset.py delegates all locked transformation behavior to
re_sft_common.py. For each source row it performs the following operations:
subject_start:subject_end and
object_start:object_end recover the stored mention strings exactly.entity_type_ar_mapper.json.The number of displayed options ranges from 1 to 10 according to the directed
entity-type pair. A one-option row contains only لا توجد علاقة.
With identical source JSONL bytes, resource files, transformation code, seed
42, and context size 500, the transformed JSONL content is deterministic.
dataset_summary.json contains a creation timestamp, so that metadata field can
differ even when the three JSONL hashes are identical.
Each JSONL line is a self-contained object. Important fields are:
| Field | Meaning |
|---|---|
id | Stable transformed-row ID |
sentence_id, triple_id | Source identifiers |
sentence | Original unmodified sentence |
subject, object | Original mention strings |
subject_start/end | Original first-entity character span |
Canonical fields exist for evaluation and reversibility; the prompt-leakage audit examines only conversation content and confirms that these identifiers do not reach the model as text.
The resources/ directory is part of the experiment definition:
| File | Purpose |
|---|---|
wojood_ontology.json | Coarse entity types and directed relation domain/range rules |
entity_type_ar_mapper.json | Exact mapping for all 21 coarse types |
relation_mapper.json | 40 full labels plus no_relation mapping |
relation_template_ar_mapper.json | Exact packaged Arabic answer for each positive relation |
relation_template_ar_reverse_mapper.json | Arabic-answer-to-relation reverse mapping |
These files are not suggestions. Changing templates, type names, or ontology compatibility creates a different experiment and requires a new prompt/data version.
The independent audit_dataset.py pass validates all 19,447 transformed rows:
dataset_summary.json;no_relation;لا توجد علاقة present exactly once and last;The audit writes validation_audit.json beside the downloaded/built dataset.
This is not regex loss, sequence-wide language-model loss, or loss over the Arabic relation phrase. It is full-vocabulary causal-language-model cross-entropy on exactly one assistant token: the gold displayed option code.
For every encoded row:
| Token region | Label |
|---|---|
| System message | -100 |
| User message | -100 |
| Chat-template prompt boundary | -100 |
| Assistant decision code | Actual one-token ID |
| Any trailing template token | -100 |
| Padding | -100 |
Conceptually, one encoded training example looks like this:
text
input_ids = [all native-chat-template prompt tokens, gold_letter, trailing tokens]labels = [-100, -100, ... , gold_letter, -100, ...]
Transformers' causal language-model implementation shifts the labels
internally. Therefore, the logits at the position immediately before the gold
letter are trained to predict that letter. If g is the token ID of the gold
letter and z_v is the model logit for vocabulary token v, the one-row loss
is exactly:
text
loss = -log(exp(z_g) / sum(exp(z_v) for every token v in Yehia's vocabulary))
Equivalently, it is -log(p_gold). Some useful intuition:
| Probability assigned to the correct letter | One-token loss |
|---|---|
| 0.90 | 0.105 |
| 0.70 | 0.357 |
| 0.50 | 0.693 |
| 0.10 | 2.303 |
| 0.01 | 4.605 |
The batch loss is the mean over the supervised letter positions in that batch. Because there is exactly one supervised token per row, a long prompt does not silently contribute more loss than a short prompt.
The mapping step is outside the differentiable loss:
-100 label.No regular expression changes the training labels or loss. No fuzzy phrase match is used in backpropagation. No 41-class classification head, 41 new tokens, or global letter-to-relation table exists. Regex/fuzzy parsing belonged only to abandoned free-form prompt experiments and is not part of the final trainer.
The trainer applies Yehia's native chat template exactly once to the prompt and once to the complete labeled conversation, finds their exact common boundary, and verifies that the first assistant token is the expected code. It rejects a row if the boundary differs, the sequence exceeds 1,024 tokens, or no decision token remains.
Full-vocabulary loss means the correct code must outrank every token in Yehia's
vocabulary. A candidate-only softmax objective was evaluated during engineering,
but it was not selected: the tested path was slower and did not directly
penalize unrelated vocabulary tokens. The clean model.py contains only the
selected full-vocabulary objective.
The saved masking evidence includes a positive example, a negative example, and a long prompt. In every audited row:
See
evidence/decision_token_masking_audit.json.
Only LoRA weights receive gradients. The 4-bit base weights, embeddings, and language-model output head remain frozen. Thus the model learns the task by changing low-rank adapter paths throughout the transformer while preserving the gated base checkpoint as an external dependency.
The selected configuration in .env.example and model.py is:
| Setting | Value |
|---|---|
| Base model | Navid-AI/Yehia-7B-preview |
| Exact base revision | b9dda4715eafee7e8090d2c83cfe078d75f4ebb8 |
| Quantization | 4-bit NF4 |
| Double quantization | Enabled |
| Compute dtype | BF16 |
| Model device map | Entire quantized model on GPU 0 |
| Attention implementation | SDPA |
| Maximum sequence length |
Rank 16/alpha 32 is the selected run, not the package's earlier rank 64/alpha 128 proposal. Pilot evidence showed strong learning with the smaller adapter, reducing trainable parameters and release size while retaining the required all-linear coverage.
Training uses a deterministic WeightedRandomSampler with replacement. For
class count n_c, the relative row weight is proportional to:
text
n_c ^ -0.25
This softens severe class imbalance without forcing every class to be equally
common. The exact class weights are stored in
evidence/sampling_strategy.json.
target_modules="all-linear" is passed directly to LoraConfig. The trainer
does not infer success from the presence of seven familiar suffixes.
PEFT resolves the sentinel against the concrete architecture and may serialize
the resulting suffix set (q_proj, k_proj, v_proj, o_proj, gate_proj,
up_proj, down_proj) in adapter_config.json. That serialized expansion is
not the evidence for the claim. The immutable run configuration records the
original request as the literal string "all-linear", and the before/after
full-name inventories below prove actual coverage of every supported internal
module while excluding only the output head.
Before PEFT attachment it inventories every supported torch.nn.Linear,
Transformers Conv1D, and compatible bitsandbytes linear module by full name
and class. It separately identifies embeddings and output heads. After
attachment it inventories every LoRA module and compares the full internal set.
For Yehia 7B the selected run recorded:
| Audit item | Result |
|---|---|
| Eligible supported linear/head modules | 225 |
| LoRA-wrapped internal modules | 224 |
| Transformer layers | 32 |
| Wrapped modules per layer | 7 |
Wrapped q_proj | 32 |
Wrapped k_proj | 32 |
Wrapped v_proj | 32 |
The one eligible-but-unwrapped module is the language-model output head, which PEFT intentionally excludes under all-linear targeting. Only LoRA parameters are trainable.
Full inventories are preserved in:
The selected physical batch was tested after attaching all-linear adapters with representative long examples, 1,024-token maximum length, 4-bit NF4, double quantization, BF16, and gradient checkpointing.
The recorded physical-batch-4 forward/backward smoke test produced:
| Item | Result |
|---|---|
| Status | Passed |
| Loss | 0.619789, finite |
| Physical batch | 4 |
| Gradient accumulation | 4 |
| Finite nonzero LoRA gradient tensors | 224 |
| Peak allocated VRAM | 9.044 GiB |
| Peak reserved VRAM | 10.172 GiB |
The complete active process uses additional memory for model state, optimizer,
evaluation, and allocator caching; the live L40S run remained well below the
approximately 45 GB device capacity. See
evidence/smoke_test.json.
On a smaller GPU, keep effective batch 16 by trying physical batch 2 with gradient accumulation 8. Do not silently shorten sequences, remove all-linear targeting, or alter the prompt to solve an OOM; those changes define a different experiment.
At steps 0, 250, 500, 750, ..., evaluation processes every one of the 1,687
validation examples. “Step” means optimizer/global update after gradient
accumulation, not micro-batch count.
For each validation row, the evaluator:
prompt_messages exactly once.Every metric artifact records:
The production decoder is constrained to codes displayed in the current row, so its structural invalid-output count is zero by construction. This does not prove that the unconstrained model would always emit a valid code. The evaluator therefore also records the full-vocabulary next-token argmax:
invalid_outputs: constrained decoder diagnostic;unconstrained_invalid_outputs: honest standalone formatting diagnostic.After step 250, the selected run's unconstrained invalid count was also zero in the evaluations available at packaging time.
The released decoder applies one global additive bias to the score of the final
لا توجد علاقة (no_relation) option before taking argmax. The selected
best checkpoint uses:
text
no_relation_logit_bias = -6.6600000000000215
For a row with valid option logits z, inference is equivalent to:
python
adjusted_logits = z.copy()adjusted_logits[no_relation_index] += -6.66predicted_index = argmax(adjusted_logits)
All positive-relation logits remain unchanged. If z_no_relation is the raw
no-relation logit and z_best_positive is the highest raw positive-relation
logit, the released decoder selects no-relation only when:
text
z_no_relation - z_best_positive > 6.66
Without calibration (bias = 0), no-relation wins whenever its logit exceeds
the best positive logit by any amount. The bias is therefore another expression
of a decision-threshold adjustment: Yehia still uses argmax, but only after
the no-relation score has been shifted.
The value -6.66 is not a subtraction of 6.66 percentage points. Logits are
unbounded scores, not probabilities. If the adjusted valid-option logits are
passed through softmax, this shift multiplies the odds of no-relation relative
to the positive options by:
text
exp(-6.66) ~= 0.00128
This is approximately a 781-fold reduction in the no-relation odds. In a
two-option illustration, no-relation would need a raw probability above about
99.872% to remain the winner after this shift. There is no single equivalent
probability threshold for the real decoder because a row can contain several
positive options with different logits. The magnitude is nevertheless a strong
correction toward predicting positive relations.
The correction was useful because the selected adapter's raw constrained decision rule was conservative on development data. Its effect on the complete 1,687-row labeled validation split was:
| Decision rule | Positive predictions | TP | FP | FN | Positive precision | Positive recall | Positive micro F1 |
|---|---|---|---|---|---|---|---|
Raw constrained argmax (bias = 0) | 779 | 490 | 289 | 377 | 0.6290 | 0.5652 | 0.5954 |
Released calibrated argmax (bias = -6.66) |
Calibration recovered 104 additional true positives while adding 126 false positives. Precision decreased, recall increased, and positive micro F1 improved. The calibrated system still predicted no-relation for 678 of the 1,687 validation rows; the correction does not force every row to be positive.
The evaluator selects the bias on validation by maximizing positive micro F1, with accuracy and then smaller absolute bias used as tie-breakers. This process does not retrain or alter the adapter weights. Setting the bias to zero is sufficient to recover raw constrained decoding; no retraining is required.
Because the bias was selected using validation labels, all calibrated validation
scores are development metrics rather than untouched held-out results. No
official/test labels were used to select it. tools/stage_release.py freezes
the selected value in inference_config.json, and official/test inference must
reuse that value without further tuning. Selecting or changing the bias using
hidden test labels would invalidate the held-out evaluation.
The locked checkpoint-selection metric is generation/decision-based validation positive micro F1. It is not training loss or teacher-forced validation loss. The trainer maintains:
checkpoints/checkpoint-<step>/: resumable Trainer checkpoints;latest_checkpoint/: complete named copy of the final step-2,943 checkpoint;best_checkpoint/: complete named copy of the best validation checkpoint;best_adapter/: adapter/tokenizer exported after training completes.Numbered checkpoint rotation never deletes the named latest/best copies.
These values cover every scheduled full validation from the completed selected run. They are engineering/development evidence, not final official/test claims.
| Step | Positive P | Positive R | Positive F1 | Accuracy | Macro F1 | Weighted F1 | Decision NLL | Unconstrained F1 | Unconstrained invalid |
|---|---|---|---|---|---|---|---|---|---|
| 0 | 0.2793 | 0.5225 | 0.3640 | 0.2964 | 0.5420 | 0.2423 | 2.3701 | 0.3295 | 2 |
Step 2,750 is the selected best checkpoint by the locked positive-micro-F1 metric. Step 1,500 retained the strongest accuracy and weighted F1, step 2,750 the strongest macro F1, step 500 the best decision NLL, and step 2,500 the best unconstrained positive F1. This divergence is why complete scheduled evaluation and durable best-checkpoint preservation matter even when training loss falls.
After step 2,943, the trainer reloaded the selected step-2,750 adapter and ran
the complete final validation again. It reproduced positive precision 0.588702,
recall 0.685121, F1 0.633262, accuracy 0.697095, macro F1 0.688899,
weighted F1 0.682849, and zero unconstrained invalid outputs. Aggregate
three-epoch train loss was 0.218184; this is not the checkpoint-selection
metric.
The machine-readable snapshot is
evidence/validation_history.json.
The selected run was validated on:
requirements.txt.The core pinned versions are:
| Package | Version |
|---|---|
| PyTorch | 2.5.1 |
| Transformers | 4.57.1 |
| Tokenizers | 0.22.1 |
| PEFT | 0.17.1 |
| bitsandbytes | 0.45.5 |
| Accelerate | 1.4.0 |
| datasets support | Local JSONL; no datasets runtime dependency required |
| huggingface_hub | 0.36.0 |
| scikit-learn |
Use the pinned stack first. If a different GPU/driver requires a compatibility change, record the exact failure and smallest changed dependency before claiming reproducibility.
Copy .env.example to .env and provide tokens locally. Never commit .env.
Token responsibilities are deliberately separate:
HF_TOKENONE: gated read access to Navid-AI/Yehia-7B-preview only;HF_TOKENTWO: U4RASD dataset access and optional U4RASD dataset/adapter
publication.No script prints token values. The clean repository contains token variable
names only. The preferred workflow downloads Yehia once into
models/Yehia-7B-preview and performs all subsequent model loads locally.
bash
git clone <this-repository-url> DRU-RE-Yehiacd DRU-RE-Yehiacp .env.example .env
Edit .env and set HF_TOKENONE and HF_TOKENTWO. Keep the exact dataset
revision, exact Yehia base revision, and selected hyperparameters unchanged for
a faithful reproduction. The selected base commit is
b9dda4715eafee7e8090d2c83cfe078d75f4ebb8; future downloads do not silently
follow a moving main branch.
bash
chmod +x setup_env.sh train.sh train_tmux.sh build_dataset.sh./setup_env.shsource .venv/bin/activate
The setup script installs the pinned stack, compiles every final Python entry point, checks Bash syntax, imports the QLoRA stack, reports CUDA/BF16 status, and runs the local prompt/resource self-test.
bash
python download_dataset.pypython audit_dataset.py
download_dataset.py pins the verified dataset commit from .env, downloads
only dataset/resource/card files, and checks all three row counts and hashes.
audit_dataset.py then performs the exhaustive content audit.
bash
./train_tmux.sh
Attach and detach with:
bash
tmux attach -t dru_re_yehia_training# detach with Ctrl+b, then d
To run in the foreground instead:
bash
./train.sh
On the first run, model.py uses an existing local Yehia snapshot if
models/Yehia-7B-preview/config.json exists. Otherwise it downloads the gated
base repository at the pinned revision with HF_TOKENONE into that directory.
Training and later inference load only from the local snapshot.
The already-published transformed revision is the simplest way to reproduce training. To reproduce the transformation itself, either:
SOURCE_DATASET_DIR to a local directory containing the three exact
source filenames; orHF_TOKENTWO read access to
U4RASD/WojoodRelationsAnnotated.Keep these defaults:
dotenv
PUSH_SFT_DATASET=falseOVERWRITE_DATASET_OUTPUT=falseSEED=42PROMPT_CONTEXT_CHARS=500SHUFFLE_OPTIONS=true
Then run:
bash
./build_dataset.sh
The launcher runs the builder and then the independent audit. If the output
directory already exists, the builder stops rather than deleting it. Set
OVERWRITE_DATASET_OUTPUT=true only when replacement is intentional.
To publish an independently verified rebuild, explicitly set
PUSH_SFT_DATASET=true. Publishing is never the default.
bash
tail -f logs/train_tmux.log
The internal Python log is normally logs/train.log.
bash
watch -n 1 nvidia-smi
bash
./tools/monitor_resources.sh
The default CSV is written inside the ignored run directory at
runs/DRU-RE-Yehia/artifacts/resource_monitor.csv.
Render once:
bash
python tools/plot_losses.py --once
Continuously refresh every 20 seconds:
bash
python tools/plot_losses.py
The generated PNGs are placed under runs/DRU-RE-Yehia/plots/ and are ignored
by Git.
CHOICE_RESUME=true is the selected default. When training starts, the trainer
uses Transformers' get_last_checkpoint in
runs/DRU-RE-Yehia/checkpoints/. Restarting train.sh or train_tmux.sh
therefore resumes the newest numbered checkpoint, including optimizer,
scheduler, RNG, and global-step state.
Before resuming, do not change:
Changing those values while reusing optimizer state invalidates a strict reproduction. Start a new output directory for a new experiment.
Training writes only to ignored local directories:
text
runs/DRU-RE-Yehia/├── run_config.json├── run_summary.json # written after full completion├── artifacts/│ ├── all_linear_eligible_modules.json│ ├── lora_wrapped_modules.json│ ├── decision_token_inventory.json│ ├── decision_token_masking_audit.json│ ├── sampling_strategy.json│ ├── smoke_test.json│ └── train_class_counts.json├── checkpoints/│ └── checkpoint-<step>/├── best_checkpoint/├── latest_checkpoint/├── best_adapter/ # written after completion├── metrics/│ └── validation_choice_metrics_step_<step>.json├── predictions/│ └── validation_choice_step_<step>.jsonl└── plots/├── train_loss.png└── eval_loss.png
The full 7B base model is never written inside the run or adapter repository.
After all three epochs and final validation finish, run:
bash
python tools/stage_release.py
The staging utility refuses to proceed unless:
run_summary.json exists and declares status=complete;best_adapter/ contains adapter config and safetensors;target_modules="all-linear";latest_checkpoint/ and numbered checkpoint-2943/ are byte-identical;It copies only:
All correctness checks finish before repository files are changed. Each staged file is copied through a temporary sibling and atomically renamed, preventing an interrupted copy from appearing as a complete release artifact.
It explicitly excludes:
The full repository is then ready for review and Hugging Face upload. Publish
the trained adapter and checkpoint only with the supplied Hugging Face client.
Authenticate with the U4RASD token without putting its value in a command
history or repository file. Build Azure's separate weight-free copy with
tools/stage_azure_release.py.
Staging also replaces PEFT's machine-local
base_model_name_or_path=/root/workspace/... metadata with the portable base
repository ID and exact commit. This does not alter adapter tensors; it prevents
a fresh machine from trying to resolve a path that existed only on the training
pod.
The deletion-safe release is intentionally split across durable repositories:
| Asset | Durable location | Why it is needed |
|---|---|---|
| Exact transformed train/validation/official JSONL and resources | Hugging Face dataset U4RASD/Yehia-RE-SFT at commit a060e47f56025778b97344d4d3de60a8fd53be7c | Retraining and official inference input |
| Source, resources, documentation, 41 examples, evidence, release tools, and submission artifacts | Azure Git and Hugging Face model U4RASD/DRU-RE-Yehia | Reproduce, audit, train, and understand the system |
Best adapter, tokenizer files, inference_config.json, final metrics, and release manifest | Hugging Face model repository only | Actual model inference and checkpoint provenance |
The Hugging Face adapter is not a standalone 7B model. It is deliberately about 160 MB rather than a copy of the full base weights. Future inference requires continued authorized access to the pinned gated Yehia base. If the upstream base is removed or access is revoked, the adapter alone cannot reconstruct it. Do not mirror the base into Azure or the model repository unless its license and owner explicitly permit redistribution.
After staging the full Hugging Face release, run:
bash
python tools/verify_release.py --repo-dir .
The verifier fails if any required source/resource/example/release file is missing. It also checks:
submission.zip whose CRC and recorded hashes pass;This command intentionally fails before the final adapter is staged. A source folder by itself is reproducible code, not a finished model release. The weight-free Azure package has a different intentional contract and must pass:
bash
python tools/verify_azure_release.py --repo-dir . --remote-clean
That verifier fails if it finds any safetensors file, Trainer checkpoint, adapter chunk, model binary, cache, credential file, or unexpected payload.
After local verification passes and .env contains HF_TOKENTWO, run:
bash
python tools/publish_huggingface.py
That publisher is hard-locked to U4RASD/DRU-RE-Yehia. It:
.env, caches, local data,
models, runs, logs, and the disposable predictions/ work directory (the
reviewed submission/ artifacts and root submission.zip are included);release/hf_remote_verification.json record;The printed final_revision is the immutable Hugging Face model commit to keep
with the experiment record. No token value is printed or persisted.
The durable Azure target for this release is branch
add-dru-re-entitypair-twohead of KG_eval-2026, under the new directory
DRU-RE-Yehia/. Azure is deliberately the code, documentation, evidence, and
submission mirror. It must contain no adapter weights, split adapter chunks,
optimizer state, or Trainer checkpoint. The best adapter and full latest
checkpoint are stored only in U4RASD/DRU-RE-Yehia on Hugging Face.
Stage the Azure-specific copy from the already verified full release:
bash
git clone --branch add-dru-re-entitypair-twohead --single-branch \https://dev.azure.com/AC-DU/CodeBox/_git/KG_eval-2026 KG_eval-2026python /path/to/DRU-RE-Yehia/tools/stage_azure_release.py \--source /path/to/DRU-RE-Yehia \--destination KG_eval-2026/DRU-RE-Yehia \--replacegit -C KG_eval-2026 add DRU-RE-Yehiagit -C KG_eval-2026 commit -m 'Publish weight-free DRU-RE-Yehia reproducibility release'git -C KG_eval-2026 push origin add-dru-re-entitypair-twohead
Authenticate through a credential manager or masked prompt. Never embed an Azure token in the remote URL, shell history, README, or Git files. If the Azure repository already has history, integrate it normally instead of overwriting it.
Azure DevOps Git is a durable source/model mirror, not automatically an Azure Machine Learning online endpoint. Deployment to Azure ML would additionally need an approved compute/environment image, scoring entrypoint, and endpoint configuration; those are deployment choices, not retraining prerequisites.
Before pod deletion, verify that Azure has all reproducibility and submission
files and zero model payloads. Use an environment outside the checkout so
--remote-clean examines a pristine repository:
bash
git clone --branch add-dru-re-entitypair-twohead --single-branch \https://dev.azure.com/AC-DU/CodeBox/_git/KG_eval-2026 \/tmp/KG_eval-2026-azure-checkpython3 -m venv /tmp/dru-release-verifier/tmp/dru-release-verifier/bin/pip install -r \/tmp/KG_eval-2026-azure-check/DRU-RE-Yehia/requirements.txt/tmp/dru-release-verifier/bin/python \/tmp/KG_eval-2026-azure-check/DRU-RE-Yehia/tools/verify_azure_release.py \--repo-dir /tmp/KG_eval-2026-azure-check/DRU-RE-Yehia \--remote-cleangit -C /tmp/KG_eval-2026-azure-check rev-parse HEAD
The Azure verifier rejects .safetensors, .pt, .pth, .bin, split-part
files, latest_checkpoint/, azure_large_files/, .env, and unrecognized
files. It still validates the complete 2,074-row submission, prompt/examples,
dataset/base provenance, all-linear evidence, final metrics, and immutable
Hugging Face publication pointer.
The two durable repositories together are sufficient to:
In Hugging Face, root adapter_model.safetensors is the selected step-2,750
best inference adapter. Hugging Face's
latest_checkpoint/adapter_model.safetensors is the different final step-2,943
training state accompanied by its optimizer, scheduler, RNG, trainer state,
and training arguments. Applications should use the root best adapter; the
checkpoint directory exists for archival resume/recovery. Neither file nor any
lossless encoding of its bytes belongs in Azure.
To restore it into the default run layout:
bash
python tools/restore_latest_checkpoint.py
This verifies every checkpoint hash and tensor first, then atomically creates:
text
runs/DRU-RE-Yehia/checkpoints/checkpoint-2943/
With the locked three-epoch configuration, that checkpoint is already at the training endpoint, so a normal resume performs no additional optimizer update and proceeds to finalization. Extending it to more epochs or changing the learning-rate schedule is a new experiment, not an exact continuation of the documented run. Older intermediate checkpoints are intentionally excluded.
The training machine is safe to delete only after every item below is true:
run_summary.json says status=complete.tools/stage_release.py staged the best—not merely latest—adapter.latest_checkpoint/checkpoint_manifest.json proves the complete final
step-2,943 optimizer/scheduler/RNG checkpoint is present.tools/verify_release.py passed.submission.zip contains only predictions.txt; all 2,074 lines, IDs,
class names, source order, hashes, and CRC pass the release verifier.tools/publish_huggingface.py passed its fresh final-download check.Until that entire gate passes, deleting the pod is premature. After it passes, temporary caches, plots, rolling logs, downloaded base files, transformed local copies, and live checkpoints on this machine are disposable.
The released system has two inference stages:
text
raw sentence + two mentions + two coarse types│▼prepare_input.pymarkers + compatible Arabic options + row-local codes + exact prompt│▼predict.pynext-token code scores + frozen no-relation bias + row-local decoding│▼Arabic answer + canonical relation metadata
The model is a relation classifier, not a named-entity recognizer. The caller must supply the two entity mentions, their direction, and their coarse types.
predict.py accepts already transformed rows containing prompt_messages,
aligned Arabic options/codes, and canonical mappings. After release staging:
bash
python predict.py \--input data/Yehia-RE-SFT/official.jsonl \--output predictions/official_predictions.jsonl
For a short smoke test:
bash
python predict.py --limit 8
The reference one-command path is:
bash
bash build_submission.sh
If the exact pinned transformed dataset is absent, the script downloads it from
U4RASD/Yehia-RE-SFT at commit
a060e47f56025778b97344d4d3de60a8fd53be7c. predict.py likewise downloads
the gated Yehia base at commit
b9dda4715eafee7e8090d2c83cfe078d75f4ebb8 when its verified local snapshot is
absent. It then loads the root best adapter, runs all 2,074 unlabeled official
rows, and produces:
text
submission.zipsubmission/predictions.txtsubmission/official_predictions.jsonlsubmission/run_summary.json
The builder refuses a different source hash/count/order, any official gold
label, a missing or duplicate triple_id, malformed row-local options, an
illegal prediction, an inconsistent no-relation bias, or a malformed ZIP.
submission.zip contains only root predictions.txt; each line is:
text
triple_id<TAB>canonical_relation
The dataset/model use no_relation internally. Only the competition payload
converts that value to the required no-relation. The current artifact has
SHA-256
a6bbd74ff4e32a570ac3fc1c12ce66aa3fe0c47986e66a3906a36231f3cbb0d5,
contains 904 negative and 1,170 positive predictions, and preserves the exact
official ID order. Those counts describe an unlabeled prediction distribution;
they are not an official-set accuracy claim.
For an audit-only deterministic repackage of an already generated model dump:
bash
python create_submission.py \--predictions-jsonl submission/official_predictions.jsonl
First inspect the 21 accepted canonical coarse type names:
bash
python prepare_input.py --list-types
Then prepare a row. When each mention occurs exactly once, character spans can be inferred safely:
bash
python prepare_input.py \--id demo_1 \--sentence 'يعمل أحمد في جامعة بيرزيت.' \--subject 'أحمد' \--object 'جامعة بيرزيت' \--subject-type PERS \--object-type ORG \--output predictions/prepared_inputs.jsonl
The subject is always [الأول], the object is always [الثاني], and the
question is subject-to-object. To ask the reverse question, prepare a separate
row with subject/object and their types reversed.
If a mention string occurs more than once, prepare_input.py deliberately
stops instead of guessing. Supply zero-based, end-exclusive Unicode character
offsets:
bash
python prepare_input.py \--sentence '...' \--subject '...' --subject-start 12 --subject-end 18 \--object '...' --object-start 31 --object-end 39 \--subject-type PERS --object-type ORG
Finally run the adapter on the prepared row:
bash
python predict.py \--input predictions/prepared_inputs.jsonl \--output predictions/demo_predictions.jsonl
prepare_input.py uses the same marker insertion, 21-type Arabic mapper,
directed ontology filter, 40 packaged Arabic templates, deterministic seed-42
shuffle, 500-character context logic, exact v3 prompt, and final
لا توجد علاقة rule as dataset construction. It forces any accidental input
relation field to empty, so gold labels cannot leak into inference prompts.
Pass --input a source-shaped JSONL file. One minimal line looks like:
json
{"id":"request_1","sentence":"يعمل أحمد في جامعة بيرزيت.","subject":"أحمد","object":"جامعة بيرزيت","subject_start":5,"subject_end":9,"object_start":13,"object_end":25,"subject_type":"PERS","object_type":"ORG"}
Exact spans are strongly recommended in production. The input need not include relation labels, prompt text, option lists, Arabic type names, or entity markers; the preparation utility derives them from the packaged resources.
predict.py computesThe script:
رمز الإجابة:;argmax and maps that row-local index back to the Arabic phrase,
canonical relation, and ontology ID.It does not ask the model to freely generate a phrase. Consequently, every returned code is structurally valid for its row. The saved unconstrained generation metrics remain in the evaluation evidence as an honest diagnostic of whether the model also follows the output format without this guardrail.
Each output line contains fields such as:
json
{"id": "official_example_id","predicted_option_index": 2,"predicted_code": "ج","predicted_option_ar": "الأول يعمل لدى الثاني","predicted_relation_full": "Affiliation.employee_of","predicted_relation_ontology_id": "employee_of","no_relation_logit_bias": -6.6600000000000215}
The shown bias is the frozen value selected on the labeled development split for the released best checkpoint. It is reused unchanged on official rows.
Do not call a generic text-generation pipeline on the bare adapter and then
interpret أ as a permanent class. A correct reimplementation must preserve
the row's option order, score only its valid code tokens, apply the frozen
no-relation bias, and decode through the same index. The supplied
prepare_input.py and predict.py are the reference implementation.
.env.latest_checkpoint/ copy.The official per-row predictions are intentionally included under
submission/ because they audit the submitted ZIP; unlabeled official rows are
not validation labels or training data.
The Azure mirror is intentionally narrower: in addition to everything listed
above, it omits the trained root adapter and the entire latest_checkpoint/
directory. Their hashes and provenance remain in its evidence, while their
actual bytes exist only in the Hugging Face model repository.
This keeps the model repository understandable while preserving every file needed to download, audit, rebuild, retrain from step 0, evaluate, stage, use the selected system, and restore the final optimizer state.
Confirm that HF_TOKENONE belongs to an account with access to
Navid-AI/Yehia-7B-preview. Do not substitute HF_TOKENTWO; the roles are
separate.
If HF_HUB_OFFLINE=1, Yehia must already exist at
LOCAL_YEHIA_MODEL_DIR. Disable offline mode for the first download, then
re-enable it for subsequent local-only runs.
Run:
bash
python download_dataset.pypython audit_dataset.py
Do not point the trainer directly at the source WojoodRelations files; they do not contain the transformed prompt/code fields.
This is a safety stop. Audit or move the existing directory first. Set
OVERWRITE_DATASET_OUTPUT=true only when a destructive rebuild is intended.
First use physical batch 2 and gradient accumulation 8. Keep max length 1,024, effective batch 16, all-linear targeting, and the same LoRA rank. Reduce validation batch separately if evaluation is the only failing stage.
This indicates a tokenizer/base-model mismatch. Use Yehia's native tokenizer from the same local snapshot and do not replace the option-code inventory.
Do not pre-render chat-template tokens in the dataset. Records must contain raw
messages; model.py applies the native template. A boundary error protects
against double templating or tokenizer changes.
Attach to the existing session or stop it deliberately. The launcher refuses to create a second trainer with the same session name.
Use complete scheduled validation and the locked positive-micro-F1 selection metric. Do not replace the preserved best checkpoint with the latest checkpoint based only on training loss.
Before claiming a faithful retrain, verify:
a060e47f56025778b97344d4d3de60a8fd53be7c.b9dda4715eafee7e8090d2c83cfe078d75f4ebb8.audit_dataset.py passes all 19,447 rows.yehia_re_ar_v3_one_token_choice.لا توجد علاقة is last in every row.No new license file is invented by this package. Before making a public release, the repository owner must verify and communicate the licenses, access terms, and redistribution requirements of the Yehia base model, the source dataset, and the derived dataset. A PEFT adapter does not remove those upstream obligations.
GPE.official_language |
| Unlabeled |
| Unlabeled |
| Final inference only |
template_ar42, split identity, and row identity, then shuffle only positive
compatible choices.لا توجد علاقة after the
positive shuffle. It is always the final displayed option.أ، ب، ج، د، ه، و، ز، ح، ط، ي.object_start/end | Original second-entity character span |
subject_type, object_type | Canonical metadata types, never rendered |
first_type_ar, second_type_ar | Arabic types rendered in the prompt |
marked_sentence | Context with directional markers |
window_start_char, window_end_char | Selected source-context boundary |
context_was_windowed | Whether context was shortened |
allowed_relation_full_labels | Canonical labels in displayed order |
allowed_relation_ontology_ids | Ontology IDs in displayed order |
allowed_options_ar | Arabic phrases in displayed order |
option_codes | One-token codes aligned with displayed options |
option_records | Full reversible mapping records |
gold_relation_full | Gold canonical relation on labeled splits |
gold_relation_ontology_id | Gold ontology ID for a positive row |
gold_answer_ar | Gold displayed Arabic phrase |
gold_option_index | Zero-based displayed gold position |
gold_answer_code | Actual one-token supervised target |
prompt_messages | System and user messages only |
messages | Prompt plus assistant code on labeled rows |
prompt_version | yehia_re_ar_v3_one_token_choice |
resource_manifest.json | Resource provenance and integrity metadata |
| 1,024 |
| Gradient checkpointing | Enabled |
Training use_cache | False |
| PEFT target | target_modules="all-linear" |
| LoRA rank | 16 |
| LoRA alpha | 32 |
| LoRA dropout | 0.05 |
| RS-LoRA | Enabled |
| LoRA bias | None |
| PEFT task type | CAUSAL_LM |
| Physical train batch | 4 |
| Gradient accumulation | 4 |
| Effective batch | 16 |
| Validation batch | 16 |
| Epochs | 3 |
| Optimizer steps | 2,943 on the exact training split |
| Learning rate | 5e-5 |
| Weight decay | 0.0 |
| Warmup | 5% |
| Scheduler | Cosine |
| Optimizer | paged_adamw_8bit |
| Gradient clipping | 1.0 |
| Train logging | Every 20 optimizer steps |
| Evaluation/save | Every 250 optimizer steps |
| Numbered checkpoint retention | 2 |
| Random seed | 42 |
| Sampling | Inverse class frequency, exponent 0.25 |
Wrapped o_proj | 32 |
Wrapped gate_proj | 32 |
Wrapped up_proj | 32 |
Wrapped down_proj | 32 |
Output lm_head adapted | No |
| Embeddings adapted | No |
| Quantized base weights trainable | No |
| Trainable parameters | 39,976,960 |
| Trainable percentage | 0.567811% |
| 1,009 |
| 594 |
| 415 |
| 273 |
| 0.5887 |
| 0.6851 |
| 0.6333 |
| 250 | 0.5674 | 0.6021 | 0.5842 | 0.6586 | 0.6391 | 0.6477 | 1.3317 | 0.5754 | 0 |
| 500 | 0.5785 | 0.6159 | 0.5966 | 0.6823 | 0.6691 | 0.6590 | 1.3186 | 0.5944 | 0 |
| 750 | 0.5913 | 0.5825 | 0.5869 | 0.6609 | 0.6405 | 0.6461 | 1.5595 | 0.5820 | 0 |
| 1000 | 0.5560 | 0.6471 | 0.5981 | 0.6598 | 0.6842 | 0.6458 | 1.9229 | 0.5758 | 0 |
| 1250 | 0.5356 | 0.6586 | 0.5908 | 0.6343 | 0.6699 | 0.6300 | 2.0380 | 0.5904 | 0 |
| 1500 | 0.6006 | 0.6678 | 0.6324 | 0.7030 | 0.6625 | 0.6849 | 2.2494 | 0.5807 | 0 |
| 1750 | 0.6045 | 0.6205 | 0.6124 | 0.6882 | 0.6429 | 0.6687 | 2.1202 | 0.5940 | 0 |
| 2000 | 0.6121 | 0.6298 | 0.6208 | 0.7012 | 0.6462 | 0.6814 | 2.2389 | 0.5944 | 0 |
| 2250 | 0.5824 | 0.6851 | 0.6296 | 0.6935 | 0.6849 | 0.6790 | 2.1502 | 0.5938 | 0 |
| 2500 | 0.5902 | 0.6678 | 0.6266 | 0.6924 | 0.6831 | 0.6767 | 2.3231 | 0.5999 | 0 |
| 2750 | 0.5887 | 0.6851 | 0.6333 | 0.6971 | 0.6889 | 0.6828 | 2.2822 | 0.5954 | 0 |
| 1.5.2 |
Latest complete checkpoint-2943 adapter, optimizer, scheduler, RNG, trainer state, tokenizer, and training arguments | latest_checkpoint/ in the Hugging Face model repository only | Exact archival resume state from the completed run |
| Ready official prediction archive plus auditable row-level decisions | submission.zip and submission/ in both model and Azure repositories | Submit directly or verify/regenerate all 2,074 predictions |
Yehia 7B base at commit b9dda4715eafee7e8090d2c83cfe078d75f4ebb8 | Upstream gated Navid-AI/Yehia-7B-preview | Required underneath the PEFT adapter |
| Token permissions | An external password manager or secret store, never either repository | Future access to gated/private repositories |
final_revision was recorded.tools/verify_azure_release.py --remote-clean and reported
trained_payloads_in_azure=0.target_modules="all-linear".lm_head is frozen.no-relation for the negative class).tools/verify_release.py passes on the local staged folder.markdown
tools/verify_release.py --remote-clean
tools/verify_azure_release.py --remote-clean
with zero trained payloads.