Evaluation status
Comparison completed on 1 October 2026. On the same 233 prepared English conversations, reference-field matches changed from 4693/5126 (91.6%) to 4736/5126 (92.4%); raw follow-up decisions from 214/233 (91.8%) to 216/233 (92.7%). Fully matching conversations increased from 50 to 72, but unusable outputs increased from 2 to 3 and wrong raw sales referrals remained at 6. There were 139 improved field checks and 96 regressions.
On 60 additional prepared examples (two per 30 scenarios), field matches changed from 1148/1320 (87.0%) to 1299/1320 (98.4%); raw follow-up decisions from 52/60 to 59/60. Wrong raw referrals changed from 6 to 1. These examples share construction patterns with training; they are not independent real-customer evidence. The original 233 also informed improvement work. Text equality can reject valid paraphrases. These numbers are not model confidence or live website accuracy.
See eval/v4_comparison_summary.json for counts. Other historical files under eval/ describe v3. Use supervised demonstrations and review follow-up decisions; the raw model is not ready for unattended automatic sales routing.
Use and continued training
Load Qwen/Qwen2.5-1.5B-Instruct, then this repository with PeftModel.from_pretrained. Use system_prompt.txt and inference.py for the full beacon.qualification.v1 JSON format. Recompute score and routing using the approved rules before any follow-up action.
For further training, resolve the current main revision and load this adapter with is_trainable=True. Do not initialise a new adapter. Previous weights remain available in repository history. The live Beacon website is a separate deployment and is not switched by this upload.