openevo-qwen25-7b-webshop-sd-lora API & Inference Endpoint

Reproducibility identity (verify before use)

Table with columns: Item, Value
Item	Value
`adapter_model.safetensors` SHA-256	`47e63d417f85cb2defed5ff1ee934128b96e6a4f78a8f410fa052957d919190c`
`adapter_config.json` SHA-256	`ed48aae286e03ae63fe0f70c5e75dddd650f098e6d5cba55efb11106523c5261`
Loaded tensor digest	`eedc36232f04a8a7d6e8fc6a38b935758db65206f4945a6cfa35c4ab666a48f9`
Base model revision	`0c32faf81d8f2f83cf79860d2a48111f166348d6dd38cd348576d8b3a0141c1e`
Runtime image digest	`sha256:ce92e587e7c934827025c001fd9b70ec4cb6de2deacfdf9f0cc0777d3ab928f1`

Always confirm the adapter_model.safetensors SHA-256 matches 47e63d41… before using this as the frozen policy — this is the exact artifact the evaluation pinned.

Files

Table with columns: File, Purpose
File	Purpose
`adapter_model.safetensors`	The frozen LoRA weights (this is the evaluated artifact).
`adapter_config.json`	PEFT config: LoRA r=32, α=32, dropout 0, targets `q_proj`,`v_proj`.
`openevo_sd_lora_state.safetensors`	Full SD-LoRA training state (direction/scale components), for provenance.
`openevo_sd_lora_state.json`	Training-state manifest (adaptation scope, replay/direction metadata).
`openevo_sd_lora_replay.jsonl`

Only adapter_model.safetensors + adapter_config.json are needed for inference; the openevo_sd_lora_* files document how the adapter was produced.

Usage

python
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-7B-Instruct", device_map="auto")
model = PeftModel.from_pretrained(base, "miyuki17/openevo-qwen25-7b-webshop-sd-lora")
tok = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-7B-Instruct")

Output-format note (important for WebShop): this policy emits its chosen action wrapped in [action]…[/action] (sometimes a malformed [action> open tag) after a <think> block. If you evaluate it with SEED's released webshop_projection, that parser only recognises <action>…</action> and will silently fall back to a 20-character tail fragment, scoring every episode 0. Use a projection that accepts both wrappers (see the evaluation repo).

Evaluation (SEED official held-out WebShop, env.seed=0, goal_idx 0–499, 128-task panel)

Table with columns: Contract, BASE (no adapter), OpenEvo SD-LoRA (this)
Contract	BASE (no adapter)	OpenEvo SD-LoRA (this)
OpenEvo-native diagnostic — task score ×100	14.6	28.3
OpenEvo-native diagnostic — exact success	2.3%	1.6%
SEED-strict primary — task score ×100	0.0	0.0 (measurement-invalid, see note)

The SEED-strict primary result is measurement-invalid: SEED's parser cannot read this policy's [action] wrapper (see Output-format note), so both arms score 0 despite emitting valid search[...]/click[...] commands. It is not a policy result.
The valid local measurement is the OpenEvo-native diagnostic: the adapter roughly doubles continuous task score (14.6 → 28.3) but does not improve exact success — more partial progress, not more completed purchases.
SEED paper-reported reference (Table 1, Qwen2.5-7B-Instruct): 89.7 score / 78.1% success. This is a paper number; SEED training/checkpoint was not locally reproduced, and the 128-task panel is SEED-compatible, not the exact paper denominator.

Full evidence (all 512 episode records, reconciliation, analysis) lives in the evaluation repository: mykcs/openevo-experiment → docs/evidence/seed-official-heldout-comparison-v1/ (see RESULTS.md).

Training summary

Algorithm family: SD-LoRA — continual SFT (causal_lm_continual_sft_v4) with bounded trajectory replay and a frozen global unit-Frobenius direction.
Effective rank: 32; target modules: q_proj, v_proj.
Peak GPU memory during training: ~16.4 GB.
Source campaign: H1.38B method-control (20260820-0129-h138b-method-control).

Limitations & claim boundary

A single frozen adapter, not a model sweep; evaluated inference-only.
Does not reproduce SEED's 89.7; no exact apples-to-apples or causal claim is made.
Exact success on the held-out panel is low (≤2.3%); the adapter improves partial task progress, not completed purchases.

Citation

If you use this artifact, cite the OpenEvo experiment repository (mykcs/openevo-experiment) and the Qwen2.5 base model.

中文模型卡

这是基于 Qwen/Qwen2.5-7B-Instruct 的 冻结 SD-LoRA 适配器（rank 32），由 OpenEvo H1.38B method-control 战役（20260820-0129-h138b-method-control）产出，并在 SEED 官方 held-out WebShop 对比（20260825-0624-webshop-seed-official-heldout-comparison）中作为 OPEN_EVO_SD 臂被评测。

这是一个仅推理的评测产物：适配器在此冻结、原样发布，供独立 Agent 复现/审阅 held-out WebShop 测量。没有再训练、没有调参、没有扫参。

可复现身份（使用前必校验）

Table with columns: 项, 值
项	值
`adapter_model.safetensors` SHA-256	`47e63d417f85cb2defed5ff1ee934128b96e6a4f78a8f410fa052957d919190c`
运行时镜像 digest	`sha256:ce92e587e7c934827025c001fd9b70ec4cb6de2deacfdf9f0cc0777d3ab928f1`

把本适配器当作冻结政策使用前，务必确认 adapter_model.safetensors 的 SHA-256 等于 47e63d41… —— 这正是评测所 pin 的那个产物。

输出格式注意（WebShop 必看）

本政策在 <think> 块之后，用 [action]…[/action]（偶见残缺开标签 [action>）包裹动作。 SEED 官方 webshop_projection 只认 <action>…</action>，找不到时会退化成取末尾 20 字符的碎片，导致每个 episode 都得 0。评测时请使用同时兼容两种 wrapper 的 projection（见评测仓库）。

评测结果（SEED 官方 held-out，env.seed=0，goal_idx 0–499，128 任务面板）

Table with columns: 合同, BASE（无适配器）, OpenEvo SD-LoRA（本模型）
合同	BASE（无适配器）	OpenEvo SD-LoRA（本模型）
OpenEvo-native diagnostic — task score ×100	14.6	28.3
OpenEvo-native diagnostic — exact success	2.3%	1.6%
SEED-strict primary — task score ×100	0.0	0.0（测量无效，见上）

SEED-strict primary 是测量无效：SEED 解析器读不了本政策的 [action] 包裹，两臂虽吐出了合法 search[...]/click[...] 指令却全记 0 —— 这不是政策真实水平。
有效的本地测量是 OpenEvo-native diagnostic：本适配器把连续 task score 约翻倍（14.6 → 28.3），但不提升 exact success —— 更多“部分进度”，而非更多“完成购买”。
SEED 论文参考值（Table 1，Qwen2.5-7B-Instruct）：89.7 分 / 78.1% 成功率。这是论文 reported 数字；SEED 训练/checkpoint 未在本地复现，128 面板是 SEED-compatible，并非论文确切分母。

完整证据（全部 512 条 episode、reconciliation、analysis）在评测仓库 mykcs/openevo-experiment 的 docs/evidence/seed-official-heldout-comparison-v1/ （见 RESULTS.md）。

训练摘要

算法族：SD-LoRA —— continual SFT（causal_lm_continual_sft_v4），bounded trajectory replay + 冻结全局单位 Frobenius 方向。
有效 rank：32；目标模块：q_proj、v_proj；训练峰值显存约 16.4 GB。

边界与限制

单一冻结适配器，非模型扫参；仅推理评测。
未复现 SEED 89.7；不做完全对齐或因果性声明。
held-out 面板 exact success 较低（≤2.3%）；适配器提升的是部分进度，不是完成购买。

引用

使用本产物请引用 OpenEvo 实验仓库（mykcs/openevo-experiment）与 Qwen2.5 基座模型。

Reproducibility identity (verify before use)

Table with columns: Item, Value
Item	Value
`adapter_model.safetensors` SHA-256	`47e63d417f85cb2defed5ff1ee934128b96e6a4f78a8f410fa052957d919190c`
`adapter_config.json` SHA-256	`ed48aae286e03ae63fe0f70c5e75dddd650f098e6d5cba55efb11106523c5261`
Loaded tensor digest	`eedc36232f04a8a7d6e8fc6a38b935758db65206f4945a6cfa35c4ab666a48f9`
Base model revision	`0c32faf81d8f2f83cf79860d2a48111f166348d6dd38cd348576d8b3a0141c1e`
Runtime image digest	`sha256:ce92e587e7c934827025c001fd9b70ec4cb6de2deacfdf9f0cc0777d3ab928f1`

Always confirm the adapter_model.safetensors SHA-256 matches 47e63d41… before using this as the frozen policy — this is the exact artifact the evaluation pinned.

Files

Table with columns: File, Purpose
File	Purpose
`adapter_model.safetensors`	The frozen LoRA weights (this is the evaluated artifact).
`adapter_config.json`	PEFT config: LoRA r=32, α=32, dropout 0, targets `q_proj`,`v_proj`.
`openevo_sd_lora_state.safetensors`	Full SD-LoRA training state (direction/scale components), for provenance.
`openevo_sd_lora_state.json`	Training-state manifest (adaptation scope, replay/direction metadata).
`openevo_sd_lora_replay.jsonl`

Only adapter_model.safetensors + adapter_config.json are needed for inference; the openevo_sd_lora_* files document how the adapter was produced.

Usage

python
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-7B-Instruct", device_map="auto")
model = PeftModel.from_pretrained(base, "miyuki17/openevo-qwen25-7b-webshop-sd-lora")
tok = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-7B-Instruct")

Evaluation (SEED official held-out WebShop, env.seed=0, goal_idx 0–499, 128-task panel)

Table with columns: Contract, BASE (no adapter), OpenEvo SD-LoRA (this)
Contract	BASE (no adapter)	OpenEvo SD-LoRA (this)
OpenEvo-native diagnostic — task score ×100	14.6	28.3
OpenEvo-native diagnostic — exact success	2.3%	1.6%
SEED-strict primary — task score ×100	0.0	0.0 (measurement-invalid, see note)

The SEED-strict primary result is measurement-invalid: SEED's parser cannot read this policy's [action] wrapper (see Output-format note), so both arms score 0 despite emitting valid search[...]/click[...] commands. It is not a policy result.
The valid local measurement is the OpenEvo-native diagnostic: the adapter roughly doubles continuous task score (14.6 → 28.3) but does not improve exact success — more partial progress, not more completed purchases.
SEED paper-reported reference (Table 1, Qwen2.5-7B-Instruct): 89.7 score / 78.1% success. This is a paper number; SEED training/checkpoint was not locally reproduced, and the 128-task panel is SEED-compatible, not the exact paper denominator.

Training summary

Algorithm family: SD-LoRA — continual SFT (causal_lm_continual_sft_v4) with bounded trajectory replay and a frozen global unit-Frobenius direction.
Effective rank: 32; target modules: q_proj, v_proj.
Peak GPU memory during training: ~16.4 GB.
Source campaign: H1.38B method-control (20260820-0129-h138b-method-control).

Limitations & claim boundary

A single frozen adapter, not a model sweep; evaluated inference-only.
Does not reproduce SEED's 89.7; no exact apples-to-apples or causal claim is made.
Exact success on the held-out panel is low (≤2.3%); the adapter improves partial task progress, not completed purchases.

Citation

If you use this artifact, cite the OpenEvo experiment repository (mykcs/openevo-experiment) and the Qwen2.5 base model.

中文模型卡

这是一个仅推理的评测产物：适配器在此冻结、原样发布，供独立 Agent 复现/审阅 held-out WebShop 测量。没有再训练、没有调参、没有扫参。

可复现身份（使用前必校验）

Table with columns: 项, 值
项	值
`adapter_model.safetensors` SHA-256	`47e63d417f85cb2defed5ff1ee934128b96e6a4f78a8f410fa052957d919190c`
运行时镜像 digest	`sha256:ce92e587e7c934827025c001fd9b70ec4cb6de2deacfdf9f0cc0777d3ab928f1`

把本适配器当作冻结政策使用前，务必确认 adapter_model.safetensors 的 SHA-256 等于 47e63d41… —— 这正是评测所 pin 的那个产物。

输出格式注意（WebShop 必看）

评测结果（SEED 官方 held-out，env.seed=0，goal_idx 0–499，128 任务面板）

Table with columns: 合同, BASE（无适配器）, OpenEvo SD-LoRA（本模型）
合同	BASE（无适配器）	OpenEvo SD-LoRA（本模型）
OpenEvo-native diagnostic — task score ×100	14.6	28.3
OpenEvo-native diagnostic — exact success	2.3%	1.6%
SEED-strict primary — task score ×100	0.0	0.0（测量无效，见上）

SEED-strict primary 是测量无效：SEED 解析器读不了本政策的 [action] 包裹，两臂虽吐出了合法 search[...]/click[...] 指令却全记 0 —— 这不是政策真实水平。
有效的本地测量是 OpenEvo-native diagnostic：本适配器把连续 task score 约翻倍（14.6 → 28.3），但不提升 exact success —— 更多“部分进度”，而非更多“完成购买”。
SEED 论文参考值（Table 1，Qwen2.5-7B-Instruct）：89.7 分 / 78.1% 成功率。这是论文 reported 数字；SEED 训练/checkpoint 未在本地复现，128 面板是 SEED-compatible，并非论文确切分母。

完整证据（全部 512 条 episode、reconciliation、analysis）在评测仓库 mykcs/openevo-experiment 的 docs/evidence/seed-official-heldout-comparison-v1/ （见 RESULTS.md）。

训练摘要

算法族：SD-LoRA —— continual SFT（causal_lm_continual_sft_v4），bounded trajectory replay + 冻结全局单位 Frobenius 方向。
有效 rank：32；目标模块：q_proj、v_proj；训练峰值显存约 16.4 GB。

边界与限制

单一冻结适配器，非模型扫参；仅推理评测。
未复现 SEED 89.7；不做完全对齐或因果性声明。
held-out 面板 exact success 较低（≤2.3%）；适配器提升的是部分进度，不是完成购买。

引用

使用本产物请引用 OpenEvo 实验仓库（mykcs/openevo-experiment）与 Qwen2.5 基座模型。

openevo-qwen25-7b-webshop-sd-lora

README

Reproducibility identity (verify before use)

Files

Usage

Evaluation (SEED official held-out WebShop, env.seed=0, goal_idx 0–499, 128-task panel)

Training summary

Limitations & claim boundary

Citation

中文模型卡

可复现身份（使用前必校验）

输出格式注意（WebShop 必看）

评测结果（SEED 官方 held-out，env.seed=0，goal_idx 0–499，128 任务面板）

训练摘要

边界与限制

引用

Explore FriendliAI today

README

Reproducibility identity (verify before use)

Files

Usage

Evaluation (SEED official held-out WebShop, env.seed=0, goal_idx 0–499, 128-task panel)

Training summary

Limitations & claim boundary

Citation

中文模型卡

可复现身份（使用前必校验）

输出格式注意（WebShop 必看）

评测结果（SEED 官方 held-out，env.seed=0，goal_idx 0–499，128 任务面板）

训练摘要

边界与限制

引用