Commit Graph

34 Commits

Author SHA1 Message Date
sinohqb
3aac5e8068 feat(campaign): drawer create form and scenario-lane run timeline
Create surface moves from a 640px modal to a 920px two-column drawer:
basic info + time/speed on the left, plan preview and a grid-aligned
plan editor (headers, searchable scenario selects, scrollable entries)
on the right, so many-entry plans stay editable. The expanded-row
process timeline switches from a single crowded axis to per-scenario
lanes with a now-line for active campaigns.
2026-08-02 00:58:03 +08:00
sinohqb
67a574da18 feat(campaign): expandable row process timeline
Each campaign row expands to a per-Run process timeline: on expand it fetches
GET /campaigns/{id}/timeline and places each child Run on the shared
WindowTimeline by accelerated window offset, coloured by run status, with
scenario/pass-rate/latency tooltips and click-through to the run report.
Running campaigns refresh on the existing 5s poll; terminal ones fetch once.
WindowTimeline gains a colorMap prop for semantic status colours. (v0.6 ticket 10)
2026-07-31 17:03:10 +08:00
sinohqb
b295f7370d feat(campaign): live plan timeline preview in create form
Above the plan editor, adapt each plan entry (offset, scenario, count) into
WindowTimeline markers coloured per scenario with a count badge, re-rendering
live as the form changes via Form.useWatch. Empty plan shows a placeholder.
Purely form-local, no backend change. (v0.6 ticket 09)
2026-07-31 16:59:03 +08:00
sinohqb
f4db90a989 feat(campaign): add shared read-only WindowTimeline component
A domain-agnostic horizontal timeline: an axis over the service-cycle window
with offset-positioned markers, stable per-colorKey colours, optional badges,
hover tooltips, click callbacks, and an optional legend. Shared base for the
plan preview (09) and process timeline (10). (v0.6 ticket 08)
2026-07-31 16:57:42 +08:00
sinohqb
0e1ab75d77 feat(campaign): reverse time-scale input as target runtime
Users now pick a window length + "how long it should actually take" and the
form derives time_scale (window ÷ target), showing 倍速 ×N and 加速后耗时
read-only; a 实时 switch pins real wall-clock (×1). Validation blocks a target
longer than the window. List/detail display accelerated duration instead of raw
×N. POST /campaigns contract unchanged. (v0.6 ticket 07)
2026-07-31 16:55:43 +08:00
sinohqb
8bc5aa6979 feat(campaign): add per-Run timeline seam + endpoint
build_campaign_timeline flattens a campaign's child Runs into offset-sorted
per-Run entries (distinct from the report's 12-bucket aggregation), reusing a
shared _run_window_offset口径 so both views place a run identically. Exposes
GET /campaigns/{id}/timeline and the api.ts type/call. (v0.6 ticket 06)
2026-07-31 16:49:04 +08:00
sinohqb
983a58d013 refactor(verdict): unify read path on authoritative case_outcomes
Read paths recomputed per-case pass/connectivity independently — report
generation, the logs endpoint, and the frontend each derived it, and the
frontend's every(passed) recompute ignored the engine's authoritative
verdict. Extract resolve_case_verdicts: a single pure seam that prefers
stored case_outcomes verbatim and approximates only for legacy runs. The
logs endpoint now surfaces case_verdicts so the frontend reads instead of
recomputing.
2026-07-31 14:11:58 +08:00
sinohqb
050c674ee2 refactor(frontend): extract useResource/usePolling shared hooks
Some checks failed
CI / test (push) Failing after 1m10s
Seven pages repeated the same load-on-mount + loading + try/finally +
reload-button skeleton, each re-implementing tab-active refresh, silent
polling, and (in two places) a hand-rolled requestId race guard. Extract two
composable hooks: useResource(fetcher, {tabPath, deps}) owning data/loading/
reload with a built-in race guard and auto tab-active refresh, and
usePolling(fn, ms, enabled) replacing the hand-written setInterval effects.
Migrate all seven pages onto them; Targets/Scenarios/ModelConfigs also gain a
uniform tab-active refresh they previously lacked. Verified via tsc --noEmit
and npm run build (no frontend test runner exists).
2026-07-31 11:01:24 +08:00
sinohqb
782916a283 refactor(metrics): type Run summary and converge cross-run aggregation
Give EvalRun.summary a typed RunSummary value (unified RunError, lenient
legacy parsing) so readers stop reaching into a schemaless dict, and route
every cross-run rollup — dashboard, scenario ranking, trend, campaign
report — through one aggregate_runs seam. Fixes the divergence where
stats averaged pass_rate over completed-only runs while the campaign
report counted faults as 0.0. Cross-run rule (ADR-0004): genuine faults
count 0.0, user-cancelled runs are excluded from both denominators.
2026-07-31 01:57:56 +08:00
sinohqb
7ed765726f feat(campaigns): live list progress, polling, and richer drill-down
Embed compact progress (completed/planned total + overall pass_rate,
reusing the report's aggregation) into GET /campaigns so the list drops
its N+1 report fetch. Poll list and open report drawer every 5s while the
tab is active and a campaign is still running. Show scenario version and
trigger source tags in the child-run drill-down.
2026-07-30 15:35:18 +08:00
sinohqb
c82532398b feat(campaigns): add Campaign management page with dual-axis report
Register a keep-alive "评估活动" tab that creates campaigns (target,
window, time_scale, static plan), lists them with live progress and
pass-rate, and opens a report drawer with a time-trend line, capability
summary, and drill-down into child Runs.
2026-07-30 14:09:12 +08:00
sinohqb
f1aa61edd0 chore: sync package-lock version to 0.5.0-dev
Some checks failed
CI / test (push) Failing after 41s
2026-07-29 15:43:29 +08:00
sinohqb
25b4c98dc8 docs(release): v0.5「准」发布说明与里程碑收尾
Some checks failed
CI / test (push) Has been cancelled
版本升至 0.5.0-dev;新增 release-notes-v0.5.md(判定语义 + 场景版本化
+ 领域文档基线);AGENT.md/AGENTS.md 更新里程碑路线图、判定语义速查与
CONTEXT.md/ADR 指引。
2026-07-29 15:43:15 +08:00
sinohqb
cbab55843b fix(reports): export via authed axios blob download instead of window.open
Some checks failed
CI / test (push) Failing after 2m50s
启用登录鉴权后,导出 HTML/MD/JSON 用 window.open 直连 API 无法携带
X-Auth-Token,服务端返回 401 导致导出失效。改为经 axios 拉取 blob
(拦截器自动附加凭据)后触发浏览器下载。
2026-07-29 15:35:34 +08:00
sinohqb
5dd3c4ff1c style(dashboard): redesign stat cards with fixed-height compact layout
Some checks failed
CI / test (push) Failing after 44s
累计执行卡片的“/ 今日 N”后缀按数值字号渲染导致换行、卡片高度不一。
StatCard 改为紧凑三行固定高度布局(标题/数值/副行),今日次数移到
副行小字显示,六张卡片高度一致。
2026-07-29 15:26:38 +08:00
sinohqb
d23b321225 fix(runs): mark orphaned running/pending runs failed on startup
Some checks failed
CI / test (push) Failing after 59s
评测任务是进程内 asyncio 任务,服务重启会中断执行且状态永远停在
running。启动时将遗留的 running/pending 运行标记为 failed(summary
写入 interrupted 错误),清理为尽力而为,不阻断启动。另将仪表盘最近
评测记录的触发方式与版本号标签位置对调。
2026-07-29 14:48:14 +08:00
sinohqb
fbac28bc7e feat(frontend): always show trigger source labels (手动/AI 助手/CLI)
Some checks failed
CI / test (push) Failing after 49s
运行列表、报告页运行下拉、仪表盘最近评测行不再隐藏手动标签;
报告头与对比 A/B 卡片新增"触发方式"(报告 payload 补 triggered_by)。
2026-07-29 14:34:32 +08:00
sinohqb
8e3cf82fcc feat(frontend): surface scenario version across dropdowns and views
Some checks failed
CI / test (push) Failing after 43s
场景启动下拉、报告页运行下拉(A/B)、场景筛选(版本范围 v1/v2)、
仪表盘最近评测行、场景预览与编辑弹窗标题统一带上考纲版本标识。
2026-07-29 14:20:09 +08:00
sinohqb
770d260750 feat(report): compare requires same scenario version (ticket 05)
Some checks failed
CI / test (push) Failing after 39s
对比报告可比性收紧为同场景同考纲版本(ADR-0001):跨版本 API 返回 400
(detail 含双方版本号),报告生成层抛 ValueError;前端对比候选按
同场景 + 同版本过滤,A 变更后自动清空不可比的 B。文档"尚未实现"标注移除。
2026-07-29 11:21:52 +08:00
sinohqb
0a47260237 feat(run): snapshot scenario version at run creation (ticket 04)
运行创建时快照场景考纲版本,三种触发来源(手动/AI 助手/CLI)一致;
迁移回填存量运行为其场景当前版本,孤儿运行回填 1。运行列表、
报告头与对比卡片展示 v{n} 版本标签。
2026-07-29 10:59:44 +08:00
sinohqb
43e05ee38d feat(scenario): system-maintained syllabus version (ticket 03)
场景新增整型 version(迁移回填 1,batch mode)。仅考纲字段
(cases / model_bindings / llm_config)变更时升版,元数据编辑不升版,
API 传入的 version 被忽略(ADR-0001)。前端场景列表展示版本标签。
2026-07-29 10:42:35 +08:00
sinohqb
8a526599ab feat(report): annotate connectivity cases and add judged pass rate (ticket 02)
报告层推导连通用例标记(无判定结果 + 每轮有回复 + 无用例级错误),
summary 新增 connectivity_cases 与 judged_pass_rate(无判定型用例时为 null)。
对比报告同步标注且连通用例按引擎口径计通过;总通过率口径不变(ADR-0002)。
2026-07-29 10:32:56 +08:00
sinohqb
9c564b575e feat(frontend): v0.4 login page, dashboard rebuild, reports UX, keep-alive refresh
- 登录页 + App 鉴权门 + X-Auth-Token 拦截器 + 菜单头部退出按钮
- 仪表盘重构:6 指标卡 / 趋势图 + 场景表现 / 最近记录 + 快捷操作 + 来源分布
- Reports 页重做:场景筛选、富选项下拉、allowClear、一键重置、同场景对比约束
- useOnTabActive:标签页激活自动刷新(根治 AI 助手评测记录"消失")
- ModelConfigs 12 列合并为 6 列;来源 Tag;chunk 告警阈值修正并记录原因
2026-07-28 17:41:08 +08:00
sinohqb
affbf60945 feat(models): add model capability metadata 2026-07-17 21:41:32 +08:00
sinohqb
cc79d3a625 feat(models): support mainstream model protocols 2026-07-17 20:58:27 +08:00
sinohqb
457dfed252 fix(frontend): refresh scenario model references 2026-07-17 20:34:45 +08:00
sinohqb
470ff5875f feat(models): add centralized model configuration 2026-07-17 20:02:43 +08:00
sinohqb
9293f9e842 feat(files): improve category and location layout 2026-07-17 18:13:43 +08:00
sinohqb
d7514f4e65 refactor(files): harden storage and split frontend
Add transactional file storage workflows, typed API contracts, recursive category handling, frontend component separation, and Files API coverage.
2026-07-17 17:41:19 +08:00
sinohqb
f765bee7a1 fix(frontend): 修复 vendor-charts 循环 chunk 导致的运行时崩溃
## 问题
线上打开首页报错:
vendor-charts-Z9UiCjXy.js: Uncaught ReferenceError:
Cannot access 'd' before initialization

## 根因
T4 bundle 优化把 @ant-design/charts 单独拆为 vendor-charts chunk,但它的深层
@antv/* 依赖图与 vendor chunk 交叉引用,形成 Circular chunk(构建时已有警告
"vendor-charts -> vendor -> vendor-charts",当时被忽略)。循环 chunk 导致
ES module 求值顺序错乱 → TDZ 错误。

尝试把 @antv 也并入 vendor-charts 仍循环(charts 生态还依赖 lodash/d3 等
公共库,与 vendor 交叉)。

## 修复
移除 vendor-charts 单独拆分,让 @ant-design/charts + @antv 回到 vendor 主 chunk,
消除跨 chunk 循环。保留 vendor-monaco / vendor-antd(构建证明无循环)。
charts 在 Home 首页(默认路由)即需加载,单独拆分的懒加载收益本就微乎其微;
真正的懒加载收益来自 App.tsx 的页面级 React.lazy(保留)。

构建验证:无 Circular chunk 警告,无 vendor-charts chunk。

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-17 16:04:09 +08:00
sinohqb
12c1732617 v0.4-t3t4: OpenClaw 通道 + 前端 bundle 优化
## T3: OpenClaw 直连通道
- channels/openclaw.py: OpenClawChannel
  - send: POST /api/v1/chat/completions
  - poll_reply: GET /api/v1/chat/completions/{msg_id}
  - health_check: GET /api/health
  - 默认从 settings 读取 upstream/auth_token,channel_config 可覆盖
- channels/factory.py: 注册 ChannelType.OPENCLAW → OpenClawChannel
- Targets.tsx: 通道类型下拉新增「HTTP 通用」和「OpenClaw」选项
- 8 个 OpenClawChannel 单元测试(发送/轮询/超时/健康检查/默认配置)

## T4: 前端 Bundle 优化
- App.tsx: 7 个页面改为 React.lazy 懒加载 + Suspense fallback(Spin)
- vite.config.ts: 精细化 manualChunks
  - vendor-antd / vendor-monaco / vendor-charts 独立拆分
  - 主 index 68KB → 7KB,页面按需加载
  - 无循环依赖警告

## 测试
- 178/178 全绿,覆盖率维持 77%

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-17 15:37:14 +08:00
sinohqb
17aeba80dd v0.3-s4: 场景模板库 + WS 自动重连 + PageWrapper 复用
## 场景模板库(S4-1)
- scenarios/templates.py: 6 个内置模板
  - 单轮问答基础 / 多轮对话 / 动态 LLM 生成 / 安全合规检测 / JSON 接口校验 / 加权评分
  - 每个模板附带对应规则配置(含 v0.3 新规则)
- routers/scenarios.py: GET /api/scenarios/templates + GET /api/scenarios/templates/{id}
- api.ts: scenariosApi.listTemplates() / getTemplate()
- Scenarios.tsx: 「从模板新建」按钮 + 卡片式模板选择弹窗
  - 选择后预填名称/描述/标签/cases JSON/llm_config,直接进入编辑 Drawer

## WebSocket 自动重连(S4-2)
- useRunSession.ts: connectWs() 函数 + 指数退避重连
  - 异常断开(非 1000/clean)时自动重试,最多 5 次
  - 延迟:1s → 2s → 4s → 8s → 16s(上限 30s)
  - 超出重试次数后降级 REST 获取最终状态
  - reconnectTimerRef 在组件卸载时清理,无内存泄漏

## PageWrapper 复用(S4-3)
- PageWrapper.tsx: 升级 inline 模式匹配全高页面的 padding 页头样式
- Home / Targets / Scenarios: 用 PageWrapper inline+fullHeight 替换重复内联页头
- Home.tsx: 去掉 unused `colors` import

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-17 12:05:18 +08:00
sinohqb
349200e51f v0.3-s3: Webhook + OpenClaw HTTP Skill + Markdown/对比报告
## Webhook 通知(S3-1)
- settings.py: 增加 AGENTEVAL_WEBHOOK_URL / AGENTEVAL_WEBHOOK_SECRET
- utils/webhook.py: send_run_webhook(),非阻断,任何异常仅 warning log
- runs.py: run 完成后自动触发 webhook(payload 含 run_id/status/summary/report_url)
- .env.example: 新增 webhook 配置示例

## OpenClaw Skill HTTP 改造(S3-2)
- plugins/openclaw/agenteval_skill.py: 完全重写
  - 改用 HTTP API(POST /api/runs + GET /api/runs/{id} 轮询 + GET /api/reports/{id})
  - 移除 subprocess + CLI 依赖
  - 轮询等待至 completed/failed,支持配置 poll_interval / timeout
  - 返回结构化中文摘要(summary_text),直接可用于 OpenClaw 对话展示

## Markdown 报告导出(S3-3)
- report.py: render_markdown_report() — 完整的 Markdown 表格 + 对话展示
- save_report: 支持 fmt="markdown",输出 .md 文件
- reports.py: GET /api/reports/{run_id}/markdown,Content-Disposition 附件下载
- api.ts: reportsApi.markdownUrl()
- Reports.tsx: 「导出 MD」按钮

## 对比报告(S3-4)
- report.py: generate_compare_report(run_id_1, run_id_2)
  - run_a / run_b 汇总 + delta(pass_rate / passed_cases / passed_rules)
  - case-level diff,标记 changed 用例
- reports.py: GET /api/reports/compare?run1=&run2=
- api.ts: reportsApi.compare()
- Reports.tsx: 完整对比视图
  - Segmented 切换「单次报告」/「对比报告」
  - 双 Select(报告 A vs B)+ 对比按钮
  - 汇总 delta card(pass_rate 变化 + 变化用例数徽章)
  - 用例对比表(通过/失败/改善↑/退步↓)+ 展开规则明细

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-17 11:44:54 +08:00
sinohqb
a77cd83e6a v0.2.0-dev: 文件管理 + 页面布局统一 + 6 个 bug 修复
## 新增功能
- 文件管理模块:分类树 + 文件上传/下载/删除
- 文件上传支持拖拽(Dragger)+ 手动上传(customRequest 模式)

## 页面布局统一(参照评测执行页)
- 仪表盘/评测对象/评测场景/评测报告 全部改为全高 flex 布局
- 统一内联页头样式(h2 + 竖线分隔 + 描述)
- 表格撑满高度、overflow 处理
- 每页添加刷新按钮

## Bug 修复
- 分类树操作按钮 hover 不可见(CSS 规则缺失)
- 文件上传失败(multipart boundary 缺失)
- LLM API 响应 content blocks 数组格式支持(_extract_content_from_api_response)
- response_time_max_ms 被静默忽略(隐式规则传空 params)
- 空 messages 导致 IndexError 崩溃
- poll_reply 异常中止整个 run(缺 try/catch)
- engine finally 未关闭 session
- 3 个页面 UTC 时间戳解析偏差 8 小时

## 后端
- EvalEngine: poll_reply 异常保护、空 dialog 保护、session 关闭
- LLM API 响应解析支持 content-block-array 格式
- 隐式 response_time 规则正确传递 max_ms 参数

## 前端
- api.ts: 移除手动 Content-Type(让浏览器自动添加 boundary)
- Files.tsx: customRequest 替代 beforeUpload、布局优化
- index.css: 分类树 hover 规则
- Targets/Scenarios/Home/Reports: 全高布局改造
- 3 个页面时间戳改用 formatDateTime()(修复 UTC 偏差)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-16 15:25:22 +08:00