solahqb/AgentEvalTool
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases 1 Wiki Activity
193 Commits 13 Branches 4 Tags 6 MiB
25920280f6
Commit Graph

193 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
sinohqb
62485684ca fix(architecture): enforce lifecycle consistency 2026-08-07 10:59:27 +08:00
sinohqb
c896ab3f71 refactor(architecture): deepen evaluation lifecycle and read model 2026-08-07 03:11:37 +08:00
sinohqb
f26c34a340 docs(ui-consistency): archive completed refactor, record drawer evolution in ADR-0005
Some checks failed
CI / test (push) Failing after 34s
Details
2026-08-06 02:01:36 +08:00
sinohqb
804880f1e3 fix(intelligent-eval): add top padding in drawer bodies 
抽屉 body padding 为 0，顶栏按钮与提示贴住头部分割线；
详情/报告根容器补 16px 顶部内边距。
 2026-08-06 01:52:46 +08:00
sinohqb
1fbd2bceed docs(ui-consistency): check off all ticket acceptance items after t480 walkthrough 2026-08-06 01:46:37 +08:00
sinohqb
9f25728c2f docs(agents): detail views may use right-side Drawer per ADR-0005 2026-08-06 01:42:13 +08:00
sinohqb
01f451c155 feat(intelligent-eval): align detail/report drawer layouts with other modules 
详情抽屉改两列布局（左静态上下文/右动态流程），会话进度加 Progress；
报告抽屉顶部加统计概览行与维度得分柱状图，移除总结卡内重复的分数展示。
 2026-08-06 01:03:28 +08:00
sinohqb
22cc437e2d feat(intelligent-eval): adjust drawer top-bar layout 
详情页移除返回列表按钮（抽屉自带关闭），操作按钮右对齐；
报告页保留返回详情，导出 Markdown 按钮右对齐。
 2026-08-06 00:37:02 +08:00
sinohqb
1d2c37277c feat(intelligent-eval): open detail/report in right-side drawer 
列表常驻，详情与报告改为右侧抽屉弹出，减少页面层级嵌套。
 2026-08-06 00:29:52 +08:00
sinohqb
935f0a1f57 refactor(ui): open intelligent eval via action-column buttons 
Replace the name link and row-click with explicit detail/report
buttons, matching the static-eval table pages. Report button only
shows for completed evals; list pagination now follows the 20-row
threshold.
 2026-08-05 21:00:19 +08:00
sinohqb
0832445862 docs(ui): record consistency baseline in AGENTS.md (ui-consistency ticket 09) 
Add the ADR-0005 interaction standard to the frontend architecture
section and fix the tabStore extension. Backend pytest and frontend
tsc both green.
 2026-08-05 20:29:07 +08:00
sinohqb
11b15908c7 refactor(ui): align Campaigns page with consistency baseline (ui-consistency ticket 08) 
Create-activity form switched to FormDrawer (width 920, footer submit),
20-row pagination threshold on campaign and sub-run tables. Report
drawer stays a plain Drawer (read-only with export action). Timeline,
exploration, and analysis interactions unchanged.
 2026-08-05 20:27:54 +08:00
sinohqb
5077f088b1 refactor(ui): align Reports page shell with consistency baseline (ui-consistency ticket 07) 
PageWrapper inline fullHeight with an inner flex column to keep the
selection bar / content split, and 20-row pagination threshold on the
compare-case table. Report selection, charts, and compare logic
unchanged.
 2026-08-05 20:26:43 +08:00
sinohqb
b9011aad80 refactor(ui): align Runs page shell with consistency baseline (ui-consistency ticket 06) 
Replace custom header with PageWrapper inline fullHeight and add
Popconfirm to the stop action. The nav-only-tabs content-split hack is
preserved; WebSocket session, rerun, and list logic untouched.
 2026-08-05 20:25:17 +08:00
sinohqb
1588972a8e refactor(ui): migrate Scenarios page to consistency baseline (ui-consistency ticket 05) 
FormDrawer with footer submit at width 720 (keeps Monaco room),
submitting guard, 20-row conditional pagination, Empty locale,
unified delete Popconfirm wording. Preview/template Modals kept as
read-only surfaces.
 2026-08-05 20:23:46 +08:00
sinohqb
f1316578fe refactor(ui): migrate ModelConfigs page to consistency baseline (ui-consistency ticket 04) 
FormDrawer with footer submit and submitting guard, 20-row conditional
pagination, Empty locale, unified delete Popconfirm wording. Capability
and modality validation logic unchanged.
 2026-08-05 20:22:14 +08:00
sinohqb
2e78c4befd refactor(ui): align Files page with consistency baseline (ui-consistency ticket 03) 
Delete confirmations already existed (d7514f4), so no behavior fix was
needed. Switch empty states to antd Empty (table + category tree) and
apply the 20-row pagination standard.
 2026-08-05 15:40:54 +08:00
sinohqb
15d0d127ae refactor(ui): migrate Targets page to consistency baseline (ui-consistency ticket 02) 
FormDrawer with footer submit, submitting guard, 20-row conditional
pagination, Empty locale, unified delete Popconfirm wording.
 2026-08-05 15:38:37 +08:00
sinohqb
d4c254e121 refactor(ui): baseline FormDrawer component and IntelligentEvals dogfood (ui-consistency ticket 01) 
Extract the standard form drawer (width 640, footer actions, destroyOnClose)
as FormDrawer; reuse existing usePolling hook. IntelligentEvals becomes the
first consumer — create drawer and both 5s pollings switched with zero
behaviour change. Includes ui-consistency spec, tickets, and ADR-0005.
 2026-08-05 15:34:31 +08:00
sinohqb
6cc2efafb6 feat(frontend): detail auto-refresh, session list, pinned AI-assistant tab
Some checks failed
CI / test (push) Failing after 40s
Details 
- Detail page silently polls every 5s while planning/executing/pending_approval
- Detail page shows per-session progress (persona/status/goal/turns), polled while executing
- /openclaw tab pinned by default like the dashboard (tabStore now .tsx)
 2026-08-05 14:01:26 +08:00
sinohqb
9cdbc41808 docs(intelligent-eval): commit v1.0 spec, tickets, and post-v1.0 improvements 
All 8 tickets' acceptance criteria checked off (incl. the real OpenClaw
E2E verified on t480). Trivial import-sort fix from ruff included.
 2026-08-05 14:01:24 +08:00
sinohqb
e1491dfc97 feat(intelligent-eval): OpenClaw planner/evaluator/analyst skills (ticket 08)
Some checks failed
CI / test (push) Failing after 43s
Details 
Three role-split skills synced via the existing deploy pipeline:
planner produces the coarse plan for approval, evaluator self-wakes by
time distribution to run virtual-user sessions, analyst aggregates session
evidence into the structured report.
 2026-08-05 04:07:33 +08:00
sinohqb
cbee749da2 feat(intelligent-eval): frontend list/detail/report pages (tickets 05-07)
Some checks failed
CI / test (push) Failing after 1m31s
Details 
List page with create drawer, detail panel with plan approval/rejection,
and structured report view with severity-sorted findings and session
chat drill-down. Views switch inside the /intelligent-evals keep-alive tab.
 2026-08-05 03:49:13 +08:00
sinohqb
da6ccc265d feat(frontend): restructure admin menu into two-level groups and add intelligent-eval entry
Some checks failed
CI / test (push) Failing after 50s
Details 
Reorganize the sidebar into top-level items plus four groups (静态评估 / 智能评估 /
配置中心 / 系统), add the /intelligent-evals route with a placeholder page, and tune
the expanded-submenu background so nested groups sit harmoniously on the dark sider.
 2026-08-05 03:27:48 +08:00
sinohqb
1317552701 feat(intelligent-eval): add backend for OpenClaw-driven intelligent evaluation (tickets 01-04) 
Introduce 智能评估 as an evaluation paradigm parallel to static evaluation,
driven by OpenClaw. The platform supplies storage, lifecycle, and reporting;
OpenClaw plans and executes.

- Data model: IntelligentEval + Session + Message tables (new, not reusing exploration)
- Lifecycle state machine: draft → planning → pending_approval → executing → completed/cancelled/failed
- Session API: create/message (channel-forwarded)/close with turn accounting
- Report API: pydantic-validated structured report, executing → completed, Markdown export (pure renderer)
- Alembic migration for the three tables; domain glossary added to CONTEXT.md
 2026-08-05 03:18:52 +08:00
sinohqb
f93d2f224f fix(frontend): auto-fetch report data when campaignId changes
Some checks failed
CI / test (push) Failing after 11s
Details 
useCampaignReport hook 缺少 useEffect 自动触发数据获取，导致打开报告抽屉时
显示空白。添加 useEffect 监听 campaignId 变化，自动调用 refetch。

- 新增 useEffect 在 campaignId 变化时自动获取数据
 2026-08-04 13:39:41 +08:00
sinohqb
160332665e refactor(exploration): absorb settlement.py into ExplorationSessionRepository
Some checks failed
CI / test (push) Failing after 12s
Details 
将 settlement.py 的 settle_campaign_sessions 函数吸收为
ExplorationSessionRepository.expire_running_sessions 方法。删除浅模块
settlement.py（30 行，接口宽如实现），会话生命周期操作集中在 repository。

- 新增 ExplorationSessionRepository.expire_running_sessions(campaign_id)
- 更新 campaigns.py 和 campaign_runner.py 两个调用点
- 删除 backend/agenteval/exploration/settlement.py
- 所有测试通过，行为不变
 2026-08-04 13:28:23 +08:00
sinohqb
0aa3ef81c5 refactor(repository): extract AsyncJobRepository base class for analysis/comparison 
提取 AsyncJobRepository 泛型基类，消除 CampaignAnalysisRepository 和
CampaignPeriodComparisonRepository 的重复代码。基类提供 get_by_campaign
和 mark_orphans_failed 通用逻辑，子类只需指定 _table 类型和实现 upsert。

- 新增 AsyncJobRepository[DB] 泛型基类
- mark_orphans_failed 接受 error_message 参数，子类传入特定错误信息
- 删除约 60 行重复代码（两个 __init__、两个 get_by_campaign、两个 mark_orphans_failed 实现）
- 所有测试通过，行为不变
 2026-08-04 13:25:42 +08:00
sinohqb
cbfdf86b36 refactor(frontend): extract useCampaignReport hook from Campaigns.tsx 
将活动报告抽屉的数据获取逻辑从 Campaigns.tsx 抽离到 useCampaignReport hook。
消除 5 个独立状态变量（report/reportRuns/reportTimeline/analysis/comparison）
和 fetchReport 函数的 25 行样板代码。hook 封装 5 个并行 API 调用和状态管理，
组件只负责渲染和交互。

- 新增 useCampaignReport(campaignId) hook
- 返回 {report, runs, timeline, analysis, comparison, loading, refetch, setAnalysis, setComparison}
- Campaigns.tsx 从 1122 行缩减状态管理复杂度
- TypeScript 类型检查通过，后端测试全绿
 2026-08-04 11:45:15 +08:00
sinohqb
c24998c762 refactor(metrics): extract dashboard aggregation to compute_dashboard 
仪表盘聚合逻辑从 stats.py router 下沉到 metrics.py 的 compute_dashboard
纯函数。_settled 重命名为 settled_runs 并公开，_ts 重命名为 _sortable_ts。
router 从 40 行聚合逻辑缩到 5 行，只负责数据获取和序列化。

- 新增 compute_dashboard(runs, scenario_names, target_names) -> dict
- 新增 settled_runs(runs) 公开接口（原 _settled）
- trend 端点同步迁移到 settled_runs
- 5 个新测试覆盖 dashboard 聚合逻辑
 2026-08-04 11:36:55 +08:00
sinohqb
42be31dd1f feat(report): add load_campaign_view as unified campaign read model 
活动级读模型单一出口：一次取齐报告 / 探索 / 分析 / 对比四大数据源。
markdown handler 从 30 行拼装逻辑缩到 3 行；分析执行器同步迁移。
comparison.py 内部的 8 次 load_campaign_report 调用暂不动（跨请求冗余，
缓存收益有限，改动风险高）。

- 新增 load_campaign_view(session, campaign) -> dict[str, Any]
- 返回 {report, exploration, analysis, comparison} 四键
- 迁移 markdown handler 和分析执行器两个调用点
- 4 个新测试覆盖 view 的组装逻辑
 2026-08-04 11:33:45 +08:00
sinohqb
2fddce8c92 refactor(case-verdict): extract build_case_evidence as single evidence-construction seam 
用例判定证据构建收敛到 case_verdict.py 的 build_case_evidence 纯函数，
report.py 和 runs.py 各删 ~15 行重复逻辑，换一行调用。locality 回归：
证据构建改一处，全局生效。

- 新增 build_case_evidence(turns, results) -> dict[str, CaseEvidence]
- report.py:76-83 证据构建替换为一行调用
- runs.py:175-201 证据构建替换为一行调用
- 5 个新测试覆盖 build_case_evidence（纯函数，无 DB 依赖）
 2026-08-04 11:23:56 +08:00
sinohqb
2b6cab6cb2 feat(comparison): unify read model and validation for period comparison 
周期对比读模型升位为单一出口（load_comparison_view），GET/POST/markdown
三处调用点统一走同一 view 投影，消除「取数三件套」重复。校验逻辑收敛到
validate_comparison_request，router 捕获映射 400，执行器捕获落 failed 行，
校验顺序权威不再漂移。

- 新增 load_comparison_view：无行返回 status=none + auto_baseline，有行
  返回完整 comparison dict（含 model_name 标签）+ metric_diff
- 新增 validate_comparison_request：活动终态 → 模型 → 基线 → 分析，违
  规抛 ComparisonError
- execute_campaign_comparison 内联校验替换为 validate_comparison_request
  调用，catch ComparisonError 落 failed 行
- router 三处迁移：GET /comparison、POST /comparison、markdown 导出
- 删除 build_comparison_payload（已吸收进 load_comparison_view）
- 8 个新测试覆盖读模型三态 + 校验五错
 2026-08-04 10:49:14 +08:00
sinohqb
f3a528611e refactor(tasks): route LLM background tasks through TaskRegistry 
架构保养第二轮候选 1：分析 / 周期对比 / judge 复核三条 LLM 任务链
收进各自的模块级 TaskRegistry（强引用防 GC、按 id 幂等、shutdown
统一收敛），删除 judge 的 _BACKGROUND_TASKS 私货，start_* 不再返回
无人消费的 Task。启动清理块补两笔 orphan 清扫：滞留的 generating
分析与对比行标记为 failed，与僵尸运行清扫同构。新增 7 个单测。
 2026-08-04 09:58:20 +08:00
sinohqb
df76edcf55 refactor(exploration): move ledger and state machine into domain modules
Some checks failed
CI / test (push) Failing after 33s
Details 
架构保养候选 3：探索生命周期的账本规则与状态机从 HTTP 层落入
exploration/lifecycle.py（open/conduct/close）与 patrol.py（巡检读模型），
违规改用类型化领域异常（NotFound/Guardrail/Channel），router 瘦回纯
HTTP 翻译（404/409/502 映射），领域层不再依赖 fastapi，可脱离
TestClient 直测（新增 12 个单元测试）。
 2026-08-04 03:46:51 +08:00
sinohqb
38849d46f1 refactor(repository): narrow atomic updates for patrol/cancel/scheduler writes 
全行 update 逼着调用方承诺「持有最新完整快照」，巡检水位、活动取消、
调度循环三方并发写入会互相覆盖（后写者抹掉先写者的 summary/status）。
CampaignRepository 新增三个窄口径原子操作：touch_patrol_watermark、
mark_cancelled、save_scheduler_state，各自只写自己的列；三处调用点迁移。
 2026-08-04 03:39:42 +08:00
sinohqb
f8d8450b1e refactor(report): unify campaign report loading behind one read model 
「活动报告取数三件套」此前在报告/markdown/分析/对比等 7 处手写重复，
唯一深化产物 build_campaign_report_dict 被锁在周期对比私有角落。
升位为 report.py 的 load_campaign_report(session, campaign) 单一出口
（探索线 summarize_campaign_exploration 同口径），并把 8 处
scenario_names 推导式收敛为 ScenarioRepository.name_map() 窄方法。
纯结构重排、零行为变更，572 项测试全绿。
 2026-08-04 03:30:29 +08:00
sinohqb
a665b496b0 chore(v0.9): wrap over-length lines and record spec rulings
Some checks failed
CI / test (push) Failing after 39s
Details 
Wrap the judge prompt and two docstrings past the 120-col convention;
record three implementation rulings in the v0.9 spec (exploration read
outlets, round-based sampling, findings carrying all ratings).
 2026-08-04 02:52:45 +08:00
sinohqb
b26c432f3a refactor(frontend): reuse ChatBubble for exploration drill-down 
The drawer hand-rolled chat bubbles duplicating ChatBubble's shape.
Add a mirrored layout prop (virtual user left, target right) and reuse
the shared component.
 2026-08-04 02:50:26 +08:00
sinohqb
ef4c094082 refactor(exploration): share one fetch+aggregate helper across outlets 
report, markdown and analysis each repeated the
summarize_exploration(repo.list_by_campaign(...)) shape; collapse it
into summarize_campaign_exploration so the aggregation has one home.
 2026-08-04 02:48:59 +08:00
sinohqb
936640fb36 fix(exploration): include all judge findings instead of poor-only 
Ticket 05 asks the judge review conclusions to flow into the report
verbatim; the aggregation silently dropped good/acceptable dimensions.
Collect every finding sorted poor-first and color drawer tags by rating.
 2026-08-04 02:45:41 +08:00
sinohqb
7f68afd765 docs(v0.9): mark ticket 06 done after Playwright browser verification 2026-08-04 02:27:37 +08:00
sinohqb
9abf572949 fix(exploration): flatten dict reply payloads instead of storing str(dict) 
E2E on t480 showed assistant bubbles rendering {'content': '...'} because
tutu replies carry msgBody as a parsed object and the router stored
str(reply.content). Coerce to the inner text before persisting.
 2026-08-04 02:26:24 +08:00
sinohqb
958cefc380 docs(v0.9): mark ticket 08 done after t480 cron registration and E2E 2026-08-04 02:09:36 +08:00
sinohqb
140bb1eb89 docs(v0.9): mark ticket 08 skill deliverables done pending t480 ops 2026-08-04 01:32:38 +08:00
sinohqb
fc167a8e4e feat(openclaw): patrol/exploration skill with loop-based skill sync 2026-08-04 01:32:13 +08:00
sinohqb
3c97e6e1ba docs(v0.9): mark ticket 07 finalize settlement done 2026-08-04 01:24:00 +08:00
sinohqb
62f7031b5b feat(exploration): expire dangling sessions on campaign finalize 2026-08-04 01:23:37 +08:00
sinohqb
b21dce73c5 docs(v0.9): mark ticket 06 code complete pending browser verification 2026-08-04 00:25:12 +08:00
sinohqb
a12eb09da3 feat(exploration): report drawer exploration findings section with session drill-down 2026-08-04 00:24:45 +08:00