Commit Graph

185 Commits

Author SHA1 Message Date
sinohqb
1d2c37277c feat(intelligent-eval): open detail/report in right-side drawer
列表常驻,详情与报告改为右侧抽屉弹出,减少页面层级嵌套。
2026-08-06 00:29:52 +08:00
sinohqb
935f0a1f57 refactor(ui): open intelligent eval via action-column buttons
Replace the name link and row-click with explicit detail/report
buttons, matching the static-eval table pages. Report button only
shows for completed evals; list pagination now follows the 20-row
threshold.
2026-08-05 21:00:19 +08:00
sinohqb
0832445862 docs(ui): record consistency baseline in AGENTS.md (ui-consistency ticket 09)
Add the ADR-0005 interaction standard to the frontend architecture
section and fix the tabStore extension. Backend pytest and frontend
tsc both green.
2026-08-05 20:29:07 +08:00
sinohqb
11b15908c7 refactor(ui): align Campaigns page with consistency baseline (ui-consistency ticket 08)
Create-activity form switched to FormDrawer (width 920, footer submit),
20-row pagination threshold on campaign and sub-run tables. Report
drawer stays a plain Drawer (read-only with export action). Timeline,
exploration, and analysis interactions unchanged.
2026-08-05 20:27:54 +08:00
sinohqb
5077f088b1 refactor(ui): align Reports page shell with consistency baseline (ui-consistency ticket 07)
PageWrapper inline fullHeight with an inner flex column to keep the
selection bar / content split, and 20-row pagination threshold on the
compare-case table. Report selection, charts, and compare logic
unchanged.
2026-08-05 20:26:43 +08:00
sinohqb
b9011aad80 refactor(ui): align Runs page shell with consistency baseline (ui-consistency ticket 06)
Replace custom header with PageWrapper inline fullHeight and add
Popconfirm to the stop action. The nav-only-tabs content-split hack is
preserved; WebSocket session, rerun, and list logic untouched.
2026-08-05 20:25:17 +08:00
sinohqb
1588972a8e refactor(ui): migrate Scenarios page to consistency baseline (ui-consistency ticket 05)
FormDrawer with footer submit at width 720 (keeps Monaco room),
submitting guard, 20-row conditional pagination, Empty locale,
unified delete Popconfirm wording. Preview/template Modals kept as
read-only surfaces.
2026-08-05 20:23:46 +08:00
sinohqb
f1316578fe refactor(ui): migrate ModelConfigs page to consistency baseline (ui-consistency ticket 04)
FormDrawer with footer submit and submitting guard, 20-row conditional
pagination, Empty locale, unified delete Popconfirm wording. Capability
and modality validation logic unchanged.
2026-08-05 20:22:14 +08:00
sinohqb
2e78c4befd refactor(ui): align Files page with consistency baseline (ui-consistency ticket 03)
Delete confirmations already existed (d7514f4), so no behavior fix was
needed. Switch empty states to antd Empty (table + category tree) and
apply the 20-row pagination standard.
2026-08-05 15:40:54 +08:00
sinohqb
15d0d127ae refactor(ui): migrate Targets page to consistency baseline (ui-consistency ticket 02)
FormDrawer with footer submit, submitting guard, 20-row conditional
pagination, Empty locale, unified delete Popconfirm wording.
2026-08-05 15:38:37 +08:00
sinohqb
d4c254e121 refactor(ui): baseline FormDrawer component and IntelligentEvals dogfood (ui-consistency ticket 01)
Extract the standard form drawer (width 640, footer actions, destroyOnClose)
as FormDrawer; reuse existing usePolling hook. IntelligentEvals becomes the
first consumer — create drawer and both 5s pollings switched with zero
behaviour change. Includes ui-consistency spec, tickets, and ADR-0005.
2026-08-05 15:34:31 +08:00
sinohqb
6cc2efafb6 feat(frontend): detail auto-refresh, session list, pinned AI-assistant tab
Some checks failed
CI / test (push) Failing after 40s
- Detail page silently polls every 5s while planning/executing/pending_approval
- Detail page shows per-session progress (persona/status/goal/turns), polled while executing
- /openclaw tab pinned by default like the dashboard (tabStore now .tsx)
2026-08-05 14:01:26 +08:00
sinohqb
9cdbc41808 docs(intelligent-eval): commit v1.0 spec, tickets, and post-v1.0 improvements
All 8 tickets' acceptance criteria checked off (incl. the real OpenClaw
E2E verified on t480). Trivial import-sort fix from ruff included.
2026-08-05 14:01:24 +08:00
sinohqb
e1491dfc97 feat(intelligent-eval): OpenClaw planner/evaluator/analyst skills (ticket 08)
Some checks failed
CI / test (push) Failing after 43s
Three role-split skills synced via the existing deploy pipeline:
planner produces the coarse plan for approval, evaluator self-wakes by
time distribution to run virtual-user sessions, analyst aggregates session
evidence into the structured report.
2026-08-05 04:07:33 +08:00
sinohqb
cbee749da2 feat(intelligent-eval): frontend list/detail/report pages (tickets 05-07)
Some checks failed
CI / test (push) Failing after 1m31s
List page with create drawer, detail panel with plan approval/rejection,
and structured report view with severity-sorted findings and session
chat drill-down. Views switch inside the /intelligent-evals keep-alive tab.
2026-08-05 03:49:13 +08:00
sinohqb
da6ccc265d feat(frontend): restructure admin menu into two-level groups and add intelligent-eval entry
Some checks failed
CI / test (push) Failing after 50s
Reorganize the sidebar into top-level items plus four groups (静态评估 / 智能评估 /
配置中心 / 系统), add the /intelligent-evals route with a placeholder page, and tune
the expanded-submenu background so nested groups sit harmoniously on the dark sider.
2026-08-05 03:27:48 +08:00
sinohqb
1317552701 feat(intelligent-eval): add backend for OpenClaw-driven intelligent evaluation (tickets 01-04)
Introduce 智能评估 as an evaluation paradigm parallel to static evaluation,
driven by OpenClaw. The platform supplies storage, lifecycle, and reporting;
OpenClaw plans and executes.

- Data model: IntelligentEval + Session + Message tables (new, not reusing exploration)
- Lifecycle state machine: draft → planning → pending_approval → executing → completed/cancelled/failed
- Session API: create/message (channel-forwarded)/close with turn accounting
- Report API: pydantic-validated structured report, executing → completed, Markdown export (pure renderer)
- Alembic migration for the three tables; domain glossary added to CONTEXT.md
2026-08-05 03:18:52 +08:00
sinohqb
f93d2f224f fix(frontend): auto-fetch report data when campaignId changes
Some checks failed
CI / test (push) Failing after 11s
useCampaignReport hook 缺少 useEffect 自动触发数据获取,导致打开报告抽屉时
显示空白。添加 useEffect 监听 campaignId 变化,自动调用 refetch。

- 新增 useEffect 在 campaignId 变化时自动获取数据
2026-08-04 13:39:41 +08:00
sinohqb
160332665e refactor(exploration): absorb settlement.py into ExplorationSessionRepository
Some checks failed
CI / test (push) Failing after 12s
将 settlement.py 的 settle_campaign_sessions 函数吸收为
ExplorationSessionRepository.expire_running_sessions 方法。删除浅模块
settlement.py(30 行,接口宽如实现),会话生命周期操作集中在 repository。

- 新增 ExplorationSessionRepository.expire_running_sessions(campaign_id)
- 更新 campaigns.py 和 campaign_runner.py 两个调用点
- 删除 backend/agenteval/exploration/settlement.py
- 所有测试通过,行为不变
2026-08-04 13:28:23 +08:00
sinohqb
0aa3ef81c5 refactor(repository): extract AsyncJobRepository base class for analysis/comparison
提取 AsyncJobRepository 泛型基类,消除 CampaignAnalysisRepository 和
CampaignPeriodComparisonRepository 的重复代码。基类提供 get_by_campaign
和 mark_orphans_failed 通用逻辑,子类只需指定 _table 类型和实现 upsert。

- 新增 AsyncJobRepository[DB] 泛型基类
- mark_orphans_failed 接受 error_message 参数,子类传入特定错误信息
- 删除约 60 行重复代码(两个 __init__、两个 get_by_campaign、两个 mark_orphans_failed 实现)
- 所有测试通过,行为不变
2026-08-04 13:25:42 +08:00
sinohqb
cbfdf86b36 refactor(frontend): extract useCampaignReport hook from Campaigns.tsx
将活动报告抽屉的数据获取逻辑从 Campaigns.tsx 抽离到 useCampaignReport hook。
消除 5 个独立状态变量(report/reportRuns/reportTimeline/analysis/comparison)
和 fetchReport 函数的 25 行样板代码。hook 封装 5 个并行 API 调用和状态管理,
组件只负责渲染和交互。

- 新增 useCampaignReport(campaignId) hook
- 返回 {report, runs, timeline, analysis, comparison, loading, refetch, setAnalysis, setComparison}
- Campaigns.tsx 从 1122 行缩减状态管理复杂度
- TypeScript 类型检查通过,后端测试全绿
2026-08-04 11:45:15 +08:00
sinohqb
c24998c762 refactor(metrics): extract dashboard aggregation to compute_dashboard
仪表盘聚合逻辑从 stats.py router 下沉到 metrics.py 的 compute_dashboard
纯函数。_settled 重命名为 settled_runs 并公开,_ts 重命名为 _sortable_ts。
router 从 40 行聚合逻辑缩到 5 行,只负责数据获取和序列化。

- 新增 compute_dashboard(runs, scenario_names, target_names) -> dict
- 新增 settled_runs(runs) 公开接口(原 _settled)
- trend 端点同步迁移到 settled_runs
- 5 个新测试覆盖 dashboard 聚合逻辑
2026-08-04 11:36:55 +08:00
sinohqb
42be31dd1f feat(report): add load_campaign_view as unified campaign read model
活动级读模型单一出口:一次取齐报告 / 探索 / 分析 / 对比四大数据源。
markdown handler 从 30 行拼装逻辑缩到 3 行;分析执行器同步迁移。
comparison.py 内部的 8 次 load_campaign_report 调用暂不动(跨请求冗余,
缓存收益有限,改动风险高)。

- 新增 load_campaign_view(session, campaign) -> dict[str, Any]
- 返回 {report, exploration, analysis, comparison} 四键
- 迁移 markdown handler 和分析执行器两个调用点
- 4 个新测试覆盖 view 的组装逻辑
2026-08-04 11:33:45 +08:00
sinohqb
2fddce8c92 refactor(case-verdict): extract build_case_evidence as single evidence-construction seam
用例判定证据构建收敛到 case_verdict.py 的 build_case_evidence 纯函数,
report.py 和 runs.py 各删 ~15 行重复逻辑,换一行调用。locality 回归:
证据构建改一处,全局生效。

- 新增 build_case_evidence(turns, results) -> dict[str, CaseEvidence]
- report.py:76-83 证据构建替换为一行调用
- runs.py:175-201 证据构建替换为一行调用
- 5 个新测试覆盖 build_case_evidence(纯函数,无 DB 依赖)
2026-08-04 11:23:56 +08:00
sinohqb
2b6cab6cb2 feat(comparison): unify read model and validation for period comparison
周期对比读模型升位为单一出口(load_comparison_view),GET/POST/markdown
三处调用点统一走同一 view 投影,消除「取数三件套」重复。校验逻辑收敛到
validate_comparison_request,router 捕获映射 400,执行器捕获落 failed 行,
校验顺序权威不再漂移。

- 新增 load_comparison_view:无行返回 status=none + auto_baseline,有行
  返回完整 comparison dict(含 model_name 标签)+ metric_diff
- 新增 validate_comparison_request:活动终态 → 模型 → 基线 → 分析,违
  规抛 ComparisonError
- execute_campaign_comparison 内联校验替换为 validate_comparison_request
  调用,catch ComparisonError 落 failed 行
- router 三处迁移:GET /comparison、POST /comparison、markdown 导出
- 删除 build_comparison_payload(已吸收进 load_comparison_view)
- 8 个新测试覆盖读模型三态 + 校验五错
2026-08-04 10:49:14 +08:00
sinohqb
f3a528611e refactor(tasks): route LLM background tasks through TaskRegistry
架构保养第二轮候选 1:分析 / 周期对比 / judge 复核三条 LLM 任务链
收进各自的模块级 TaskRegistry(强引用防 GC、按 id 幂等、shutdown
统一收敛),删除 judge 的 _BACKGROUND_TASKS 私货,start_* 不再返回
无人消费的 Task。启动清理块补两笔 orphan 清扫:滞留的 generating
分析与对比行标记为 failed,与僵尸运行清扫同构。新增 7 个单测。
2026-08-04 09:58:20 +08:00
sinohqb
df76edcf55 refactor(exploration): move ledger and state machine into domain modules
Some checks failed
CI / test (push) Failing after 33s
架构保养候选 3:探索生命周期的账本规则与状态机从 HTTP 层落入
exploration/lifecycle.py(open/conduct/close)与 patrol.py(巡检读模型),
违规改用类型化领域异常(NotFound/Guardrail/Channel),router 瘦回纯
HTTP 翻译(404/409/502 映射),领域层不再依赖 fastapi,可脱离
TestClient 直测(新增 12 个单元测试)。
2026-08-04 03:46:51 +08:00
sinohqb
38849d46f1 refactor(repository): narrow atomic updates for patrol/cancel/scheduler writes
全行 update 逼着调用方承诺「持有最新完整快照」,巡检水位、活动取消、
调度循环三方并发写入会互相覆盖(后写者抹掉先写者的 summary/status)。
CampaignRepository 新增三个窄口径原子操作:touch_patrol_watermark、
mark_cancelled、save_scheduler_state,各自只写自己的列;三处调用点迁移。
2026-08-04 03:39:42 +08:00
sinohqb
f8d8450b1e refactor(report): unify campaign report loading behind one read model
「活动报告取数三件套」此前在报告/markdown/分析/对比等 7 处手写重复,
唯一深化产物 build_campaign_report_dict 被锁在周期对比私有角落。
升位为 report.py 的 load_campaign_report(session, campaign) 单一出口
(探索线 summarize_campaign_exploration 同口径),并把 8 处
scenario_names 推导式收敛为 ScenarioRepository.name_map() 窄方法。
纯结构重排、零行为变更,572 项测试全绿。
2026-08-04 03:30:29 +08:00
sinohqb
a665b496b0 chore(v0.9): wrap over-length lines and record spec rulings
Some checks failed
CI / test (push) Failing after 39s
Wrap the judge prompt and two docstrings past the 120-col convention;
record three implementation rulings in the v0.9 spec (exploration read
outlets, round-based sampling, findings carrying all ratings).
2026-08-04 02:52:45 +08:00
sinohqb
b26c432f3a refactor(frontend): reuse ChatBubble for exploration drill-down
The drawer hand-rolled chat bubbles duplicating ChatBubble's shape.
Add a mirrored layout prop (virtual user left, target right) and reuse
the shared component.
2026-08-04 02:50:26 +08:00
sinohqb
ef4c094082 refactor(exploration): share one fetch+aggregate helper across outlets
report, markdown and analysis each repeated the
summarize_exploration(repo.list_by_campaign(...)) shape; collapse it
into summarize_campaign_exploration so the aggregation has one home.
2026-08-04 02:48:59 +08:00
sinohqb
936640fb36 fix(exploration): include all judge findings instead of poor-only
Ticket 05 asks the judge review conclusions to flow into the report
verbatim; the aggregation silently dropped good/acceptable dimensions.
Collect every finding sorted poor-first and color drawer tags by rating.
2026-08-04 02:45:41 +08:00
sinohqb
7f68afd765 docs(v0.9): mark ticket 06 done after Playwright browser verification 2026-08-04 02:27:37 +08:00
sinohqb
9abf572949 fix(exploration): flatten dict reply payloads instead of storing str(dict)
E2E on t480 showed assistant bubbles rendering {'content': '...'} because
tutu replies carry msgBody as a parsed object and the router stored
str(reply.content). Coerce to the inner text before persisting.
2026-08-04 02:26:24 +08:00
sinohqb
958cefc380 docs(v0.9): mark ticket 08 done after t480 cron registration and E2E 2026-08-04 02:09:36 +08:00
sinohqb
140bb1eb89 docs(v0.9): mark ticket 08 skill deliverables done pending t480 ops 2026-08-04 01:32:38 +08:00
sinohqb
fc167a8e4e feat(openclaw): patrol/exploration skill with loop-based skill sync 2026-08-04 01:32:13 +08:00
sinohqb
3c97e6e1ba docs(v0.9): mark ticket 07 finalize settlement done 2026-08-04 01:24:00 +08:00
sinohqb
62f7031b5b feat(exploration): expire dangling sessions on campaign finalize 2026-08-04 01:23:37 +08:00
sinohqb
b21dce73c5 docs(v0.9): mark ticket 06 code complete pending browser verification 2026-08-04 00:25:12 +08:00
sinohqb
a12eb09da3 feat(exploration): report drawer exploration findings section with session drill-down 2026-08-04 00:24:45 +08:00
sinohqb
51980016a4 docs(v0.9): mark ticket 05 exploration findings flow done 2026-08-03 19:16:47 +08:00
sinohqb
2484c207af feat(exploration): findings flow into report, analysis and export
Exploration sessions aggregate into a single exploration summary
(session counts, goal-achievement rate, issue lists from experience
records, judge conclusions when reviewed) that feeds three exits:
the campaign report gains an exploration dimension, the v0.7 analysis
stage-two input gains the summary (stats only, never full dialogues),
and the Markdown export appends a findings appendix after analysis and
comparison. With no exploration data every output stays unchanged.
2026-08-03 19:16:33 +08:00
sinohqb
9cf64ab0e2 test(report): aggregation edge cases and output format validation
Adds unit tests for the campaign report beyond value correctness:
cancellation exclusion from denominators (ADR-0004), window clamping,
naive/aware timestamp equivalence, plus schema validation of the report
dict (key sets, types, rate bounds, contiguous buckets, capability sort,
ISO-UTC timestamps) and consistency with the single aggregate_runs seam.
2026-08-03 19:03:44 +08:00
sinohqb
8ed4221b0f docs(v0.9): mark ticket 04 judge sampling review done 2026-08-03 18:59:53 +08:00
sinohqb
2285a25009 feat(exploration): judge sampling review after session close
After an exploration session closes, the platform samples up to 3
conversation rounds and runs an independent judge-role review through
the v0.7 ChatClient seam, persisting quality-dimension conclusions
(attitude, professionalism, hallucination) into the session's
judge_review. The review runs as a background task: failures are
recorded without touching session state or the first-hand experience
record, and a missing model config skips silently.
2026-08-03 18:59:40 +08:00
sinohqb
3e485bfbe6 docs(v0.9): mark ticket 03 patrol API done
Some checks failed
CI / test (push) Failing after 43s
2026-08-03 18:12:32 +08:00
sinohqb
6340ec503c feat(exploration): stateless patrol API with watermark increments
Resident agents call GET /api/exploration/patrol once per cycle to see
every running production-line campaign that opted into exploration
(seed set present), the new results since the last watermark (reusing
campaign report aggregation), and the remaining exploration budget.
The watermark advances after each call so subsequent calls only report
increments; accelerated and terminal campaigns are excluded.
2026-08-03 18:12:11 +08:00
sinohqb
53afb9b5d1 feat(campaigns): exploration seed set and budget config per campaign
v0.9 ticket 02. Campaigns now carry an exploration seed set (seed
personas × seed goals — the comparability unit for exploratory
evaluation) and an optional budget override, stored as JSON columns
isomorphic to plan. Empty seeds normalize to null, marking the campaign
as opted out of exploration. resolve_budget merges per-field overrides
into platform defaults; enforcement stays server-side. The create form
gains seed lists and budget inputs (minutes → seconds), submitting null
when left empty.
2026-08-03 17:52:25 +08:00