Commit Graph

271 Commits

Author SHA1 Message Date
sinohqb
4c45dba95d test(intelligent-eval): add cron_pool boundary tests (#4)
Some checks failed
CI / test (push) Has been cancelled
T4 auto-scale boundary: busy/total == 0.8 (exact threshold) must not fire
T4 auto-scale oscillation: repeated flips across 0.8 must respect MIN/MAX bounds
T9 stuck-detection critical edge: strict '<' semantics at 10min threshold
T9 detect_stuck_crons ignores non-busy: idle/stuck crons excluded even if stale
2026-08-13 02:54:21 +08:00
sinohqb
3025dcd255 test(intelligent-eval): add worker task resilience tests (#3)
All checks were successful
CI / test (push) Successful in 4m21s
T1 concurrent assign: only one worker wins (sequential + dual-session)
T1 complete×stuck convergence: task reaches terminal state, never lost
T10 fault loop: heartbeat busy → cron stuck → handler fails task + creates retry

发现真 bug(assign_task 缺原子 CAS,详见 .scratch/v111-architecture-scan.md §6.1):
两个独立 session 并发 assign 同一 task 双认领。
以 xfail(strict=False) 留守卫,CI 不阻塞;修复后移除 xfail 即转绿。
该 bug 修复不在本 #3 范围,待开独立 issue。
2026-08-13 02:45:56 +08:00
sinohqb
9535a18a44 ci: verify pipeline end-to-end
All checks were successful
CI / test (push) Successful in 6m46s
2026-08-12 18:10:42 +08:00
sinohqb
7e3d8fd669 fix(ci-cd): cap job container memory/cpu to protect weak Gitea host (1.6G)
All checks were successful
CI / test (push) Successful in 4m15s
2026-08-12 17:53:07 +08:00
sinohqb
9eccee0a67 fix(ci-cd): set AGENTEVAL_PROD_DIR to actual production path
Some checks failed
CI / test (push) Failing after 11m21s
2026-08-12 15:21:23 +08:00
sinohqb
b451ac9625 feat(ci-cd): add self-hosted Gitea Actions runner + CI & production CD
Some checks are pending
CI / test (push) Waiting to run
- CI image (deploy/ci/Dockerfile) with warm deps, no GitHub/Docker Hub dependency
- act_runner deployment templates (compose/config/.env) for Gitea host 47.111.21.147
- Rewrite ci.yml: manual git clone (GITHUB_TOKEN), runs-on agenteval-ci, reuse ci-check.sh
- Add cd-production.yml: tag v*/dispatch trigger, test gate + version gate + ssh deploy
  via scripts/deploy-volcengine-102.sh (unchanged)
- t480 stays manual (internal network, unreachable from Gitea host)
2026-08-12 15:02:29 +08:00
sinohqb
fed52f3920 fix(deploy): resolve cron pool initialization issues
Some checks failed
CI / test (push) Failing after 32s
- Add Cron Pool Monitor entry to sidebar menu
- Install Docker CE CLI from Aliyun mirror for container exec
- Mount Docker socket for managing openclaw-eval container
- Fix OpenClaw CLI commands (cron add/rm/list instead of automations)
- Add gateway token auth for CLI access
2026-08-12 13:40:18 +08:00
sinohqb
4e9145db46 fix(lint): resolve ruff lint errors
Some checks failed
CI / test (push) Failing after 32s
- Remove unused imports (json, datetime, timedelta, Any, Optional, IntelligentEval)
- Remove unused variable (estimated_sessions)
- Remove whitespace from blank line
- Organize import blocks

All ruff checks passing.
2026-08-12 11:13:19 +08:00
sinohqb
617cc3909d chore: clean up release notes and update lockfile
- Remove old release-notes-v1.1.md (renamed to v1.1-legacy.md)
- Update package-lock.json after version bump to 1.1.0
2026-08-12 11:12:29 +08:00
sinohqb
6f2be0e68f docs(release): add v1.1.0 release notes
Document the intelligent evaluation cron pool architecture:
- Cron pool management (5-20 workers)
- Task queue with priority scheduling
- Worker skill with autonomous decision logic
- Config snapshot management and comparison
- Decision process tracking and visualization
- Monitoring and alerting system
- Fault tolerance and recovery mechanisms
- Frontend UI for config history, decision process, and cron pool monitoring

Rename legacy v1.1 release notes to v1.1-legacy.md.
2026-08-12 11:11:34 +08:00
sinohqb
15699e0dd0 chore(release): bump version to 1.1.0
Add intelligent evaluation cron pool architecture:
- Cron pool management (5-20 workers)
- Task queue with priority-based scheduling
- Worker skill with autonomous decision logic
- Config snapshot management and comparison
- Decision process tracking and visualization
- Monitoring and alerting system
- Fault tolerance and recovery mechanisms
- Frontend UI for config history, decision process, and cron pool monitoring

All 853 tests passing.
2026-08-12 11:09:38 +08:00
sinohqb
244feae505 feat(intelligent-eval): add cron pool monitoring UI (ticket 10)
- Add openclawCronPoolApi with all cron pool API calls
- Add CronPoolMonitor page with pool status, metrics, and alerts
- Implement real-time refresh (5 second polling)
- Implement manual scaling with target size input
- Add alert history table with resolve functionality
- Add route /cron-pool for cron pool monitoring page
- Pass TypeScript type checking

All 853 tests passing.
2026-08-12 11:05:51 +08:00
sinohqb
ee639afb0d feat(intelligent-eval): add decision process UI (ticket 09)
- Add list_decision_logs API endpoint
- Add DecisionProcess component with timeline, list, filter, and export
- Add decision log API calls to api.ts
- Add "决策过程" button in EvalDetail to access decision history
- Implement decision log export to JSON
- Pass TypeScript type checking

All 853 tests passing.
2026-08-12 10:59:51 +08:00
sinohqb
4b8afa892b feat(intelligent-eval): add config snapshots UI (ticket 08)
- Add ConfigSnapshots component with list, detail, compare, and export features
- Add config snapshot API calls to api.ts
- Add "配置历史" button in EvalDetail to access config history
- Implement snapshot comparison with diff view
- Implement snapshot export to JSON
- Pass TypeScript type checking

All 853 tests passing.
2026-08-12 10:50:11 +08:00
sinohqb
2e7d419f05 feat(intelligent-eval): implement monitoring and alerting (ticket 07)
- Add metrics.py with pool utilization, task backlog, stuck rate, avg processing time, eval completion rate
- Add alerts.py with alert rules (pool utilization > 90%, task backlog > 50, stuck rate > 10%)
- Implement alert history and webhook notifications
- Add metrics and alerts APIs
- Add database migration for alert history table
- Add 11 unit tests for metrics, 10 unit tests for alerts, 8 integration tests
- Update migration tests to include new alert history table

All 853 tests passing.
2026-08-12 10:41:15 +08:00
sinohqb
1d9228fd86 feat(intelligent-eval): implement fault tolerance and recovery (ticket 06)
- Add fault_tolerance.py with stuck cron detection and handling
- Implement state reconciliation (platform DB vs OpenClaw state)
- Implement platform restart recovery (requeue inactive tasks)
- Implement OpenClaw restart recovery (sync cron states)
- Add 6 unit tests and 5 integration tests

All 824 tests passing.
2026-08-12 10:30:05 +08:00
sinohqb
e6f98aaa6d feat(intelligent-eval): implement config snapshot management (ticket 05)
- Add config_snapshot.py with save/list/get/compare functions
- Auto-save snapshots on eval creation and plan submission
- Implement snapshot query APIs (list, get single)
- Implement snapshot comparison API (diff two snapshots)
- Add 8 unit tests and 7 integration tests

Snapshots track config changes over time (created/plan_submitted/config_updated).
All 813 tests passing.
2026-08-12 10:19:07 +08:00
sinohqb
fe3399297c feat(intelligent-eval): implement decision logic and e2e flow (ticket 04)
- Add decision.py with worker decision logic (execute_session/wait/start_analysis)
- Implement time slot parsing and current slot detection
- Implement session deficit calculation per time slot
- Implement high severity issue detection
- Implement eval completion detection
- Add 12 unit tests for decision logic
- Add 2 end-to-end tests for complete lifecycle

All 798 tests passing.
2026-08-12 10:10:42 +08:00
sinohqb
30b9cac224 feat(intelligent-eval): implement worker skill and APIs (ticket 03)
- Create agenteval-intelligent-worker SKILL.md with decision logic
- Implement heartbeat API (POST /api/openclaw/crons/{id}/heartbeat)
- Implement decision log API (POST /api/intelligent-evals/{id}/decision-logs)
- Skill includes idle/busy state management and cron state handling
- Deployment script already syncs skills automatically
- Add 6 integration tests

All 784 tests passing.
2026-08-12 10:01:56 +08:00
sinohqb
2ff023a65b feat(intelligent-eval): implement cron pool management (ticket 02)
- Add OpenClawClient wrapping CLI commands (create/delete/list crons)
- Implement pool initialization, scale up/down, auto-scaling logic
- Implement cron state sync and stuck cron detection
- Add pool status and manual scaling APIs
- Add 13 unit tests and 5 integration tests

Pool automatically scales between 5-20 crons based on load.
All 778 tests passing.
2026-08-12 09:47:04 +08:00
sinohqb
1aa453ef0a feat(intelligent-eval): add cron pool data model and task queue API
Implement Ticket 01 of intelligent eval cron pool architecture (ADR-0007):

- Add 4 new tables: task_queue, cron_pool, config_snapshots, decision_logs
- Implement task enqueueing logic with priority calculation
- Implement task assignment and completion APIs
- Add unit tests (9) and integration tests (7)
- Update CONTEXT.md with new vocabulary
- Add ADR-0007 documenting cron pool architecture decision

All 760 tests passing.
2026-08-12 02:13:21 +08:00
sinohqb
0326ec5d03 fix(deploy): serialize production frontend build
Some checks failed
CI / test (push) Failing after 8m44s
2026-08-11 14:57:22 +08:00
sinohqb
941df4c8c7 release: normalize project version to v1.0.0 2026-08-11 14:07:05 +08:00
sinohqb
1782b245bf refactor(architecture): deepen campaign runtime modules 2026-08-11 13:18:48 +08:00
sinohqb
10a089e740 docs(deploy): record production access topology
Some checks failed
CI / test (push) Failing after 1m31s
2026-08-09 03:49:06 +08:00
sinohqb
864ae2b7fe fix(review): address release correctness findings 2026-08-09 03:20:40 +08:00
sinohqb
4a0709b456 fix(frontend): synchronize production lockfile 2026-08-09 01:52:21 +08:00
sinohqb
855455c0b4 feat(deploy): add production release workflow 2026-08-07 15:53:36 +08:00
sinohqb
62485684ca fix(architecture): enforce lifecycle consistency 2026-08-07 10:59:27 +08:00
sinohqb
c896ab3f71 refactor(architecture): deepen evaluation lifecycle and read model 2026-08-07 03:11:37 +08:00
sinohqb
f26c34a340 docs(ui-consistency): archive completed refactor, record drawer evolution in ADR-0005
Some checks failed
CI / test (push) Failing after 34s
2026-08-06 02:01:36 +08:00
sinohqb
804880f1e3 fix(intelligent-eval): add top padding in drawer bodies
抽屉 body padding 为 0,顶栏按钮与提示贴住头部分割线;
详情/报告根容器补 16px 顶部内边距。
2026-08-06 01:52:46 +08:00
sinohqb
1fbd2bceed docs(ui-consistency): check off all ticket acceptance items after t480 walkthrough 2026-08-06 01:46:37 +08:00
sinohqb
9f25728c2f docs(agents): detail views may use right-side Drawer per ADR-0005 2026-08-06 01:42:13 +08:00
sinohqb
01f451c155 feat(intelligent-eval): align detail/report drawer layouts with other modules
详情抽屉改两列布局(左静态上下文/右动态流程),会话进度加 Progress;
报告抽屉顶部加统计概览行与维度得分柱状图,移除总结卡内重复的分数展示。
2026-08-06 01:03:28 +08:00
sinohqb
22cc437e2d feat(intelligent-eval): adjust drawer top-bar layout
详情页移除返回列表按钮(抽屉自带关闭),操作按钮右对齐;
报告页保留返回详情,导出 Markdown 按钮右对齐。
2026-08-06 00:37:02 +08:00
sinohqb
1d2c37277c feat(intelligent-eval): open detail/report in right-side drawer
列表常驻,详情与报告改为右侧抽屉弹出,减少页面层级嵌套。
2026-08-06 00:29:52 +08:00
sinohqb
935f0a1f57 refactor(ui): open intelligent eval via action-column buttons
Replace the name link and row-click with explicit detail/report
buttons, matching the static-eval table pages. Report button only
shows for completed evals; list pagination now follows the 20-row
threshold.
2026-08-05 21:00:19 +08:00
sinohqb
0832445862 docs(ui): record consistency baseline in AGENTS.md (ui-consistency ticket 09)
Add the ADR-0005 interaction standard to the frontend architecture
section and fix the tabStore extension. Backend pytest and frontend
tsc both green.
2026-08-05 20:29:07 +08:00
sinohqb
11b15908c7 refactor(ui): align Campaigns page with consistency baseline (ui-consistency ticket 08)
Create-activity form switched to FormDrawer (width 920, footer submit),
20-row pagination threshold on campaign and sub-run tables. Report
drawer stays a plain Drawer (read-only with export action). Timeline,
exploration, and analysis interactions unchanged.
2026-08-05 20:27:54 +08:00
sinohqb
5077f088b1 refactor(ui): align Reports page shell with consistency baseline (ui-consistency ticket 07)
PageWrapper inline fullHeight with an inner flex column to keep the
selection bar / content split, and 20-row pagination threshold on the
compare-case table. Report selection, charts, and compare logic
unchanged.
2026-08-05 20:26:43 +08:00
sinohqb
b9011aad80 refactor(ui): align Runs page shell with consistency baseline (ui-consistency ticket 06)
Replace custom header with PageWrapper inline fullHeight and add
Popconfirm to the stop action. The nav-only-tabs content-split hack is
preserved; WebSocket session, rerun, and list logic untouched.
2026-08-05 20:25:17 +08:00
sinohqb
1588972a8e refactor(ui): migrate Scenarios page to consistency baseline (ui-consistency ticket 05)
FormDrawer with footer submit at width 720 (keeps Monaco room),
submitting guard, 20-row conditional pagination, Empty locale,
unified delete Popconfirm wording. Preview/template Modals kept as
read-only surfaces.
2026-08-05 20:23:46 +08:00
sinohqb
f1316578fe refactor(ui): migrate ModelConfigs page to consistency baseline (ui-consistency ticket 04)
FormDrawer with footer submit and submitting guard, 20-row conditional
pagination, Empty locale, unified delete Popconfirm wording. Capability
and modality validation logic unchanged.
2026-08-05 20:22:14 +08:00
sinohqb
2e78c4befd refactor(ui): align Files page with consistency baseline (ui-consistency ticket 03)
Delete confirmations already existed (d7514f4), so no behavior fix was
needed. Switch empty states to antd Empty (table + category tree) and
apply the 20-row pagination standard.
2026-08-05 15:40:54 +08:00
sinohqb
15d0d127ae refactor(ui): migrate Targets page to consistency baseline (ui-consistency ticket 02)
FormDrawer with footer submit, submitting guard, 20-row conditional
pagination, Empty locale, unified delete Popconfirm wording.
2026-08-05 15:38:37 +08:00
sinohqb
d4c254e121 refactor(ui): baseline FormDrawer component and IntelligentEvals dogfood (ui-consistency ticket 01)
Extract the standard form drawer (width 640, footer actions, destroyOnClose)
as FormDrawer; reuse existing usePolling hook. IntelligentEvals becomes the
first consumer — create drawer and both 5s pollings switched with zero
behaviour change. Includes ui-consistency spec, tickets, and ADR-0005.
2026-08-05 15:34:31 +08:00
sinohqb
6cc2efafb6 feat(frontend): detail auto-refresh, session list, pinned AI-assistant tab
Some checks failed
CI / test (push) Failing after 40s
- Detail page silently polls every 5s while planning/executing/pending_approval
- Detail page shows per-session progress (persona/status/goal/turns), polled while executing
- /openclaw tab pinned by default like the dashboard (tabStore now .tsx)
2026-08-05 14:01:26 +08:00
sinohqb
9cdbc41808 docs(intelligent-eval): commit v1.0 spec, tickets, and post-v1.0 improvements
All 8 tickets' acceptance criteria checked off (incl. the real OpenClaw
E2E verified on t480). Trivial import-sort fix from ruff included.
2026-08-05 14:01:24 +08:00
sinohqb
e1491dfc97 feat(intelligent-eval): OpenClaw planner/evaluator/analyst skills (ticket 08)
Some checks failed
CI / test (push) Failing after 43s
Three role-split skills synced via the existing deploy pipeline:
planner produces the coarse plan for approval, evaluator self-wakes by
time distribution to run virtual-user sessions, analyst aggregates session
evidence into the structured report.
2026-08-05 04:07:33 +08:00