Commit Graph

127 Commits

Author SHA1 Message Date
sinohqb
a29f78ff4b feat(llm_score): 支持多维度独立评分
All checks were successful
CI / test (pull_request) Successful in 4m9s
扩展 llm_score 规则,支持通过 dimensions 参数配置多个评分维度,
每个维度独立评分 0-10 分。

- RuleResult 新增 details 字段存储结构化多维度分数
- 向后兼容:原有 criteria 单维度模式继续有效
- 多维度模式下各维度并行调用 LLM,返回平均分和明细
- 新增 4 项单元测试

Closes #19
2026-08-25 14:17:49 +08:00
sinohqb
bf1ec16ef6 perf(eval): case/rule/campaign 并发执行
- EvalEngine case 循环改 asyncio.gather + semaphore(默认 3 并发)
- 规则评估并行(默认 5 并发),LLM 评分耗时从串行求和降为最慢一条
- _resolve_model 改 async + 双重检查锁,保护共享模型缓存
- _case_errors / case_outcomes 并发写入加状态锁
- Campaign occurrence 派生并行(默认 2 并发),claim 拒绝时提前收敛
- 新增配置:max_concurrent_cases=3 / max_concurrent_rules=5 / max_concurrent_runs=2

SQLite StaticPool 单连接下 DB 写入仍天然串行,并行收益集中在
channel I/O 与 LLM 调用的等待重叠。
2026-08-24 23:18:11 +08:00
sinohqb
ca208232c7 perf(db): WAL 模式 + 性能索引 + N+1 查询消除
- SQLite 启用 WAL,允许读写并发
- 新增 6 个索引(eval_runs.status/campaign_id、eval_results.run_id、
  turns.run_id、intelligent_evals.status、task_queue.assigned_at)
- 幂等 Alembic 迁移(列/索引存在性检查)
- domain.py 计数改 func.count 聚合,get_attention_reason 单次加载 sessions
- scenario list_all 批量加载 bindings(1+N → 2 查询)
- mark_orphans_failed 批量加载 campaigns(N → 1 IN 查询)
2026-08-24 23:17:51 +08:00
sinohqb
3705945a7d test: 完整测试覆盖补全(+163 用例)
架构重构(候选 1-6):
- storage/repository.py 按域拆分为包(target/scenario/run/campaign/result)
- storage/db.py 按域拆分为包(eval/campaign/file/model_config/intelligent_eval)
- intelligent_eval/lifecycle.py 按状态机阶段拆分为包
- services/runs.py 编排逻辑下沉
- Campaigns.tsx 拆分为 campaigns/ 子组件

测试补全(候选 7):
前端(+125 用例,107→232):
- utils/ 纯函数:date/campaignTime/ruleLabels/fileTree/fileFormat/colors
- stores/tabStore 状态管理
- 核心组件:FormDrawer/PageWrapper/ChatBubble/GeneratedMessages/SectionHeader/StatCard/TurnList
- 业务组件:CaseBlock/CaseDetail/RuleOverview/WindowTimeline/RunList/TabBar/CampaignRunTimeline
- 文件管理:FileCategoryTree/FileTable
- hooks:sessionReducer/useFiles/useRunSession

后端(+38 用例,916→954):
- targets API CRUD + 404 路径
- WebSocket 连接管理器
- proxy 头部重写(CSP/X-Frame-Options)
- target 仓储 update 方法
- app 健康检查 + SPA 404
- scenarios 模板端点 + 404
- files API 边缘分支(404 场景 + 500 兜底)
- files service update_category
- 智能评估状态机迁移测试

门禁状态:
- 前端:tsc 干净 + 232 passed
- 后端:954 passed + ruff 全绿
2026-08-24 15:56:09 +08:00
sinohqb
7eae6de52d refactor(evaluation/storage): 结算统一与 repository 拆分(Phase 2 + 3)
合并两个不可分割的深化:

Phase 2 — 智能作业结算统一(ADR-0012)
- intelligence_jobs.execute(job_kind, campaign_id, ...) 作为结算的
  唯一实现:建行 → 认领 → 校验 → generating → 落账,一处编排、
  一处截断(500 字符)。两个 executor 退化为 ensure_queued /
  validate / work_fn 三个小 adapter。
- analysis.validate_analysis_request() 共享校验入口(活动终态 →
  模型),路由捕获映射 400、executor 捕获落 failed 行,与
  validate_comparison_request 先例同构。
- campaign_runner._auto_start_analysis 的跳过守卫收敛至
  auto_intelligence_eligible 单一判断点。
- comparison.py 删除零调用的 build_comparison_payload;
  load_comparison_view 投影归位至 campaign_read_model。
- 新增 characterization 测试(认领竞争、重复触发、截断、恢复上限)。

Phase 3 — storage/repository.py 拆分
- AsyncJobRepository 及两个子类迁至
  storage/async_job_repository.py(Phase 2 的 intelligence_jobs
  与 comparison 必须 import 自该路径,故与 Phase 2 同 commit)。
- ExplorationSession / ExplorationMessage 迁至
  storage/exploration_repository.py;repository.py 由 1180 行降至
  约 814 行,grep 确认无残留符号。
- exploration 子模块与路由 import 全部更新;测试 import 跟随。

刻意不做:CAS 共享原语、app.py 五 registry 关停顺序归一
(ADR-0006 精神,等真实需求出现再议)。
2026-08-24 05:50:27 +08:00
sinohqb
71543f042a refactor(intelligent-eval): 可见性接缝收敛(Phase 1)
将「已删即 404」语义收进 IntelligentEvalRepository 单一接缝,消除三处独立裁决;
任务监控开始隐藏已删评估的任务(本 Phase 唯一刻意行为变化)。

- repository.py 新增 visible() 谓词与 require_live_eval() 服务接缝;
  六处裸谓词统一走它,get()/get_including_deleted() 语义不变。
- decision_logs.py 删除本地 _require_eval,三处调用迁至 repository 接缝;
  count_decisions 由 len(.all()) 改为 func.count。
- task_queue.py list_tasks 与 stats 过滤已删评估的任务(行为变化)。
- web/routers/intelligent_evals.py: _require_eval_exists → _require_live_eval,
  把 LookupError 翻译为 404;expired 会话 Markdown 标注下沉至
  read_model.report_markdown_by_eval;配置快照 11 字段序列化收至
  config_snapshot.snapshot_to_dict 单一出口。
- AGENTS.md 登记可见性纪律(已知陷阱 #6)。
- 补 characterization 测试锁定四处契约;更新 task_queue 测试以使用
  真实 eval_id(可见性过滤后字面 eval_id 不再可见)。
2026-08-24 05:47:00 +08:00
sinohqb
913dc9ae86 perf: address remaining heuristic issues from code review
All checks were successful
CI / test (push) Successful in 3m18s
- decision_logs.py: add LIMIT 100 to dedup query to avoid loading all records
- ExecutionProcess.tsx: document N+1 API pattern and explain why acceptable
- scheduler.py: document why scan_once runs synchronously (thread pool would break fire-and-forget)

All 880 tests pass.
2026-08-24 02:01:04 +08:00
sinohqb
5a81c570c0 style: fix ruff whitespace warnings
All checks were successful
CI / test (push) Successful in 3m16s
2026-08-24 01:53:35 +08:00
sinohqb
09ff2ed123 refactor(intelligent-eval): reduce nesting complexity in supplement_decision_logs
Extract helper functions _supplement_executing and _supplement_completed
to flatten the nested conditional logic. This improves readability and
makes the code easier to test and maintain.

Addresses code review finding: supplement_decision_logs nested complexity
2026-08-24 01:53:15 +08:00
sinohqb
da7dd434dd perf(intelligent-eval): 修复 N+1 查询和参数名混淆
- expire_stale_running_sessions: 使用单次 JOIN 查询替代 N+1 查询
  将每个会话单独查询最后消息时间改为一次性获取所有 running 会话及其最后消息时间

- submit_report/evals_needing_analyst_nudge: 消除 session/sessions 参数名混淆
  将局部变量 sessions 重命名为 eval_sessions,避免与数据库会话参数 session 混淆

这些改进提升了查询性能并增强了代码可读性。
2026-08-24 01:52:05 +08:00
sinohqb
84627a3c6a refactor(intelligent-eval): 消除 lifecycle.py 和 scheduler.py 中的重复延迟导入
All checks were successful
CI / test (push) Successful in 3m16s
- 将延迟导入移至模块顶部,消除 Shotgun Surgery 气味
- lifecycle.py: 移除 41 行重复导入
- scheduler.py: 移除 14 行重复导入
- 修复测试:更新 monkeypatch 以补丁 scheduler 模块的引用而非原始模块
- 符合代码规范:避免函数内重复导入

Closes code-review finding: repeated deferred imports (Shotgun Surgery)
2026-08-24 01:47:47 +08:00
sinohqb
876d75f9ed refactor(intelligent-eval): 将 _get_raw 改为公开方法 get_including_deleted
All checks were successful
CI / test (push) Successful in 3m22s
- 消除 lifecycle.delete_eval 对私有方法的访问
- 明确该方法用于删除幂等性检查的用途
- 符合代码规范:避免跨模块访问私有接口
2026-08-24 01:37:58 +08:00
sinohqb
2d2c5a2904 feat(intelligent-eval): logical delete for terminal evals
All checks were successful
CI / test (push) Successful in 3m24s
Add `deleted` terminal status: completed/cancelled/failed → deleted via
DELETE /api/intelligent-evals/{id} (idempotent, 409 for non-terminal).
Deleted evals are hidden from list, detail, stats, and all sub-resource
endpoints (sessions/report/decision-logs/config-snapshots); child tables
are untouched (audit-safe). Frontend shows a Popconfirm-guarded delete
button for terminal evals only.
2026-08-21 14:34:04 +08:00
sinohqb
2a0bcdd185 refactor(intelligent-eval): drop ADR-0008 transitional wrappers, callers use domain
All checks were successful
CI / test (push) Successful in 3m8s
架构审查候选④:ADR-0008 收敛调度域时为保测试兼容留下的过渡 wrapper 使命结束。

删除 8 个浅封装:task_queue 的 _is_slot_due / _calculate_session_deficit
(零调用死函数)+ _calculate_priority / _get_attention_reason,decision 的
_parse_time_slot(零调用死函数)+ _get_current_slot / _count_sessions_in_slot /
_has_high_severity_issues。调用方直接使用 domain 模块。

5 个隔着 wrapper 测 domain 行为的测试迁到新文件
test_intelligent_eval_domain.py,直接锁定 domain,覆盖零丢失。
删除测试通过:复杂度直接消失,时段/欠账/优先级知识只剩 domain 一处。
870 tests passed,零行为变化。
2026-08-21 03:50:44 +08:00
sinohqb
58c2ad0227 refactor(intelligent-eval): unify watchdog failures behind fail_eval seam
All checks were successful
CI / test (push) Successful in 3m10s
架构审查候选③(状态机归一):_TRANSITIONS 成为评估状态机的唯一真相。

三个 watchdog(planning 双闸 / executing 兜底 / 触发失败闸门)原先直接
row.status = FAILED 绕过转换表、手工重复写字段。收编到新的公开接缝
fail_eval(session, eval_id, reason, decision_type, context):表校验 →
repo CAS 条件写(status/plan_feedback/updated_at/completed_at 一条 SQL)
→ append_decision_log 留痕。

CAS 冲突(如用户在扫描间隙抢先取消)跳过并留日志,不当故障。API 路径
本已在表内,不动。新增 4 个契约测试,870 tests passed,零行为变化。
2026-08-21 03:35:05 +08:00
sinohqb
182b0e59cb refactor(intelligent-eval): extract scheduler runtime + internalize decision-log dedup
All checks were successful
CI / test (push) Successful in 3m9s
架构深化两则(架构审查候选①②):

① scheduler 抽取:web/app.py 约 400 行触发式执行编排(60s 扫描循环、
docker exec 触发、失败落账)沉入 intelligent_eval/scheduler.py,runtime
单例 start()/stop()/scan_once() 与 campaign_runtime 惯例一致;worker/planner
两处重复触发代码合并为一个触发原语;_supplement_decision_logs 归入
decision_logs.py。测试改为直接驱动 scan_once(interface 即测试面)。

② 决策日志去重内化:create_decision_log 的去重只服务 agent 上报路径;
新增 append_decision_log(平台落账纯追加)与 count_decisions(计数原语),
lifecycle/task_queue 全部平台落账切换,调用方不再需要塞 attempt 骗去重。

零行为变化:提示词、60s 节拍、编排顺序、闸门语义原样保留,866 tests passed。
2026-08-21 03:20:14 +08:00
sinohqb
d0487b54b4 fix(intelligent-eval): respect time-window slots before nudging analyst
冒烟发现 analyst_nudge 不考虑时间窗口:首会话完成后 10 分钟即催促,
但后续时段尚未到期,导致报告提前收敛、漏掉计划内会话。催促闸门新增
"窗口未结束且会话数未达计划则跳过";worker 触发指令同步明确仅当所有
时段会话都达终态才转 analyst。
2026-08-20 16:27:05 +08:00
sinohqb
eb4944a8bd feat(intelligent-eval): terminal-state discipline watchdogs (ADR-0011)
常见故障自愈有上限,超限收敛终态且可见:任务 attempts 上限、会话过期、
planning 双闸、executing 超窗兜底、触发失败计数判死、孤儿 agent 双管、
fire-and-forget 触发;open_session 预算硬闸门、settle 按终态区分、报告
scores 归一化;cron 池遗留面全删。
2026-08-20 14:34:17 +08:00
sinohqb
5de46d514a fix(intelligent-eval): trigger worker/planner with isolated session
All checks were successful
CI / test (push) Successful in 4m0s
t480 排查:评估 cd636d71 一直'等待 OpenClaw 创建会话',scan loop 每 60s 触发
worker,但 worker 被触发后 0 次工具调用、直接幻觉输出'评估 pending_approval'
(实际 executing),任务永不认领。

根因:openclaw agent --agent main 复用 main 持久 session,多次触发累积上下文
缓存(~12 万 token)后 LLM 不再执行 worker skill 的 API 步骤。
验证:独立 --session-id 触发 worker → 正常取任务、建会话、close。

修复:worker/planner 触发命令加 --session-id(每次唯一 agenteval-worker-*/planner-*),
避免 main session 污染;timeout 300→600(独立 session 首次加载 skill 更慢)。
测试:触发断言含 --session-id,901 passed
2026-08-18 17:22:30 +08:00
sinohqb
71d38ebef6 fix(intelligent-eval): settle stale tasks of finished evals
All checks were successful
CI / test (push) Successful in 3m57s
任务队列出现'评估已 completed 却有待认领/执行中任务'的残留:评估离开
executing 后,其 pending/assigned 任务无人清理(requeue_stale_assigned_tasks
只处理 executing 评估的 assigned,评估结束被跳过)。

- 新增 task_queue.settle_tasks_for_finished_evals:评估非 executing 时,
  其 pending/assigned 任务回收为 completed;executing 评估的任务保留
- scan loop 每 60s 在 requeue+scan 后调用清理
测试:+1(已结束评估的 pending/assigned 回收、executing 保留、幂等),901 passed
2026-08-18 16:04:46 +08:00
sinohqb
56d709c44f fix(intelligent-eval): reject report until all sessions closed
All checks were successful
CI / test (push) Successful in 3m59s
t480 实测:评估 completed 但 completed_sessions=0(进度 0%)。根因
(diagnosing-bugs):submit_report 无条件 CAS executing→completed,不看会话
状态——worker 建会话未 close 时,analyst 仍可提交报告把评估标完成。

- lifecycle.submit_report 加校验:存在会话时须全部 close(completed),
  否则抛 IntelligentEvalTransitionError(409)——杜绝'评估完成但进度 0%'
- 无会话的评估仍可直接提交(analyst 正常路径不变)
回归测试(诊断闭环 Phase 1 反馈环先红后绿):
- +test_submit_report_rejects_when_sessions_not_closed(running 会话→409)
- +test_submit_report_allows_when_sessions_closed(全 close→200)
900 passed
2026-08-18 15:30:51 +08:00
sinohqb
8959e7ac42 fix(intelligent-eval): respect time-window slot distribution + flat scores
All checks were successful
CI / test (push) Successful in 3m57s
1) 时段分布失效:方案③'立即完成'触发让 worker 一次性创建所有计划会话,
   1h 窗口在几分钟内跑完,time_distribution 时段分布失效。
   - 触发指令明确'仅执行当前到期时段内欠账的会话,绝不创建未来时段会话'
   - worker skill 决策逻辑强化:严格按 current_offset 定位当前时段、只建该时段
     配额内的会话;完成当前时段后标记任务完成,由平台后续时段重新入队触发
2) 综合评分为空:analyst 产出扁平 scores {维度:分},前端 EvalReport 只认
   嵌套 {overall, dimensions} → overall 空。
   - EvalReport 兼容两种结构:维度取 dimensions(嵌套)或整个 scores(扁平),
     综合分取 overall 或维度平均
898 passed, tsc 0, vitest 16
2026-08-18 13:43:14 +08:00
sinohqb
60c54a67e4 fix(intelligent-eval): parse minute-level time slots (1h windows)
All checks were successful
CI / test (push) Successful in 3m54s
planner 对短窗口(1h)产出分钟级时段(如 0-20min/20-40min/40-60min),
但 parse_time_slot 只支持小时级(8-10h),分钟格式解析失败返回 None →
is_slot_due=False → 审批后评估永不入队、不触发 worker。
- parse_time_slot 支持 h/min 后缀,统一换算成小时(float)返回
- is_slot_due/get_current_slot/count_sessions_in_slot 用 timedelta(hours=float)
  兼容两种格式;decision._parse_time_slot 类型标注同步 float
测试:+1(分钟格式时段解析与到期判断),898 passed
2026-08-18 13:15:46 +08:00
sinohqb
9d87ecf736 fix(intelligent-eval): trigger OpenClaw planner for planning evals
All checks were successful
CI / test (push) Successful in 3m59s
方案③只自动化了 executing(worker)→completed(analyst),缺少 planning 阶段的
planner 触发——新建/被打回的评估永远停在 planning(无任何机制唤醒
agenteval-intelligent-planner skill)。
- 新增 _has_planning_eval + _trigger_intelligent_planner:scan loop 每 60s
  对 planning 状态评估触发 planner skill(产出粗计划并 PUT /plan 提交),
  与 worker 触发同模式(docker exec openclaw agent -m 带'立即完成'语义)
- scan loop 在 worker 触发后追加 planner 触发
测试:+2(无 planning 不触发 / 有 planning 触发 planner skill),897 passed
2026-08-18 11:30:02 +08:00
sinohqb
00ad929d68 feat(intelligent-eval): v4.2 UI — stat bar + single-line table + unified detail drawer
All checks were successful
CI / test (push) Successful in 3m58s
评估列表页(v4.2):
- 顶部状态统计条(stat-chip 点击筛选,后端列表接口新增 stats 状态分布 +
  status 筛选参数)
- 单行表格(名称/评测对象 ellipsis 省略)+ 会话进度迷你进度条 +
  最后一列醒目查看按钮 + 整行可点击
- 详情/报告合一单层抽屉(1000px,Tabs:概览/决策过程/配置历史/报告),
  审批/取消在抽屉头部;移除独立详情页 /intelligent-evals/detail 与
  intelligentEvalNav store

任务队列页:
- 与列表页统一 stat-chip 统计条 + 单行表格(评估状态小标、失败原因 tooltip)

清理:删除 IntelligentEvalDetail.tsx / intelligentEvalNav.ts;App.tsx 移除
detail 路由。任务队列保持独立菜单页。tsc 0 错误 vitest 19 passed 895 passed
2026-08-17 20:03:55 +08:00
sinohqb
2dd023fdd9 docs(intelligent-eval): align domain language with trigger-driven execution (ADR-0009)
All checks were successful
CI / test (push) Successful in 4m2s
方案③落地后,智能评估执行机制从'常驻 cron 每分钟自唤醒'改为'平台每 60s
扫描入队 + 按需触发无状态 headless agent'(触发式执行)。对齐领域语言:
- CONTEXT.md:Cron 池/工作单元(Worker)标 deprecated;新增触发式执行词条;
  修正时间窗口(cron 自唤醒→平台扫描时段到期)、任务队列(消费端)、决策日志
- ADR-0009 新增:记录触发式执行取代 cron 池的决策(原因:cron 需外部 channel,
  OpenClaw webchat 非 channel 账号无法 delivery);ADR-0007 标 superseded
- 代码标 deprecated:cron_pool / fault_tolerance / openclaw_cron_pool 路由 /
  CronPoolMonitor 页(导航入口已从 App.tsx 移除,监控由 TaskQueueMonitor 承担)
895 passed, vitest 19 passed
2026-08-17 16:16:12 +08:00
sinohqb
cc2ac8da89 feat(intelligent-eval): paginate the eval list
All checks were successful
CI / test (push) Successful in 4m5s
GET /api/intelligent-evals 支持 page/page_size(默认不传仍返回全部,向后兼容):
repository 加 count/list_page,lifecycle 加 list_evals_page,router 返回 total。
前端服务端分页:useIntelligentEvalRead 接 page/pageSize,list 存 total,
IntelligentEvals 表格 showSizeChanger + 页码切换重新加载;5s 轮询保持当前页。
测试:+3 后端分页 + hook 页码透传/总数断言,895 passed,vitest 19 passed
2026-08-17 14:09:17 +08:00
sinohqb
b4f9c887f4 feat(intelligent-eval): task queue monitor (方案③可视化)
All checks were successful
CI / test (push) Successful in 4m1s
方案③的定时触发(scan loop 每 60s 入队 + 触发 worker)此前只有 Worker
消费端 API,无可查看的列表。新增:
- GET /api/intelligent-evals/tasks:任务明细(含评估名/状态)+ 状态分布统计
  (注册在 /{eval_id} 之前避免被捕获为 eval_id="tasks")
- 前端 TaskQueueMonitor 组件 + 智能评估页任务队列入口:5s 轮询
  (usePolling),状态卡 + 状态筛选 + 明细表
测试:+3(列表/筛选/不被 {eval_id} 遮蔽),892 passed,tsc 通过
2026-08-17 13:57:00 +08:00
sinohqb
b0969ae582 feat(intelligent-eval): backfill decision logs for completed evals
All checks were successful
CI / test (push) Successful in 4m2s
COMPLETED 状态的评估(历史/异常路径)可能完全没有决策日志,
导致旧报告决策过程为空。扩展 _supplement_decision_logs 支持
COMPLETED:按时段补 execute_session + 补 start_analysis(历史回填),
scan loop 每分钟自动回填,无需一次性脚本。幂等,只补缺失类型。
2026-08-17 13:06:27 +08:00
sinohqb
32f63e80ae style: ruff format (decision-log backfill)
Some checks failed
CI / test (push) Failing after 36s
2026-08-17 05:12:09 +08:00
sinohqb
25920280f6 feat(intelligent-eval): platform audit backfill for decision logs + disable legacy cron workers
Some checks failed
CI / test (push) Failing after 35s
- _supplement_decision_logs: executing evals missing a decision log get a
  platform-derived execute_session (deficit) or start_analysis (all sessions
  done) entry. Audit backfill only — records observable state, does not change
  agent execution. Called each scan tick after requeue+scan.
- t480 legacy cron workers (5) disabled: superseded by platform-triggered
  headless agent (plan C); they kept firing every minute and failing on
  Channel-required.
2026-08-17 05:11:18 +08:00
sinohqb
71c7cd3d39 fix(intelligent-eval): requeue stale assigned tasks (worker crash recovery)
All checks were successful
CI / test (push) Successful in 3m45s
方案③ worker 由平台触发 openclaw agent(cron=manual-run,非真实 cron),
fault_tolerance 的 stuck 检测不适用——agent 中断/失败时任务永久卡 assigned,
scan 只查 pending 不再入队(死锁)。

requeue_stale_assigned_tasks:assigned 超过 10 分钟且评估仍 executing 的
任务重置为 pending(清空认领),平台 scan 循环随后重新触发 worker 重试。
接入 scan loop,每轮先 requeue 再 scan。
2026-08-17 03:30:42 +08:00
sinohqb
8e65e2e7b0 fix(intelligent-eval): worker trigger message must demand immediate execution
All checks were successful
CI / test (push) Successful in 3m47s
openclaw agent has no cron state; a bare 'agenteval-intelligent-worker'
message made the worker skill decide then 'wait for the next tick',
deadlocking (task assigned, session never created). The trigger message now
demands '立即完成当前任务,不要等待下一节拍' and, when all sessions are
done, delegates to agenteval-intelligent-analyst. Verified end-to-end on
t480: 1h-window eval went executing -> session (2 real turns) -> close ->
report -> completed, fully agent-driven, no external IM channel.
2026-08-17 02:57:08 +08:00
sinohqb
775b070bab feat(intelligent-eval): platform triggers OpenClaw agent as worker (avoid external IM channel)
All checks were successful
CI / test (push) Successful in 3m48s
OpenClaw cron requires a channel (announce->last fail-closed); webchat is a
Control-UI feature, not an addressable channel, and platform-side static
execution would degrade the intelligent eval into a static evaluation.

Solution (plan C): the platform keeps the scan loop and, when the queue has
a pending task, invokes the headless agent:
  docker exec openclaw-eval openclaw agent --agent main \
    -m agenteval-intelligent-worker --json
--deliver defaults to false, so no cron delivery channel is involved. The
worker skill runs unchanged under the OpenClaw agent (LLM decisions +
evaluator/analyst skills). Verified headless invocation returns ok.
2026-08-17 02:37:37 +08:00
sinohqb
5836b84681 fix(intelligent-eval): add lifespan scan-loop for worker task enqueue
All checks were successful
CI / test (push) Successful in 3m53s
scan_and_enqueue_tasks had no scheduler: the OpenClaw Worker wakes every
minute but the platform never enqueued executing evals, so the queue was
always empty. lifespan now starts an asyncio background task that scans
executing intelligent evals every 60s (aligned with the Worker wake),
cancelled cleanly on shutdown. Verified by a new startup test (879 total).
2026-08-17 02:08:40 +08:00
sinohqb
3376caca7b fix(decision-logs): remove unused context_json + json import (ruff F841/F401)
All checks were successful
CI / test (push) Successful in 3m57s
2026-08-14 15:23:36 +08:00
sinohqb
4bcab065f1 fix(decision-logs): dedupe on (eval_id, decision_type, context) (resolves T8)
Some checks failed
CI / test (push) Failing after 37s
create_decision_log now checks for an existing log with the same
(eval_id, decision_type, context) tuple before inserting. If found, it
returns the existing row's dict instead of appending a duplicate. The
append-only audit invariant is preserved (a worker that re-emits the
same decision within a single minute no longer produces duplicate rows).

Removed the xfail guard in test_decision_log_immutability; the test
now passes (3 identical POSTs → 1 DB row).
2026-08-14 15:14:05 +08:00
sinohqb
6d32653675 fix(intelligent-eval): link check_alerts to auto-scale_up (resolves T7)
All checks were successful
CI / test (push) Successful in 3m56s
AlertManager gains an optional openclaw_client. check_alerts records
each newly created alert and AlertManager.maybe_autoscale (called
from the async router for each alert) invokes cron_pool.scale_up(1).
scale_up itself caps at MAX_POOL_SIZE so repeated invocations are safe.

Removed the xfail guard in test_alert_autoscale_link; rewrote the test
to use task_backlog (duration_minutes=0) so a single check_alerts call
fires an alert and triggers auto-scale.
2026-08-14 15:08:24 +08:00
sinohqb
f85eca11ca fix(alerts): webhook retry + dedupe (resolves §6.2)
All checks were successful
CI / test (push) Successful in 3m57s
_send_webhook:
- Dedupe: skip if alert.webhook_sent is already True (guards against
  repeated check_alerts ticks re-sending the same alert).
- Retry: up to 3 attempts with exponential backoff (1s, 2s) before
  giving up. webhook_sent=True is persisted only on success.

Two xfail guards in test_openclaw_client_and_webhook now pass (876/4 xfail).
2026-08-14 14:54:57 +08:00
sinohqb
38e3817433 fix(intelligent-eval): atomic CAS in assign_task and complete_task (resolves §6.1)
Some checks failed
CI / test (push) Has been cancelled
Replace read-check-write in task_queue.assign_task with UPDATE...WHERE
status='pending' and decide on rowcount so two concurrent workers
cannot both claim the same task. Also harden complete_task with the
same CAS pattern (status='assigned') so a worker + stuck-handler
double-complete leaves the DB in one state.

The xfail guard in test_worker_task_resilience now passes (4/4).
2026-08-14 14:52:34 +08:00
sinohqb
37da87c3b0 refactor(intelligent-eval): openclaw_client token from settings (S5) + ADR-0008
Some checks failed
CI / test (push) Failing after 4m30s
P4 boundary (issue #10):
- S5: openclaw_client reads gateway_token and container_name from
  Settings (AGENTEVAL_OPENCLAW_GATEWAY_TOKEN / CONTAINER_NAME), with
  backwards-compatible defaults. Test injection still works via __init__.
- ADR-0008 documents the full deepening: S1 (domain convergence), S4
  (stuck-task settlement), S2 (router → service), S3 deferral rationale,
  S5 (token config), S6 (frontend polling deferred).

S6 (CronPoolMonitor unified polling) deferred to independent issue.
No behaviour change — 873 passed + 5 xfailed unchanged.
2026-08-13 14:20:49 +08:00
sinohqb
3852c6f87d refactor(intelligent-eval): router logic down to service layer (P3, S2)
Some checks failed
CI / test (push) Failing after 4m16s
P3 deepening (issue #9): remove direct ORM from router handlers.

- decision_logs.py (new): create_decision_log / list_decision_logs service
- cron_pool.heartbeat: encapsulate heartbeat cron lookup + update + commit
- cron_pool.scale_to: encapsulate scale direction decision (if/elif/else)
- task_queue.get_next_task_with_eval: encapsulate eval-loading + dict-building
- Router endpoints now delegate to services, only handling HTTP-level
  validation (status codes, 404 translation via LookupError).

No observable behaviour change — 873 passed + 5 xfailed unchanged.
T3 router ORM contract guards (5/5) continue to pass.
2026-08-13 13:57:34 +08:00
sinohqb
975ed7a6ff refactor(intelligent-eval): converge scheduling domain (S1) + stuck-task settlement (S4)
Some checks failed
CI / test (push) Failing after 4m21s
P1 deepening (issue #7):

S1:  is the single source of truth for time-slot parsing,
   slot-due checks, session deficit, priority, attention reason, and
   high-severity detection.  and  now delegate
   their internal helpers to  while keeping the same signatures
   (tests continue to pass via the thin wrappers).

S4:  encapsulates the stuck-cron
   task settlement (fail current task + enqueue retry).
   calls it instead of the previous runtime import of .

No observable behaviour change — 873 passed + 5 xfailed unchanged.
2026-08-13 10:04:24 +08:00
sinohqb
fed52f3920 fix(deploy): resolve cron pool initialization issues
Some checks failed
CI / test (push) Failing after 32s
- Add Cron Pool Monitor entry to sidebar menu
- Install Docker CE CLI from Aliyun mirror for container exec
- Mount Docker socket for managing openclaw-eval container
- Fix OpenClaw CLI commands (cron add/rm/list instead of automations)
- Add gateway token auth for CLI access
2026-08-12 13:40:18 +08:00
sinohqb
4e9145db46 fix(lint): resolve ruff lint errors
Some checks failed
CI / test (push) Failing after 32s
- Remove unused imports (json, datetime, timedelta, Any, Optional, IntelligentEval)
- Remove unused variable (estimated_sessions)
- Remove whitespace from blank line
- Organize import blocks

All ruff checks passing.
2026-08-12 11:13:19 +08:00
sinohqb
ee639afb0d feat(intelligent-eval): add decision process UI (ticket 09)
- Add list_decision_logs API endpoint
- Add DecisionProcess component with timeline, list, filter, and export
- Add decision log API calls to api.ts
- Add "决策过程" button in EvalDetail to access decision history
- Implement decision log export to JSON
- Pass TypeScript type checking

All 853 tests passing.
2026-08-12 10:59:51 +08:00
sinohqb
2e7d419f05 feat(intelligent-eval): implement monitoring and alerting (ticket 07)
- Add metrics.py with pool utilization, task backlog, stuck rate, avg processing time, eval completion rate
- Add alerts.py with alert rules (pool utilization > 90%, task backlog > 50, stuck rate > 10%)
- Implement alert history and webhook notifications
- Add metrics and alerts APIs
- Add database migration for alert history table
- Add 11 unit tests for metrics, 10 unit tests for alerts, 8 integration tests
- Update migration tests to include new alert history table

All 853 tests passing.
2026-08-12 10:41:15 +08:00
sinohqb
1d9228fd86 feat(intelligent-eval): implement fault tolerance and recovery (ticket 06)
- Add fault_tolerance.py with stuck cron detection and handling
- Implement state reconciliation (platform DB vs OpenClaw state)
- Implement platform restart recovery (requeue inactive tasks)
- Implement OpenClaw restart recovery (sync cron states)
- Add 6 unit tests and 5 integration tests

All 824 tests passing.
2026-08-12 10:30:05 +08:00
sinohqb
e6f98aaa6d feat(intelligent-eval): implement config snapshot management (ticket 05)
- Add config_snapshot.py with save/list/get/compare functions
- Auto-save snapshots on eval creation and plan submission
- Implement snapshot query APIs (list, get single)
- Implement snapshot comparison API (diff two snapshots)
- Add 8 unit tests and 7 integration tests

Snapshots track config changes over time (created/plan_submitted/config_updated).
All 813 tests passing.
2026-08-12 10:19:07 +08:00
sinohqb
fe3399297c feat(intelligent-eval): implement decision logic and e2e flow (ticket 04)
- Add decision.py with worker decision logic (execute_session/wait/start_analysis)
- Implement time slot parsing and current slot detection
- Implement session deficit calculation per time slot
- Implement high severity issue detection
- Implement eval completion detection
- Add 12 unit tests for decision logic
- Add 2 end-to-end tests for complete lifecycle

All 798 tests passing.
2026-08-12 10:10:42 +08:00