Commit Graph

10 Commits

Author SHA1 Message Date
sinohqb
182b0e59cb refactor(intelligent-eval): extract scheduler runtime + internalize decision-log dedup
All checks were successful
CI / test (push) Successful in 3m9s
架构深化两则(架构审查候选①②):

① scheduler 抽取:web/app.py 约 400 行触发式执行编排(60s 扫描循环、
docker exec 触发、失败落账)沉入 intelligent_eval/scheduler.py,runtime
单例 start()/stop()/scan_once() 与 campaign_runtime 惯例一致;worker/planner
两处重复触发代码合并为一个触发原语;_supplement_decision_logs 归入
decision_logs.py。测试改为直接驱动 scan_once(interface 即测试面)。

② 决策日志去重内化:create_decision_log 的去重只服务 agent 上报路径;
新增 append_decision_log(平台落账纯追加)与 count_decisions(计数原语),
lifecycle/task_queue 全部平台落账切换,调用方不再需要塞 attempt 骗去重。

零行为变化:提示词、60s 节拍、编排顺序、闸门语义原样保留,866 tests passed。
2026-08-21 03:20:14 +08:00
sinohqb
eb4944a8bd feat(intelligent-eval): terminal-state discipline watchdogs (ADR-0011)
常见故障自愈有上限,超限收敛终态且可见:任务 attempts 上限、会话过期、
planning 双闸、executing 超窗兜底、触发失败计数判死、孤儿 agent 双管、
fire-and-forget 触发;open_session 预算硬闸门、settle 按终态区分、报告
scores 归一化;cron 池遗留面全删。
2026-08-20 14:34:17 +08:00
sinohqb
5de46d514a fix(intelligent-eval): trigger worker/planner with isolated session
All checks were successful
CI / test (push) Successful in 4m0s
t480 排查:评估 cd636d71 一直'等待 OpenClaw 创建会话',scan loop 每 60s 触发
worker,但 worker 被触发后 0 次工具调用、直接幻觉输出'评估 pending_approval'
(实际 executing),任务永不认领。

根因:openclaw agent --agent main 复用 main 持久 session,多次触发累积上下文
缓存(~12 万 token)后 LLM 不再执行 worker skill 的 API 步骤。
验证:独立 --session-id 触发 worker → 正常取任务、建会话、close。

修复:worker/planner 触发命令加 --session-id(每次唯一 agenteval-worker-*/planner-*),
避免 main session 污染;timeout 300→600(独立 session 首次加载 skill 更慢)。
测试:触发断言含 --session-id,901 passed
2026-08-18 17:22:30 +08:00
sinohqb
9d87ecf736 fix(intelligent-eval): trigger OpenClaw planner for planning evals
All checks were successful
CI / test (push) Successful in 3m59s
方案③只自动化了 executing(worker)→completed(analyst),缺少 planning 阶段的
planner 触发——新建/被打回的评估永远停在 planning(无任何机制唤醒
agenteval-intelligent-planner skill)。
- 新增 _has_planning_eval + _trigger_intelligent_planner:scan loop 每 60s
  对 planning 状态评估触发 planner skill(产出粗计划并 PUT /plan 提交),
  与 worker 触发同模式(docker exec openclaw agent -m 带'立即完成'语义)
- scan loop 在 worker 触发后追加 planner 触发
测试:+2(无 planning 不触发 / 有 planning 触发 planner skill),897 passed
2026-08-18 11:30:02 +08:00
sinohqb
b0969ae582 feat(intelligent-eval): backfill decision logs for completed evals
All checks were successful
CI / test (push) Successful in 4m2s
COMPLETED 状态的评估(历史/异常路径)可能完全没有决策日志,
导致旧报告决策过程为空。扩展 _supplement_decision_logs 支持
COMPLETED:按时段补 execute_session + 补 start_analysis(历史回填),
scan loop 每分钟自动回填,无需一次性脚本。幂等,只补缺失类型。
2026-08-17 13:06:27 +08:00
sinohqb
32f63e80ae style: ruff format (decision-log backfill)
Some checks failed
CI / test (push) Failing after 36s
2026-08-17 05:12:09 +08:00
sinohqb
25920280f6 feat(intelligent-eval): platform audit backfill for decision logs + disable legacy cron workers
Some checks failed
CI / test (push) Failing after 35s
- _supplement_decision_logs: executing evals missing a decision log get a
  platform-derived execute_session (deficit) or start_analysis (all sessions
  done) entry. Audit backfill only — records observable state, does not change
  agent execution. Called each scan tick after requeue+scan.
- t480 legacy cron workers (5) disabled: superseded by platform-triggered
  headless agent (plan C); they kept firing every minute and failing on
  Channel-required.
2026-08-17 05:11:18 +08:00
sinohqb
8e65e2e7b0 fix(intelligent-eval): worker trigger message must demand immediate execution
All checks were successful
CI / test (push) Successful in 3m47s
openclaw agent has no cron state; a bare 'agenteval-intelligent-worker'
message made the worker skill decide then 'wait for the next tick',
deadlocking (task assigned, session never created). The trigger message now
demands '立即完成当前任务,不要等待下一节拍' and, when all sessions are
done, delegates to agenteval-intelligent-analyst. Verified end-to-end on
t480: 1h-window eval went executing -> session (2 real turns) -> close ->
report -> completed, fully agent-driven, no external IM channel.
2026-08-17 02:57:08 +08:00
sinohqb
775b070bab feat(intelligent-eval): platform triggers OpenClaw agent as worker (avoid external IM channel)
All checks were successful
CI / test (push) Successful in 3m48s
OpenClaw cron requires a channel (announce->last fail-closed); webchat is a
Control-UI feature, not an addressable channel, and platform-side static
execution would degrade the intelligent eval into a static evaluation.

Solution (plan C): the platform keeps the scan loop and, when the queue has
a pending task, invokes the headless agent:
  docker exec openclaw-eval openclaw agent --agent main \
    -m agenteval-intelligent-worker --json
--deliver defaults to false, so no cron delivery channel is involved. The
worker skill runs unchanged under the OpenClaw agent (LLM decisions +
evaluator/analyst skills). Verified headless invocation returns ok.
2026-08-17 02:37:37 +08:00
sinohqb
5836b84681 fix(intelligent-eval): add lifespan scan-loop for worker task enqueue
All checks were successful
CI / test (push) Successful in 3m53s
scan_and_enqueue_tasks had no scheduler: the OpenClaw Worker wakes every
minute but the platform never enqueued executing evals, so the queue was
always empty. lifespan now starts an asyncio background task that scans
executing intelligent evals every 60s (aligned with the Worker wake),
cancelled cleanly on shutdown. Verified by a new startup test (879 total).
2026-08-17 02:08:40 +08:00