solahqb/AgentEvalTool
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases 1 Wiki Activity
272 Commits 13 Branches 4 Tags 6 MiB
56c56a7b4a
Commit Graph

16 Commits

Author SHA1 Message Date
sinohqb
7eae6de52d refactor(evaluation/storage): 结算统一与 repository 拆分（Phase 2 + 3） 
合并两个不可分割的深化：

Phase 2 — 智能作业结算统一（ADR-0012）
- intelligence_jobs.execute(job_kind, campaign_id, ...) 作为结算的
  唯一实现：建行 → 认领 → 校验 → generating → 落账，一处编排、
  一处截断（500 字符）。两个 executor 退化为 ensure_queued /
  validate / work_fn 三个小 adapter。
- analysis.validate_analysis_request() 共享校验入口（活动终态 →
  模型），路由捕获映射 400、executor 捕获落 failed 行，与
  validate_comparison_request 先例同构。
- campaign_runner._auto_start_analysis 的跳过守卫收敛至
  auto_intelligence_eligible 单一判断点。
- comparison.py 删除零调用的 build_comparison_payload；
  load_comparison_view 投影归位至 campaign_read_model。
- 新增 characterization 测试（认领竞争、重复触发、截断、恢复上限）。

Phase 3 — storage/repository.py 拆分
- AsyncJobRepository 及两个子类迁至
  storage/async_job_repository.py（Phase 2 的 intelligence_jobs
  与 comparison 必须 import 自该路径，故与 Phase 2 同 commit）。
- ExplorationSession / ExplorationMessage 迁至
  storage/exploration_repository.py；repository.py 由 1180 行降至
  约 814 行，grep 确认无残留符号。
- exploration 子模块与路由 import 全部更新；测试 import 跟随。

刻意不做：CAS 共享原语、app.py 五 registry 关停顺序归一
（ADR-0006 精神，等真实需求出现再议）。
 2026-08-24 05:50:27 +08:00
sinohqb
eb4944a8bd feat(intelligent-eval): terminal-state discipline watchdogs (ADR-0011) 
常见故障自愈有上限，超限收敛终态且可见：任务 attempts 上限、会话过期、
planning 双闸、executing 超窗兜底、触发失败计数判死、孤儿 agent 双管、
fire-and-forget 触发；open_session 预算硬闸门、settle 按终态区分、报告
scores 归一化；cron 池遗留面全删。
 2026-08-20 14:34:17 +08:00
sinohqb
0096c22e27 docs: v1.1.1 release notes + ADR-0010 + context/CLAUDE updates
All checks were successful
CI / test (push) Successful in 3m58s
Details 
更新本次智能评估全链路修复与 UI/UX 优化的必要文档：
- ADR-0010 新增：方案③触发采用独立 OpenClaw session（main 持久 session 上下文
  缓存污染导致 worker 幻觉不执行）+ 时段分布约束
- CONTEXT.md：触发式执行词条补充时段约束与独立会话语义
- release-notes-v1.1.1.md 新增：智能评估全链路稳定化、任务队列监控、UI/UX 一致性
- docs/README.md：补 v1.1.0/v1.1.1 发布说明索引，版本升 v1.2
- CLAUDE.md：补智能评估执行机制（方案③）章节（scan loop 职责 + 独立 session/
  时段分布/状态一致性关键约束）
 2026-08-18 19:01:14 +08:00
sinohqb
2dd023fdd9 docs(intelligent-eval): align domain language with trigger-driven execution (ADR-0009)
All checks were successful
CI / test (push) Successful in 4m2s
Details 
方案③落地后，智能评估执行机制从'常驻 cron 每分钟自唤醒'改为'平台每 60s
扫描入队 + 按需触发无状态 headless agent'（触发式执行）。对齐领域语言：
- CONTEXT.md：Cron 池/工作单元（Worker）标 deprecated；新增触发式执行词条；
  修正时间窗口（cron 自唤醒→平台扫描时段到期）、任务队列（消费端）、决策日志
- ADR-0009 新增：记录触发式执行取代 cron 池的决策（原因：cron 需外部 channel，
  OpenClaw webchat 非 channel 账号无法 delivery）；ADR-0007 标 superseded
- 代码标 deprecated：cron_pool / fault_tolerance / openclaw_cron_pool 路由 /
  CronPoolMonitor 页（导航入口已从 App.tsx 移除，监控由 TaskQueueMonitor 承担）
895 passed, vitest 19 passed
 2026-08-17 16:16:12 +08:00
sinohqb
37da87c3b0 refactor(intelligent-eval): openclaw_client token from settings (S5) + ADR-0008
Some checks failed
CI / test (push) Failing after 4m30s
Details 
P4 boundary (issue #10):
- S5: openclaw_client reads gateway_token and container_name from
  Settings (AGENTEVAL_OPENCLAW_GATEWAY_TOKEN / CONTAINER_NAME), with
  backwards-compatible defaults. Test injection still works via __init__.
- ADR-0008 documents the full deepening: S1 (domain convergence), S4
  (stuck-task settlement), S2 (router → service), S3 deferral rationale,
  S5 (token config), S6 (frontend polling deferred).

S6 (CronPoolMonitor unified polling) deferred to independent issue.
No behaviour change — 873 passed + 5 xfailed unchanged.
 2026-08-13 14:20:49 +08:00
sinohqb
1aa453ef0a feat(intelligent-eval): add cron pool data model and task queue API 
Implement Ticket 01 of intelligent eval cron pool architecture (ADR-0007):

- Add 4 new tables: task_queue, cron_pool, config_snapshots, decision_logs
- Implement task enqueueing logic with priority calculation
- Implement task assignment and completion APIs
- Add unit tests (9) and integration tests (7)
- Update CONTEXT.md with new vocabulary
- Add ADR-0007 documenting cron pool architecture decision

All 760 tests passing.
 2026-08-12 02:13:21 +08:00
sinohqb
1782b245bf refactor(architecture): deepen campaign runtime modules 2026-08-11 13:18:48 +08:00
sinohqb
62485684ca fix(architecture): enforce lifecycle consistency 2026-08-07 10:59:27 +08:00
sinohqb
c896ab3f71 refactor(architecture): deepen evaluation lifecycle and read model 2026-08-07 03:11:37 +08:00
sinohqb
f26c34a340 docs(ui-consistency): archive completed refactor, record drawer evolution in ADR-0005
Some checks failed
CI / test (push) Failing after 34s
Details
2026-08-06 02:01:36 +08:00
sinohqb
d4c254e121 refactor(ui): baseline FormDrawer component and IntelligentEvals dogfood (ui-consistency ticket 01) 
Extract the standard form drawer (width 640, footer actions, destroyOnClose)
as FormDrawer; reuse existing usePolling hook. IntelligentEvals becomes the
first consumer — create drawer and both 5s pollings switched with zero
behaviour change. Includes ui-consistency spec, tickets, and ADR-0005.
 2026-08-05 15:34:31 +08:00
sinohqb
c35159ffcb docs(v2): record exploratory-evaluation consensus in CONTEXT.md and ADR-0003 
Grilling 沉淀：OpenClaw 定位从"重规划"转为"虚拟用户"双轨结构
（周期活动内嵌探索先行 + 探索活动后续里程碑），新增六个领域词条。
 2026-08-03 16:40:28 +08:00
sinohqb
782916a283 refactor(metrics): type Run summary and converge cross-run aggregation 
Give EvalRun.summary a typed RunSummary value (unified RunError, lenient
legacy parsing) so readers stop reaching into a schemaless dict, and route
every cross-run rollup — dashboard, scenario ranking, trend, campaign
report — through one aggregate_runs seam. Fixes the divergence where
stats averaged pass_rate over completed-only runs while the campaign
report counted faults as 0.0. Cross-run rule (ADR-0004): genuine faults
count 0.0, user-cancelled runs are excluded from both denominators.
 2026-07-31 01:57:56 +08:00
sinohqb
fe2a3f7579 docs(domain): add Campaign glossary terms and phasing ADR
Some checks failed
CI / test (push) Failing after 40s
Details 
领域建模拷问产出：CONTEXT.md 新增「周期评估」章节（评估活动 / 服务周期
窗口 / 活动计划）；ADR-0003 记录活动分期（v1 静态地基、v2 自适应回路）
与「OpenClaw 进调度热回路」的取舍。
 2026-07-30 11:12:57 +08:00
sinohqb
770d260750 feat(report): compare requires same scenario version (ticket 05)
Some checks failed
CI / test (push) Failing after 39s
Details 
对比报告可比性收紧为同场景同考纲版本（ADR-0001）：跨版本 API 返回 400
（detail 含双方版本号），报告生成层抛 ValueError；前端对比候选按
同场景 + 同版本过滤，A 变更后自动清空不可比的 B。文档"尚未实现"标注移除。
 2026-07-29 11:21:52 +08:00
sinohqb
e33922c3fe docs(domain): add domain glossary CONTEXT.md and first ADRs
Some checks failed
CI / test (push) Failing after 51s
Details 
拷问会话产出：14 条核心术语定义（评测对象/场景/用例/轮次/通过率/模型能力·用途等），
以及两项决策记录——场景版本化的可比性语义（ADR-0001）、通过率含执行失败的口径（ADR-0002）。
 2026-07-28 21:46:56 +08:00