Commit Graph

90 Commits

Author SHA1 Message Date
sinohqb
c956da7686 feat(v1.3.1): Phase 2 让成本/放弃率/Go-No-Go 基础设施真正生效
All checks were successful
CI / test (pull_request) Successful in 4m3s
- token 用量接入:ModelGateway 经 adapter.parse_usage 累计评测侧 LLM 调用的
  token 消耗,引擎写入 run.summary.eval_token_usage,报告透出
- 放弃率落地:CaseOutcome 新增 abandoned 标记(对话中途发送/接收失败),
  build_run_summary 统计 abandoned_cases / abandonment_rate
- Go/No-Go 可配置:Scenario 新增 acceptance_criteria 字段(DB 列 + 幂等迁移),
  报告按场景标准出 verdict,缺省回退全局默认;标准变更不触发考纲升版
2026-08-25 18:38:18 +08:00
sinohqb
c1a3cdbaa9 fix(rules): 修复代码审查发现的三处规则层缺陷
All checks were successful
CI / test (pull_request) Successful in 4m5s
- response_time: 仅配置 max_ms 时恢复 v0.3 评分语义(最后一轮评分 + 超限线性惩罚),扩展指标共存时才用均值评分
- safety: 移除 moderation API 的黑名单命中跳过守卫,两层安全检查独立执行、发现均上报
- llm_score: 多维度评分添加 Semaphore 并发上限(5),防止维度数多时无限扇出模型请求
2026-08-25 18:13:11 +08:00
e7e3716325 Merge pull request 'feat: 用户体验指标(放弃率+流畅度评估)' (#32) from feat/user-experience-metrics into main
All checks were successful
CI / test (push) Successful in 4m9s
2026-08-25 08:40:45 +00:00
sinohqb
21cc6ed407 feat: 用户体验指标(放弃率+流畅度评估)
All checks were successful
CI / test (pull_request) Successful in 4m11s
新增用户体验跟踪能力。

- RunSummary 新增 abandoned_cases 和 abandonment_rate 字段
- 新增 fluency 规则:LLM 评估对话流畅度和自然性(0-10 分)
- 流畅度评估考虑:自然性、重复性、连贯性、响应质量
- 新增 10 项单元测试(676 tests passed)

注:放弃率的实际计算逻辑需要在引擎中集成,本 PR 提供数据模型和规则基础设施。

Closes #26
2026-08-25 16:36:28 +08:00
sinohqb
56c56a7b4a feat: 成本效率跟踪基础设施
All checks were successful
CI / test (pull_request) Successful in 4m9s
新增成本跟踪模块,为对话级和任务级成本计算提供基础。

- Turn 模型新增 prompt_tokens、completion_tokens、total_tokens 字段
- OpenAI 协议适配器新增 parse_usage() 提取 token 使用量
- 新增 evaluation/cost_tracking.py 模块:
  - ModelPricing:模型定价配置
  - TokenUsage:token 使用量聚合
  - CostBreakdown:成本明细
  - calculate_cost():根据 token 使用量和定价计算费用
  - calculate_turn_cost()、calculate_case_cost()、calculate_run_cost()
- 内置常见模型定价(GPT-4o、GPT-4o-mini、Claude 等)
- 新增 11 项单元测试(677 tests passed)

注:引擎集成(实际捕获 API 调用的 token 使用量)留待后续实现。

Closes #25
2026-08-25 16:07:08 +08:00
2c79abf1de Merge pull request 'feat(safety): 扩展规则覆盖幻觉、越权和合规检查' (#30) from feat/safety-extended into main
All checks were successful
CI / test (push) Successful in 4m7s
2026-08-25 08:02:05 +00:00
sinohqb
192fe0fc5f feat(safety): 扩展规则覆盖幻觉、越权和合规检查
All checks were successful
CI / test (pull_request) Successful in 4m2s
扩展 safety 规则,新增三个安全检查维度。

- 幻觉检测(check_hallucination):LLM 判断回答是否编造事实
- 越权操作检测(unauthorized_actions):模式匹配检测越权行为
- 合规性检查(required_disclaimers):检查必需声明是否存在
- 保持原有 blacklist + moderation API 向后兼容
- RuleResult.details 包含所有问题列表
- 新增 10 项单元测试(656 tests passed)

Closes #22
2026-08-25 15:30:00 +08:00
sinohqb
737c9ab80e feat(response_time): 支持平均延迟和吞吐量指标
All checks were successful
CI / test (pull_request) Successful in 4m6s
扩展 response_time 规则,支持多维度延迟和吞吐量测量。

- 新增 avg_latency_max_ms:多轮对话平均延迟阈值
- 新增 throughput_min:最低吞吐量(turns/min)
- 保持 max_ms 向后兼容(单轮延迟阈值)
- RuleResult.details 包含延迟统计(avg/min/max/latencies)
- 新增 10 项单元测试(656 tests passed)

注:首字延迟(first token latency)需要流式数据支持,当前 Turn 模型未提供,留待后续增强。

Closes #21
2026-08-25 15:03:54 +08:00
2f08e7bf06 Merge pull request 'feat(report): 上线验收报告 go/no-go 结论' (#28) from feat/go-no-go-verdict into main
All checks were successful
CI / test (push) Successful in 4m6s
2026-08-25 06:31:42 +00:00
sinohqb
2314ebe3bc feat(report): 上线验收报告 go/no-go 结论
All checks were successful
CI / test (pull_request) Successful in 4m7s
新增纯函数模块 evaluation/go_no_go.py,基于 RunSummary 指标自动生成
go/no-go/conditional 三级结论。

- AcceptanceCriteria 模型:judged_pass_rate_min、pass_rate_min、avg_latency_max_ms
- GoNoGoVerdict 模型:decision、summary、criteria_results
- evaluate_go_no_go() 纯函数:接收 summary dict + criteria -> 返回 verdict
- 集成到 generate_report(),报告 dict 自动附带 go_no_go 字段
- 三级结论:go(全部达标)、no_go(核心未达标)、conditional(有风险)
- 新增 9 项单元测试(642 tests passed)

Closes #20
2026-08-25 14:28:55 +08:00
sinohqb
a29f78ff4b feat(llm_score): 支持多维度独立评分
All checks were successful
CI / test (pull_request) Successful in 4m9s
扩展 llm_score 规则,支持通过 dimensions 参数配置多个评分维度,
每个维度独立评分 0-10 分。

- RuleResult 新增 details 字段存储结构化多维度分数
- 向后兼容:原有 criteria 单维度模式继续有效
- 多维度模式下各维度并行调用 LLM,返回平均分和明细
- 新增 4 项单元测试

Closes #19
2026-08-25 14:17:49 +08:00
sinohqb
90ebbd5ff4 test(scheduler): 补全触发失败落账与错误路径测试(+9 用例)
覆盖 scheduler.py 中等难度边缘分支:
- _trigger_openclaw_agent 非零退出码 → on_failure 回调
- _trigger_openclaw_agent subprocess 异常 → on_failure 回调
- _record_worker_trigger_failure 成功/跳过/DB 异常兜底
- _record_planner_trigger_failure 成功/跳过/DB 异常兜底
- _fire_and_forget cancelled task 不记录异常

scheduler.py 覆盖率 75% → 81%
后端总计 963 passed
2026-08-24 17:18:52 +08:00
sinohqb
3705945a7d test: 完整测试覆盖补全(+163 用例)
架构重构(候选 1-6):
- storage/repository.py 按域拆分为包(target/scenario/run/campaign/result)
- storage/db.py 按域拆分为包(eval/campaign/file/model_config/intelligent_eval)
- intelligent_eval/lifecycle.py 按状态机阶段拆分为包
- services/runs.py 编排逻辑下沉
- Campaigns.tsx 拆分为 campaigns/ 子组件

测试补全(候选 7):
前端(+125 用例,107→232):
- utils/ 纯函数:date/campaignTime/ruleLabels/fileTree/fileFormat/colors
- stores/tabStore 状态管理
- 核心组件:FormDrawer/PageWrapper/ChatBubble/GeneratedMessages/SectionHeader/StatCard/TurnList
- 业务组件:CaseBlock/CaseDetail/RuleOverview/WindowTimeline/RunList/TabBar/CampaignRunTimeline
- 文件管理:FileCategoryTree/FileTable
- hooks:sessionReducer/useFiles/useRunSession

后端(+38 用例,916→954):
- targets API CRUD + 404 路径
- WebSocket 连接管理器
- proxy 头部重写(CSP/X-Frame-Options)
- target 仓储 update 方法
- app 健康检查 + SPA 404
- scenarios 模板端点 + 404
- files API 边缘分支(404 场景 + 500 兜底)
- files service update_category
- 智能评估状态机迁移测试

门禁状态:
- 前端:tsc 干净 + 232 passed
- 后端:954 passed + ruff 全绿
2026-08-24 15:56:09 +08:00
sinohqb
7eae6de52d refactor(evaluation/storage): 结算统一与 repository 拆分(Phase 2 + 3)
合并两个不可分割的深化:

Phase 2 — 智能作业结算统一(ADR-0012)
- intelligence_jobs.execute(job_kind, campaign_id, ...) 作为结算的
  唯一实现:建行 → 认领 → 校验 → generating → 落账,一处编排、
  一处截断(500 字符)。两个 executor 退化为 ensure_queued /
  validate / work_fn 三个小 adapter。
- analysis.validate_analysis_request() 共享校验入口(活动终态 →
  模型),路由捕获映射 400、executor 捕获落 failed 行,与
  validate_comparison_request 先例同构。
- campaign_runner._auto_start_analysis 的跳过守卫收敛至
  auto_intelligence_eligible 单一判断点。
- comparison.py 删除零调用的 build_comparison_payload;
  load_comparison_view 投影归位至 campaign_read_model。
- 新增 characterization 测试(认领竞争、重复触发、截断、恢复上限)。

Phase 3 — storage/repository.py 拆分
- AsyncJobRepository 及两个子类迁至
  storage/async_job_repository.py(Phase 2 的 intelligence_jobs
  与 comparison 必须 import 自该路径,故与 Phase 2 同 commit)。
- ExplorationSession / ExplorationMessage 迁至
  storage/exploration_repository.py;repository.py 由 1180 行降至
  约 814 行,grep 确认无残留符号。
- exploration 子模块与路由 import 全部更新;测试 import 跟随。

刻意不做:CAS 共享原语、app.py 五 registry 关停顺序归一
(ADR-0006 精神,等真实需求出现再议)。
2026-08-24 05:50:27 +08:00
sinohqb
84627a3c6a refactor(intelligent-eval): 消除 lifecycle.py 和 scheduler.py 中的重复延迟导入
All checks were successful
CI / test (push) Successful in 3m16s
- 将延迟导入移至模块顶部,消除 Shotgun Surgery 气味
- lifecycle.py: 移除 41 行重复导入
- scheduler.py: 移除 14 行重复导入
- 修复测试:更新 monkeypatch 以补丁 scheduler 模块的引用而非原始模块
- 符合代码规范:避免函数内重复导入

Closes code-review finding: repeated deferred imports (Shotgun Surgery)
2026-08-24 01:47:47 +08:00
sinohqb
2a0bcdd185 refactor(intelligent-eval): drop ADR-0008 transitional wrappers, callers use domain
All checks were successful
CI / test (push) Successful in 3m8s
架构审查候选④:ADR-0008 收敛调度域时为保测试兼容留下的过渡 wrapper 使命结束。

删除 8 个浅封装:task_queue 的 _is_slot_due / _calculate_session_deficit
(零调用死函数)+ _calculate_priority / _get_attention_reason,decision 的
_parse_time_slot(零调用死函数)+ _get_current_slot / _count_sessions_in_slot /
_has_high_severity_issues。调用方直接使用 domain 模块。

5 个隔着 wrapper 测 domain 行为的测试迁到新文件
test_intelligent_eval_domain.py,直接锁定 domain,覆盖零丢失。
删除测试通过:复杂度直接消失,时段/欠账/优先级知识只剩 domain 一处。
870 tests passed,零行为变化。
2026-08-21 03:50:44 +08:00
sinohqb
58c2ad0227 refactor(intelligent-eval): unify watchdog failures behind fail_eval seam
All checks were successful
CI / test (push) Successful in 3m10s
架构审查候选③(状态机归一):_TRANSITIONS 成为评估状态机的唯一真相。

三个 watchdog(planning 双闸 / executing 兜底 / 触发失败闸门)原先直接
row.status = FAILED 绕过转换表、手工重复写字段。收编到新的公开接缝
fail_eval(session, eval_id, reason, decision_type, context):表校验 →
repo CAS 条件写(status/plan_feedback/updated_at/completed_at 一条 SQL)
→ append_decision_log 留痕。

CAS 冲突(如用户在扫描间隙抢先取消)跳过并留日志,不当故障。API 路径
本已在表内,不动。新增 4 个契约测试,870 tests passed,零行为变化。
2026-08-21 03:35:05 +08:00
sinohqb
182b0e59cb refactor(intelligent-eval): extract scheduler runtime + internalize decision-log dedup
All checks were successful
CI / test (push) Successful in 3m9s
架构深化两则(架构审查候选①②):

① scheduler 抽取:web/app.py 约 400 行触发式执行编排(60s 扫描循环、
docker exec 触发、失败落账)沉入 intelligent_eval/scheduler.py,runtime
单例 start()/stop()/scan_once() 与 campaign_runtime 惯例一致;worker/planner
两处重复触发代码合并为一个触发原语;_supplement_decision_logs 归入
decision_logs.py。测试改为直接驱动 scan_once(interface 即测试面)。

② 决策日志去重内化:create_decision_log 的去重只服务 agent 上报路径;
新增 append_decision_log(平台落账纯追加)与 count_decisions(计数原语),
lifecycle/task_queue 全部平台落账切换,调用方不再需要塞 attempt 骗去重。

零行为变化:提示词、60s 节拍、编排顺序、闸门语义原样保留,866 tests passed。
2026-08-21 03:20:14 +08:00
sinohqb
d0487b54b4 fix(intelligent-eval): respect time-window slots before nudging analyst
冒烟发现 analyst_nudge 不考虑时间窗口:首会话完成后 10 分钟即催促,
但后续时段尚未到期,导致报告提前收敛、漏掉计划内会话。催促闸门新增
"窗口未结束且会话数未达计划则跳过";worker 触发指令同步明确仅当所有
时段会话都达终态才转 analyst。
2026-08-20 16:27:05 +08:00
sinohqb
eb4944a8bd feat(intelligent-eval): terminal-state discipline watchdogs (ADR-0011)
常见故障自愈有上限,超限收敛终态且可见:任务 attempts 上限、会话过期、
planning 双闸、executing 超窗兜底、触发失败计数判死、孤儿 agent 双管、
fire-and-forget 触发;open_session 预算硬闸门、settle 按终态区分、报告
scores 归一化;cron 池遗留面全删。
2026-08-20 14:34:17 +08:00
sinohqb
71d38ebef6 fix(intelligent-eval): settle stale tasks of finished evals
All checks were successful
CI / test (push) Successful in 3m57s
任务队列出现'评估已 completed 却有待认领/执行中任务'的残留:评估离开
executing 后,其 pending/assigned 任务无人清理(requeue_stale_assigned_tasks
只处理 executing 评估的 assigned,评估结束被跳过)。

- 新增 task_queue.settle_tasks_for_finished_evals:评估非 executing 时,
  其 pending/assigned 任务回收为 completed;executing 评估的任务保留
- scan loop 每 60s 在 requeue+scan 后调用清理
测试:+1(已结束评估的 pending/assigned 回收、executing 保留、幂等),901 passed
2026-08-18 16:04:46 +08:00
sinohqb
60c54a67e4 fix(intelligent-eval): parse minute-level time slots (1h windows)
All checks were successful
CI / test (push) Successful in 3m54s
planner 对短窗口(1h)产出分钟级时段(如 0-20min/20-40min/40-60min),
但 parse_time_slot 只支持小时级(8-10h),分钟格式解析失败返回 None →
is_slot_due=False → 审批后评估永不入队、不触发 worker。
- parse_time_slot 支持 h/min 后缀,统一换算成小时(float)返回
- is_slot_due/get_current_slot/count_sessions_in_slot 用 timedelta(hours=float)
  兼容两种格式;decision._parse_time_slot 类型标注同步 float
测试:+1(分钟格式时段解析与到期判断),898 passed
2026-08-18 13:15:46 +08:00
sinohqb
71c7cd3d39 fix(intelligent-eval): requeue stale assigned tasks (worker crash recovery)
All checks were successful
CI / test (push) Successful in 3m45s
方案③ worker 由平台触发 openclaw agent(cron=manual-run,非真实 cron),
fault_tolerance 的 stuck 检测不适用——agent 中断/失败时任务永久卡 assigned,
scan 只查 pending 不再入队(死锁)。

requeue_stale_assigned_tasks:assigned 超过 10 分钟且评估仍 executing 的
任务重置为 pending(清空认领),平台 scan 循环随后重新触发 worker 重试。
接入 scan loop,每轮先 requeue 再 scan。
2026-08-17 03:30:42 +08:00
sinohqb
f85eca11ca fix(alerts): webhook retry + dedupe (resolves §6.2)
All checks were successful
CI / test (push) Successful in 3m57s
_send_webhook:
- Dedupe: skip if alert.webhook_sent is already True (guards against
  repeated check_alerts ticks re-sending the same alert).
- Retry: up to 3 attempts with exponential backoff (1s, 2s) before
  giving up. webhook_sent=True is persisted only on success.

Two xfail guards in test_openclaw_client_and_webhook now pass (876/4 xfail).
2026-08-14 14:54:57 +08:00
sinohqb
a03067720b test(intelligent-eval): add openclaw_client + webhook tests (#5)
All checks were successful
CI / test (push) Successful in 4m18s
T5 OpenClawClient subprocess args: docker exec cmd + token-last placement,
   cron add/rm/list params, JSON parse success + failure paths.
T6 webhook: marked sent on 2xx, failure does not raise; xfail guards expose
   missing retry and missing dedupe (.scratch/v111-architecture-scan.md §6.2).
2026-08-13 02:56:32 +08:00
sinohqb
4c45dba95d test(intelligent-eval): add cron_pool boundary tests (#4)
Some checks failed
CI / test (push) Has been cancelled
T4 auto-scale boundary: busy/total == 0.8 (exact threshold) must not fire
T4 auto-scale oscillation: repeated flips across 0.8 must respect MIN/MAX bounds
T9 stuck-detection critical edge: strict '<' semantics at 10min threshold
T9 detect_stuck_crons ignores non-busy: idle/stuck crons excluded even if stale
2026-08-13 02:54:21 +08:00
sinohqb
2e7d419f05 feat(intelligent-eval): implement monitoring and alerting (ticket 07)
- Add metrics.py with pool utilization, task backlog, stuck rate, avg processing time, eval completion rate
- Add alerts.py with alert rules (pool utilization > 90%, task backlog > 50, stuck rate > 10%)
- Implement alert history and webhook notifications
- Add metrics and alerts APIs
- Add database migration for alert history table
- Add 11 unit tests for metrics, 10 unit tests for alerts, 8 integration tests
- Update migration tests to include new alert history table

All 853 tests passing.
2026-08-12 10:41:15 +08:00
sinohqb
1d9228fd86 feat(intelligent-eval): implement fault tolerance and recovery (ticket 06)
- Add fault_tolerance.py with stuck cron detection and handling
- Implement state reconciliation (platform DB vs OpenClaw state)
- Implement platform restart recovery (requeue inactive tasks)
- Implement OpenClaw restart recovery (sync cron states)
- Add 6 unit tests and 5 integration tests

All 824 tests passing.
2026-08-12 10:30:05 +08:00
sinohqb
e6f98aaa6d feat(intelligent-eval): implement config snapshot management (ticket 05)
- Add config_snapshot.py with save/list/get/compare functions
- Auto-save snapshots on eval creation and plan submission
- Implement snapshot query APIs (list, get single)
- Implement snapshot comparison API (diff two snapshots)
- Add 8 unit tests and 7 integration tests

Snapshots track config changes over time (created/plan_submitted/config_updated).
All 813 tests passing.
2026-08-12 10:19:07 +08:00
sinohqb
fe3399297c feat(intelligent-eval): implement decision logic and e2e flow (ticket 04)
- Add decision.py with worker decision logic (execute_session/wait/start_analysis)
- Implement time slot parsing and current slot detection
- Implement session deficit calculation per time slot
- Implement high severity issue detection
- Implement eval completion detection
- Add 12 unit tests for decision logic
- Add 2 end-to-end tests for complete lifecycle

All 798 tests passing.
2026-08-12 10:10:42 +08:00
sinohqb
2ff023a65b feat(intelligent-eval): implement cron pool management (ticket 02)
- Add OpenClawClient wrapping CLI commands (create/delete/list crons)
- Implement pool initialization, scale up/down, auto-scaling logic
- Implement cron state sync and stuck cron detection
- Add pool status and manual scaling APIs
- Add 13 unit tests and 5 integration tests

Pool automatically scales between 5-20 crons based on load.
All 778 tests passing.
2026-08-12 09:47:04 +08:00
sinohqb
1aa453ef0a feat(intelligent-eval): add cron pool data model and task queue API
Implement Ticket 01 of intelligent eval cron pool architecture (ADR-0007):

- Add 4 new tables: task_queue, cron_pool, config_snapshots, decision_logs
- Implement task enqueueing logic with priority calculation
- Implement task assignment and completion APIs
- Add unit tests (9) and integration tests (7)
- Update CONTEXT.md with new vocabulary
- Add ADR-0007 documenting cron pool architecture decision

All 760 tests passing.
2026-08-12 02:13:21 +08:00
sinohqb
1782b245bf refactor(architecture): deepen campaign runtime modules 2026-08-11 13:18:48 +08:00
sinohqb
864ae2b7fe fix(review): address release correctness findings 2026-08-09 03:20:40 +08:00
sinohqb
62485684ca fix(architecture): enforce lifecycle consistency 2026-08-07 10:59:27 +08:00
sinohqb
c896ab3f71 refactor(architecture): deepen evaluation lifecycle and read model 2026-08-07 03:11:37 +08:00
sinohqb
9cdbc41808 docs(intelligent-eval): commit v1.0 spec, tickets, and post-v1.0 improvements
All 8 tickets' acceptance criteria checked off (incl. the real OpenClaw
E2E verified on t480). Trivial import-sort fix from ruff included.
2026-08-05 14:01:24 +08:00
sinohqb
1317552701 feat(intelligent-eval): add backend for OpenClaw-driven intelligent evaluation (tickets 01-04)
Introduce 智能评估 as an evaluation paradigm parallel to static evaluation,
driven by OpenClaw. The platform supplies storage, lifecycle, and reporting;
OpenClaw plans and executes.

- Data model: IntelligentEval + Session + Message tables (new, not reusing exploration)
- Lifecycle state machine: draft → planning → pending_approval → executing → completed/cancelled/failed
- Session API: create/message (channel-forwarded)/close with turn accounting
- Report API: pydantic-validated structured report, executing → completed, Markdown export (pure renderer)
- Alembic migration for the three tables; domain glossary added to CONTEXT.md
2026-08-05 03:18:52 +08:00
sinohqb
c24998c762 refactor(metrics): extract dashboard aggregation to compute_dashboard
仪表盘聚合逻辑从 stats.py router 下沉到 metrics.py 的 compute_dashboard
纯函数。_settled 重命名为 settled_runs 并公开,_ts 重命名为 _sortable_ts。
router 从 40 行聚合逻辑缩到 5 行,只负责数据获取和序列化。

- 新增 compute_dashboard(runs, scenario_names, target_names) -> dict
- 新增 settled_runs(runs) 公开接口(原 _settled)
- trend 端点同步迁移到 settled_runs
- 5 个新测试覆盖 dashboard 聚合逻辑
2026-08-04 11:36:55 +08:00
sinohqb
42be31dd1f feat(report): add load_campaign_view as unified campaign read model
活动级读模型单一出口:一次取齐报告 / 探索 / 分析 / 对比四大数据源。
markdown handler 从 30 行拼装逻辑缩到 3 行;分析执行器同步迁移。
comparison.py 内部的 8 次 load_campaign_report 调用暂不动(跨请求冗余,
缓存收益有限,改动风险高)。

- 新增 load_campaign_view(session, campaign) -> dict[str, Any]
- 返回 {report, exploration, analysis, comparison} 四键
- 迁移 markdown handler 和分析执行器两个调用点
- 4 个新测试覆盖 view 的组装逻辑
2026-08-04 11:33:45 +08:00
sinohqb
2fddce8c92 refactor(case-verdict): extract build_case_evidence as single evidence-construction seam
用例判定证据构建收敛到 case_verdict.py 的 build_case_evidence 纯函数,
report.py 和 runs.py 各删 ~15 行重复逻辑,换一行调用。locality 回归:
证据构建改一处,全局生效。

- 新增 build_case_evidence(turns, results) -> dict[str, CaseEvidence]
- report.py:76-83 证据构建替换为一行调用
- runs.py:175-201 证据构建替换为一行调用
- 5 个新测试覆盖 build_case_evidence(纯函数,无 DB 依赖)
2026-08-04 11:23:56 +08:00
sinohqb
2b6cab6cb2 feat(comparison): unify read model and validation for period comparison
周期对比读模型升位为单一出口(load_comparison_view),GET/POST/markdown
三处调用点统一走同一 view 投影,消除「取数三件套」重复。校验逻辑收敛到
validate_comparison_request,router 捕获映射 400,执行器捕获落 failed 行,
校验顺序权威不再漂移。

- 新增 load_comparison_view:无行返回 status=none + auto_baseline,有行
  返回完整 comparison dict(含 model_name 标签)+ metric_diff
- 新增 validate_comparison_request:活动终态 → 模型 → 基线 → 分析,违
  规抛 ComparisonError
- execute_campaign_comparison 内联校验替换为 validate_comparison_request
  调用,catch ComparisonError 落 failed 行
- router 三处迁移:GET /comparison、POST /comparison、markdown 导出
- 删除 build_comparison_payload(已吸收进 load_comparison_view)
- 8 个新测试覆盖读模型三态 + 校验五错
2026-08-04 10:49:14 +08:00
sinohqb
f3a528611e refactor(tasks): route LLM background tasks through TaskRegistry
架构保养第二轮候选 1:分析 / 周期对比 / judge 复核三条 LLM 任务链
收进各自的模块级 TaskRegistry(强引用防 GC、按 id 幂等、shutdown
统一收敛),删除 judge 的 _BACKGROUND_TASKS 私货,start_* 不再返回
无人消费的 Task。启动清理块补两笔 orphan 清扫:滞留的 generating
分析与对比行标记为 failed,与僵尸运行清扫同构。新增 7 个单测。
2026-08-04 09:58:20 +08:00
sinohqb
df76edcf55 refactor(exploration): move ledger and state machine into domain modules
Some checks failed
CI / test (push) Failing after 33s
架构保养候选 3:探索生命周期的账本规则与状态机从 HTTP 层落入
exploration/lifecycle.py(open/conduct/close)与 patrol.py(巡检读模型),
违规改用类型化领域异常(NotFound/Guardrail/Channel),router 瘦回纯
HTTP 翻译(404/409/502 映射),领域层不再依赖 fastapi,可脱离
TestClient 直测(新增 12 个单元测试)。
2026-08-04 03:46:51 +08:00
sinohqb
38849d46f1 refactor(repository): narrow atomic updates for patrol/cancel/scheduler writes
全行 update 逼着调用方承诺「持有最新完整快照」,巡检水位、活动取消、
调度循环三方并发写入会互相覆盖(后写者抹掉先写者的 summary/status)。
CampaignRepository 新增三个窄口径原子操作:touch_patrol_watermark、
mark_cancelled、save_scheduler_state,各自只写自己的列;三处调用点迁移。
2026-08-04 03:39:42 +08:00
sinohqb
f8d8450b1e refactor(report): unify campaign report loading behind one read model
「活动报告取数三件套」此前在报告/markdown/分析/对比等 7 处手写重复,
唯一深化产物 build_campaign_report_dict 被锁在周期对比私有角落。
升位为 report.py 的 load_campaign_report(session, campaign) 单一出口
(探索线 summarize_campaign_exploration 同口径),并把 8 处
scenario_names 推导式收敛为 ScenarioRepository.name_map() 窄方法。
纯结构重排、零行为变更,572 项测试全绿。
2026-08-04 03:30:29 +08:00
sinohqb
936640fb36 fix(exploration): include all judge findings instead of poor-only
Ticket 05 asks the judge review conclusions to flow into the report
verbatim; the aggregation silently dropped good/acceptable dimensions.
Collect every finding sorted poor-first and color drawer tags by rating.
2026-08-04 02:45:41 +08:00
sinohqb
2484c207af feat(exploration): findings flow into report, analysis and export
Exploration sessions aggregate into a single exploration summary
(session counts, goal-achievement rate, issue lists from experience
records, judge conclusions when reviewed) that feeds three exits:
the campaign report gains an exploration dimension, the v0.7 analysis
stage-two input gains the summary (stats only, never full dialogues),
and the Markdown export appends a findings appendix after analysis and
comparison. With no exploration data every output stays unchanged.
2026-08-03 19:16:33 +08:00
sinohqb
9cf64ab0e2 test(report): aggregation edge cases and output format validation
Adds unit tests for the campaign report beyond value correctness:
cancellation exclusion from denominators (ADR-0004), window clamping,
naive/aware timestamp equivalence, plus schema validation of the report
dict (key sets, types, rate bounds, contiguous buckets, capability sort,
ISO-UTC timestamps) and consistency with the single aggregate_runs seam.
2026-08-03 19:03:44 +08:00
sinohqb
2285a25009 feat(exploration): judge sampling review after session close
After an exploration session closes, the platform samples up to 3
conversation rounds and runs an independent judge-role review through
the v0.7 ChatClient seam, persisting quality-dimension conclusions
(attitude, professionalism, hallucination) into the session's
judge_review. The review runs as a background task: failures are
recorded without touching session state or the first-hand experience
record, and a missing model config skips silently.
2026-08-03 18:59:40 +08:00