Commit Graph

3 Commits

Author SHA1 Message Date
sinohqb
2f09ee2bfc refactor(v1.3.1): Phase 3 报告横幅、评分逻辑收敛与成本闭环
All checks were successful
CI / test (pull_request) Successful in 3m58s
- 报告渲染 Go/No-Go 上线评估横幅(HTML 彩色 banner + Markdown 引用块)
- 抽取 scored_llm 共享模块:llm_score / fluency 直连调用与评分解析收敛
- 网关新增 chat_with_usage / embed_with_usage,规则按次归集 llm_usage
- 引擎分岗位用量归集(judge/generator/embedding/moderation)写入
  RunSummary.eval_usage_by_purpose,并发下不做总量差值
- cost_tracking 重构:data/model_pricing.json 覆盖 + 默认计价表,
  删除从未有数据支撑的 Turn 维度成本函数(偏差说明见 PR)
- 报告 summary 增加 eval_cost 分岗位成本段并在 Markdown 渲染
2026-08-26 01:59:20 +08:00
sinohqb
c956da7686 feat(v1.3.1): Phase 2 让成本/放弃率/Go-No-Go 基础设施真正生效
All checks were successful
CI / test (pull_request) Successful in 4m3s
- token 用量接入:ModelGateway 经 adapter.parse_usage 累计评测侧 LLM 调用的
  token 消耗,引擎写入 run.summary.eval_token_usage,报告透出
- 放弃率落地:CaseOutcome 新增 abandoned 标记(对话中途发送/接收失败),
  build_run_summary 统计 abandoned_cases / abandonment_rate
- Go/No-Go 可配置:Scenario 新增 acceptance_criteria 字段(DB 列 + 幂等迁移),
  报告按场景标准出 verdict,缺省回退全局默认;标准变更不触发考纲升版
2026-08-25 18:38:18 +08:00
sinohqb
9c01afa79b refactor(engine): extract build_run_summary pure seam
Single-run summary口径 (pass_rate / judged_pass_rate / avg_latency /
connectivity split) was inlined in run(), reachable only by driving a
whole async run, and report.py recomputed judged_pass_rate independently.
Extract build_run_summary — a pure function parallel to aggregate_runs
(cross-run) and combine_case_outcome (case-level). run() now collects
material and delegates; judged_pass_rate is stored in RunSummary so the
report reads it instead of recomputing.
2026-07-31 14:20:51 +08:00