Commit Graph

12 Commits

Author SHA1 Message Date
sinohqb
c896ab3f71 refactor(architecture): deepen evaluation lifecycle and read model 2026-08-07 03:11:37 +08:00
sinohqb
1317552701 feat(intelligent-eval): add backend for OpenClaw-driven intelligent evaluation (tickets 01-04)
Introduce 智能评估 as an evaluation paradigm parallel to static evaluation,
driven by OpenClaw. The platform supplies storage, lifecycle, and reporting;
OpenClaw plans and executes.

- Data model: IntelligentEval + Session + Message tables (new, not reusing exploration)
- Lifecycle state machine: draft → planning → pending_approval → executing → completed/cancelled/failed
- Session API: create/message (channel-forwarded)/close with turn accounting
- Report API: pydantic-validated structured report, executing → completed, Markdown export (pure renderer)
- Alembic migration for the three tables; domain glossary added to CONTEXT.md
2026-08-05 03:18:52 +08:00
sinohqb
2484c207af feat(exploration): findings flow into report, analysis and export
Exploration sessions aggregate into a single exploration summary
(session counts, goal-achievement rate, issue lists from experience
records, judge conclusions when reviewed) that feeds three exits:
the campaign report gains an exploration dimension, the v0.7 analysis
stage-two input gains the summary (stats only, never full dialogues),
and the Markdown export appends a findings appendix after analysis and
comparison. With no exploration data every output stays unchanged.
2026-08-03 19:16:33 +08:00
sinohqb
6340ec503c feat(exploration): stateless patrol API with watermark increments
Resident agents call GET /api/exploration/patrol once per cycle to see
every running production-line campaign that opted into exploration
(seed set present), the new results since the last watermark (reusing
campaign report aggregation), and the remaining exploration budget.
The watermark advances after each call so subsequent calls only report
increments; accelerated and terminal campaigns are excluded.
2026-08-03 18:12:11 +08:00
sinohqb
53afb9b5d1 feat(campaigns): exploration seed set and budget config per campaign
v0.9 ticket 02. Campaigns now carry an exploration seed set (seed
personas × seed goals — the comparability unit for exploratory
evaluation) and an optional budget override, stored as JSON columns
isomorphic to plan. Empty seeds normalize to null, marking the campaign
as opted out of exploration. resolve_budget merges per-field overrides
into platform defaults; enforcement stays server-side. The create form
gains seed lists and budget inputs (minutes → seconds), submitting null
when left empty.
2026-08-03 17:52:25 +08:00
sinohqb
5ecb30876e style(tests): ruff 全量清理 — 49 项修复,backend 与 tests 全绿
- ruff --fix 自动修正 44 项:移除未用 import(pytest 等)、import 块排序归一(I001)
- 手工修复剩余 5 项:test_cascade.py 两处未用赋值(F841);test_s2_rules_and_logic.py 中部 import 移至文件顶部(E402 ×3)
- 无行为变更:全量 492 项测试通过
2026-08-03 15:13:24 +08:00
sinohqb
c63236f671 feat(campaign): support per-campaign analysis model override
Campaigns can pin an analysis model config instead of following the
global analysis default. Creation validates the referenced config
exists (400 otherwise); the create form offers enabled chat configs
with the global default as the fallback option.
2026-08-03 01:53:13 +08:00
sinohqb
8bc5aa6979 feat(campaign): add per-Run timeline seam + endpoint
build_campaign_timeline flattens a campaign's child Runs into offset-sorted
per-Run entries (distinct from the report's 12-bucket aggregation), reusing a
shared _run_window_offset口径 so both views place a run identically. Exposes
GET /campaigns/{id}/timeline and the api.ts type/call. (v0.6 ticket 06)
2026-07-31 16:49:04 +08:00
sinohqb
7ed765726f feat(campaigns): live list progress, polling, and richer drill-down
Embed compact progress (completed/planned total + overall pass_rate,
reusing the report's aggregation) into GET /campaigns so the list drops
its N+1 report fetch. Poll list and open report drawer every 5s while the
tab is active and a campaign is still running. Show scenario version and
trigger source tags in the child-run drill-down.
2026-07-30 15:35:18 +08:00
sinohqb
f433ebb970 feat(campaigns): dual-axis periodic report (time trend + capability)
Add generate_campaign_report: a pure aggregator over a campaign's child Runs
producing a time-trend axis (Runs bucketed by service-window position) and a
capability-summary axis (grouped by scenario), each carrying pass_rate /
availability / latency. pass_rate keeps the single-Run case-level meaning and
counts execution failures as 0.0 (ADR-0002); time_scale only places Runs into
window-time buckets and never alters any figure. Engine summary now records
avg_latency_ms to feed the latency axis.

Expose GET /api/campaigns/{id}/report (structured) and .../report/markdown
(reusing the existing Markdown export path). Adds "可用性/Availability" to the
domain glossary.
2026-07-30 13:55:32 +08:00
sinohqb
8910fd17e0 feat(campaigns): durable scheduler loop with restart recovery and cancel
Add a thin async loop (run_campaign_loop) that ticks on real wall-clock time,
maps elapsed×time_scale to a window offset via the pure decide_schedule, spawns
due child Runs, and marks the campaign COMPLETED at window end. All authority
lives in the DB (started_at, spawned_indices, status), so the app lifespan can
resume every RUNNING campaign on startup without double-spawning and stop all
loops gracefully on shutdown. A failing plan entry is skipped and recorded
rather than wedging the campaign.

Creating a campaign now starts its loop; POST /api/campaigns/{id}/cancel stops
further spawning (completed child Runs are kept); GET /api/campaigns/{id}
reports live progress (window offset, spawned/completed Run counts).
2026-07-30 13:33:10 +08:00
sinohqb
e4404f1fa2 feat(campaigns): add Campaign persistence and create/query API
Introduce the 评估活动 (Campaign) aggregate above Run: a single-target,
service-cycle window driving a static plan. Adds Campaign/CampaignPlanEntry
models, CampaignDB table, nullable eval_runs.campaign_id, CampaignRepository,
Alembic migration, and POST/GET /api/campaigns with validation.

Ticket 01 of v0.6; no scheduling or child-run spawning yet (ADR-0003 v1).
2026-07-30 11:56:19 +08:00