Single-run summary口径 (pass_rate / judged_pass_rate / avg_latency /
connectivity split) was inlined in run(), reachable only by driving a
whole async run, and report.py recomputed judged_pass_rate independently.
Extract build_run_summary — a pure function parallel to aggregate_runs
(cross-run) and combine_case_outcome (case-level). run() now collects
material and delegates; judged_pass_rate is stored in RunSummary so the
report reads it instead of recomputing.
Read paths recomputed per-case pass/connectivity independently — report
generation, the logs endpoint, and the frontend each derived it, and the
frontend's every(passed) recompute ignored the engine's authoritative
verdict. Extract resolve_case_verdicts: a single pure seam that prefers
stored case_outcomes verbatim and approximates only for legacy runs. The
logs endpoint now surfaces case_verdicts so the frontend reads instead of
recomputing.
Seven pages repeated the same load-on-mount + loading + try/finally +
reload-button skeleton, each re-implementing tab-active refresh, silent
polling, and (in two places) a hand-rolled requestId race guard. Extract two
composable hooks: useResource(fetcher, {tabPath, deps}) owning data/loading/
reload with a built-in race guard and auto tab-active refresh, and
usePolling(fn, ms, enabled) replacing the hand-written setInterval effects.
Migrate all seven pages onto them; Targets/Scenarios/ModelConfigs also gain a
uniform tab-active refresh they previously lacked. Verified via tsc --noEmit
and npm run build (no frontend test runner exists).
report.py mixed DB-reading generation with string formatting: the four
render_*_report(run_id, session) functions each re-fetched via
generate_report, so the HTML/Markdown/JSON formatting was welded to storage
and could not be unit-tested from a plain dict. Extract the formatting into a
new pure report_render module whose renderers take the already-built report
dict (no session, no storage import). Migrate every caller to generate-then-
render, delete the old coupled renderers with no back-compat shim, and drop
the _aggregate_runs middle-man alias in favour of metrics.aggregate_runs.
Both the single-run path and the campaign scheduler drove long-lived
asyncio tasks through their own duplicated _tasks/_cancel_events dicts and
shutdown loops. Collapse them into one deep TaskRegistry module,
instantiated as run_registry and campaign_registry. launch() creates the
cancel event before the task (so a cancel during startup is never lost),
wires done-callback cleanup, and is idempotent per id; this makes runs.py's
hard-cancel fallback provably dead, so it is removed. App shutdown now
gracefully stops in-flight runs too, not just campaigns.
Extend the pure scheduler with elapsed_seconds (clock injected), decide_tick
(offset + due + lifecycle action) and resolve_finalize (cancel-race guard),
so the durable loop stops hand-coding elapsed/finished/status checks and only
does I/O. Deletes the runner's private _elapsed_seconds and converges
current_window_offset onto the one pure elapsed computation. The clock-skew
tolerance and cancel-race guard are now unit-testable at the seam.
Give EvalRun.summary a typed RunSummary value (unified RunError, lenient
legacy parsing) so readers stop reaching into a schemaless dict, and route
every cross-run rollup — dashboard, scenario ranking, trend, campaign
report — through one aggregate_runs seam. Fixes the divergence where
stats averaged pass_rate over completed-only runs while the campaign
report counted faults as 0.0. Cross-run rule (ADR-0004): genuine faults
count 0.0, user-cancelled runs are excluded from both denominators.
Embed compact progress (completed/planned total + overall pass_rate,
reusing the report's aggregation) into GET /campaigns so the list drops
its N+1 report fetch. Poll list and open report drawer every 5s while the
tab is active and a campaign is still running. Show scenario version and
trigger source tags in the child-run drill-down.
Register a keep-alive "评估活动" tab that creates campaigns (target,
window, time_scale, static plan), lists them with live progress and
pass-rate, and opens a report drawer with a time-trend line, capability
summary, and drill-down into child Runs.
Add generate_campaign_report: a pure aggregator over a campaign's child Runs
producing a time-trend axis (Runs bucketed by service-window position) and a
capability-summary axis (grouped by scenario), each carrying pass_rate /
availability / latency. pass_rate keeps the single-Run case-level meaning and
counts execution failures as 0.0 (ADR-0002); time_scale only places Runs into
window-time buckets and never alters any figure. Engine summary now records
avg_latency_ms to feed the latency axis.
Expose GET /api/campaigns/{id}/report (structured) and .../report/markdown
(reusing the existing Markdown export path). Adds "可用性/Availability" to the
domain glossary.
Add a thin async loop (run_campaign_loop) that ticks on real wall-clock time,
maps elapsed×time_scale to a window offset via the pure decide_schedule, spawns
due child Runs, and marks the campaign COMPLETED at window end. All authority
lives in the DB (started_at, spawned_indices, status), so the app lifespan can
resume every RUNNING campaign on startup without double-spawning and stop all
loops gracefully on shutdown. A failing plan entry is skipped and recorded
rather than wedging the campaign.
Creating a campaign now starts its loop; POST /api/campaigns/{id}/cancel stops
further spawning (completed child Runs are kept); GET /api/campaigns/{id}
reports live progress (window offset, spawned/completed Run counts).
Add the pure scheduling seam (campaign_scheduler.decide_schedule) that, given a
static plan and window-clock offset, decides which plan entries are due and
whether the window ended — mirroring judgement.combine_case_outcome, with
time_scale confined to the clock mapping so it never touches judgement/report.
The campaign_runner shell maps injected elapsed time to a window offset, spawns
due child Runs through the existing EvalEngine.run(existing_run=...) path with
campaign_id + RunTrigger.CAMPAIGN, and persists spawned-entry indices per entry
for idempotent, restart-recoverable progress. No auto loop yet (ticket 03).
