Refactor: Deepen evaluation architecture modules #1

New Issue

solahqb · 2026-08-06T07:09:57Z

solahqb commented

2026-08-06 07:09:57 +00:00

Refactor: Deepen evaluation architecture modules

Problem Statement

AgentEvalTool 最近的高频演进集中在评测运行、探索式评测、智能评估和评估活动。当前架构在四处泄漏了本应由 deep module 吸收的知识：三个调用方分别编排 Channel 的发送与轮询协议；智能评估状态规则与 SQLite 写入分属两个公开写入面；评估活动的创建、取消、完成、恢复和自动分析跨 HTTP adapter、runner 与 Repository 协调；智能评估列表和详情由 Router 与前端 renderer 分别拼装，产生 N+1 查询和不一致快照。

这些问题降低了 interface 的 leverage，也使测试越过真正的 seam。重构目标是在不改变领域口径、外部 HTTP 契约和管理后台信息架构的前提下，提高 locality、原子性、耐久性和可测试性。

Solution

按依赖顺序完成四次 deepening：首先让现有 Channel module 提供完整往返 interface；随后把智能评估的状态迁移与 persistence 收进单一 lifecycle module；再让评估活动 lifecycle 吸收耐久调度协调，并用持久化子 Run 身份保证跨重启不重复创建；最后建立后端 read model 与前端读取 module，集中批量查询、快照、polling 和错误状态。

每一阶段先建立新 interface 和测试面，再迁移调用方，最后删除旧 interface 与重复测试。任何临时兼容只存在于相邻提交之间，不成为长期支持面。

Commits

Phase 1 — Deepen the Channel module

Add the complete-exchange outcome model and contract tests. Introduce success and expected-failure outcomes covering send failure, reply timeout, polling failure, normalized text, latency, correlation identity and optional opaque diagnostic payload. Keep the existing transport primitives temporarily so the repository remains green.
Add the send-confirmed persistence hook to the exchange orchestration. Ensure the hook is awaited after a successful send and before polling starts. Add tests proving hook failure prevents polling and that no hook runs when sending fails.
Add narrow persistence support for completing an already-recorded evaluation Turn. Allow the evaluation ledger to record the sent fact before polling and attach reply, latency and diagnostic payload afterward without replacing unrelated fields. Verify both successful completion and timeout/failure paths.
Migrate the evaluation engine to the complete-exchange interface. Preserve Run, Case and Turn events, cancellation checkpoints, rule evaluation and error summaries. Replace engine tests that duplicate transport sequencing with tests of observable ledger and event outcomes.
Migrate exploration sessions to the complete-exchange interface. Preserve the rule that a successfully sent message immediately consumes budget and is durable before reply polling. Keep exploration-specific guardrails, typed errors and experience records outside Channel.
Migrate intelligent-evaluation sessions to the complete-exchange interface. Preserve session state checks and the two-transaction message ledger while removing local send/poll orchestration and reply-normalization duplication.
Make adapter transport primitives private and remove the old public Channel interface. Update Tutu, HTTP, OpenClaw and Mock adapters together, consolidate timeout ownership in Channel, and delete tests and helpers fully replaced by the new interface tests. Confirm no caller uses the old two-step protocol.

Phase 2 — Make intelligent-evaluation writes atomic

Introduce private compare-and-set persistence operations for intelligent evaluations. Add narrow conditional updates keyed by expected status, typed conflict results and transaction failure tests without changing Router behavior yet.
Make intelligent-evaluation creation atomic. Validate that the Target exists, create directly in planning state with correct timestamps, and remove the intermediate committed draft state. Preserve the current rule that an inactive Target is not rejected at creation.
Migrate approve and cancel to atomic lifecycle commands. Make each command a single conditional transaction and verify concurrent or repeated transitions return a conflict without overwriting the winning state.
Migrate plan submission, plan rejection and report submission to atomic lifecycle commands. Update payload fields, feedback, status and timestamps together. Add failure injection proving no half-updated plan, feedback or report can survive.
Move intelligent-evaluation session ownership and close invariants behind lifecycle. Make session creation and closing use narrow writes, enforce evaluation ownership in the lifecycle interface, and remove command-side ownership checks from the HTTP adapter.
Make intelligent-evaluation message ledger writes atomic at each durable point. Commit user message plus turn count in one transaction from the Channel send-confirmed hook, then commit the assistant reply separately. Preserve sent messages after timeout and prevent stale session objects from overwriting concurrent changes.
Close the public Repository write seam. Restrict persistence implementation exports, migrate remaining callers to lifecycle, replace direct Repository state tests with lifecycle tests over in-memory SQLite, and leave Router tests responsible only for validation and HTTP error translation.

Phase 3 — Deepen the durable Campaign lifecycle

Add nullable child-Run plan identity to the storage model and migration. Introduce plan-entry index and occurrence index with a Campaign-scoped uniqueness constraint. Keep all historical rows null, perform no backfill, and add upgrade/downgrade plus fresh-database parity tests.
Add an idempotent child-Run claim operation. Atomically create or retrieve the pending child Run for one Campaign plan occurrence, only while the Campaign remains running. Test repeated claims and competing sessions against the unique constraint.
Migrate schedule advancement to durable claims. Claim every due occurrence before execution, derive spawned progress from durable identities, and preserve per-entry error reporting. Add crash-point tests proving a restart cannot create a second Run for the same occurrence.
Add child-Run recovery reconciliation. Resume claimed pending Runs, mark process-orphaned running Runs failed with an explicit interruption reason, and never replay messages that may already have reached the Target. Verify ADR-0002 and ADR-0004 aggregation behavior remains unchanged.
Move Campaign creation behind lifecycle. Validate Target, Scenario, model and exploration inputs, then create the Campaign directly as running with its start timestamp in one transaction. Launch the in-process task only after commit so startup recovery can repair a crash between persistence and launch.
Move Campaign cancellation behind lifecycle. Atomically compare-and-set the Campaign to cancelled, write completion time and expire running exploration sessions before signaling the TaskRegistry. Add race tests proving no new claim succeeds after cancellation while already-started child Runs may finish.
Move Campaign completion and exploration settlement behind lifecycle. Atomically transition running to completed and expire dangling exploration sessions. Leave the Campaign running when settlement fails so a later tick can retry, and make completion idempotent under competing ticks.
Make production-line automatic analysis durable. Persist an analysis job before launching it, distinguish queued work from active generation, and resume unfinished jobs after restart. Preserve the rule that analysis never changes Campaign completion and may safely repeat an interrupted model call.
Unify Campaign startup recovery through the lifecycle interface. Reconcile running Campaigns, claimed child Runs and unfinished analysis jobs from database authority, while TaskRegistry remains an in-process mechanism. Remove Router and app-startup orchestration that bypasses lifecycle.
Close the public Campaign write seam. Route create, cancel, finalize and recovery through one lifecycle interface; keep the pure scheduler and persistence helpers as internal seams. Replace Router tests that spy on runner calls with lifecycle outcome tests using in-memory SQLite and a fake TaskRegistry.
Record the durable Campaign decisions. Amend the existing Campaign ADR with child-Run identity, pending-versus-running recovery, atomic exploration settlement and durable production-line analysis. State explicitly that cancelled Campaigns stop new claims but allow already-started Runs to finish.

Phase 4 — Build the intelligent-evaluation read model

Define typed list and detail projections. Make backend response models the contract authority. Keep full session messages out of the detail projection and preserve the existing on-demand transcript interface.
Implement the batched list projection. Load all session counts in a bounded number of queries and attach them to evaluation summaries without per-row reads. Add query-count regression tests and empty-list coverage.
Implement the consistent detail projection. Read evaluation state, plan, report and session summaries from one database snapshot. Centralize evaluation/session ownership checks and return not-found for cross-evaluation access.
Delegate existing read endpoints to the read model. Preserve session, message, JSON report and Markdown interfaces used by OpenClaw and other consumers while removing duplicate Router aggregation. Add compatibility tests for the unchanged response contracts.
Add the minimal frontend test foundation. Install Vitest, React Testing Library and the DOM test environment with one passing smoke test, isolated from the production Vite build. Update the lockfile and contributor commands.
Add the frontend intelligent-evaluation read adapter and state model. Represent initial loading, ready snapshot, silent refresh and initial failure without coupling the state model to Ant Design renderers. Test transitions with a fake owned-backend adapter.
Add status-aware polling to the frontend read module. Use fake timers to verify five-second refreshes for active states, stopping in terminal states, retaining the last complete snapshot on silent failure and replacing snapshots only as a whole.
Migrate the intelligent-evaluation page and detail renderer. Consume list and detail projections, remove local session polling and request choreography, and preserve keep-alive behavior, Drawer navigation, actions and user feedback.
Migrate the report renderer to the same detail snapshot. Reuse report and session summaries already present in the projection, keep transcripts lazy per expanded session and preserve Markdown export.
Remove obsolete frontend read orchestration and duplicated types. Delete redundant setters, effects and request combinations only after the new module tests cover their behavior. Keep transport compatibility methods that are still used by OpenClaw.
Run the complete regression and deployment build. Execute the full backend suite, frontend tests, type checking and production build; verify Alembic upgrade/downgrade on a copied populated SQLite database; confirm the repository contains no unexpected generated or user-owned changes.

Decision Document

The existing Channel module is deepened rather than wrapped by a parallel module.
Channel owns one complete technical exchange: send, polling, shared reply timeout, text normalization, latency and expected transport-failure classification.
Expected operational failures are returned as typed outcomes; programming and configuration errors remain exceptions.
Domain callers provide an awaited send-confirmed hook so ledger writes occur before reply polling.
Channel does not own Run, Turn, budget, state, scoring, WebSocket events, automatic send retry or immediate cancellation.
Success exposes normalized text, latency, correlation identity and an optional opaque diagnostic payload whose internal structure is not part of the contract.
Intelligent-evaluation lifecycle is the only public write authority; persistence is private implementation.
Intelligent-evaluation commands use narrow updates and expected-state compare-and-set. Ordinary commands are one transaction; message exchange commits once after send and once after reply.
Intelligent-evaluation read aggregation is deliberately deferred to the read-model phase.
SQLite is the sole Campaign state authority; TaskRegistry never determines durable status.
Campaign creation persists running before task launch. Cancellation and completion settle exploration state atomically before process-local follow-up.
New Campaign child Runs have a durable identity based on Campaign, plan entry and occurrence. Historical Runs remain null and are not backfilled.
Pending child Runs may resume; orphaned running child Runs fail rather than replay external messages.
Production-line analysis is durable and restartable. Cancelled Campaigns stop new child claims but allow already-started Runs to finish.
Intelligent-evaluation list and detail responses are stable projections from a single read snapshot. Transcripts stay lazy.
Existing read endpoints remain compatible and share the read-model implementation.
Backend response models are the contract authority; OpenAPI TypeScript generation is out of scope.
Frontend polling belongs to a dedicated read module, not page renderers. Silent failures retain the last complete snapshot.
ADR-0005 keep-alive routing, right-side Drawers and information architecture remain unchanged.

Testing Decisions

Good tests cross the same public interface as production callers and assert observable outcomes, persisted state and emitted domain events. They do not reach through the interface to inspect private persistence objects, TaskRegistry dictionaries or adapter sequencing.
Channel contract tests cover success, every expected failure category, timeout ownership, hook ordering, normalized output and opaque diagnostics. Existing HTTP/OpenClaw adapter tests provide prior art.
Evaluation-engine, exploration and intelligent-evaluation tests retain only their domain ledger, guardrail, state and event assertions, using a fake Channel adapter.
Intelligent-evaluation lifecycle tests use in-memory SQLite and cover valid/invalid transitions, expected-state races, transaction failure injection, ownership and two-stage message durability. Existing integration lifecycle tests provide prior art.
Campaign lifecycle tests use in-memory SQLite and a fake TaskRegistry. Pure scheduler tests remain internal-seam tests. Existing scheduler-loop, narrow-update, settlement and auto-analysis tests provide prior art.
Migration tests cover upgrade, downgrade, fresh-schema parity, nullable legacy rows and uniqueness for new child identities. Existing Campaign analysis migration tests provide prior art.
Read-model tests assert projection content, ownership rules, snapshot consistency and bounded query count.
Frontend tests use Vitest, React Testing Library, fake timers and a fake owned-backend adapter. They test state and user-visible behavior rather than effect implementation.
Final acceptance requires the complete backend suite, frontend unit tests, TypeScript checking, production build and migration rehearsal against a copy of populated SQLite data.

Out of Scope

Merging exploration sessions, intelligent-evaluation sessions or static evaluation Turns into one domain entity.
Changing pass-rate, availability, failure or cancellation aggregation defined by existing ADRs.
Adding automatic send retry, exponential backoff or immediate cancellation inside Channel.
Enforcing that a Target must be active when an intelligent evaluation is created.
Replaying a child Run that was already running when the process stopped.
Backfilling plan-entry identity for historical child Runs.
Removing or deprecating existing intelligent-evaluation read endpoints.
Introducing OpenAPI-generated frontend types.
Redesigning navigation, keep-alive behavior, Drawer layout or other ADR-0005 interactions.
Large-scale visual or screenshot testing.
Refactoring unrelated Repository modules solely because of file length.

Further Notes

Baseline before planning: the complete backend suite passes 674 tests, and frontend TypeScript checking passes.
Backend validation must use the repository's Python 3.11 environment; the system Python 3.9 executable is not suitable.
Before deployment, rehearse the SQLite batch migration on a copy of the t480 database and take a recoverable backup.
Preserve the existing untracked campaign helper scripts in the scratch directory; they are user-owned and outside this refactor.
The original architecture report is stored in the operating-system temporary directory and is not a durable project artifact; this issue draft is the durable decision and commit record.

# Refactor: Deepen evaluation architecture modules ## Problem Statement AgentEvalTool 最近的高频演进集中在评测运行、探索式评测、智能评估和评估活动。当前架构在四处泄漏了本应由 deep module 吸收的知识：三个调用方分别编排 Channel 的发送与轮询协议；智能评估状态规则与 SQLite 写入分属两个公开写入面；评估活动的创建、取消、完成、恢复和自动分析跨 HTTP adapter、runner 与 Repository 协调；智能评估列表和详情由 Router 与前端 renderer 分别拼装，产生 N+1 查询和不一致快照。这些问题降低了 interface 的 leverage，也使测试越过真正的 seam。重构目标是在不改变领域口径、外部 HTTP 契约和管理后台信息架构的前提下，提高 locality、原子性、耐久性和可测试性。 ## Solution 按依赖顺序完成四次 deepening：首先让现有 Channel module 提供完整往返 interface；随后把智能评估的状态迁移与 persistence 收进单一 lifecycle module；再让评估活动 lifecycle 吸收耐久调度协调，并用持久化子 Run 身份保证跨重启不重复创建；最后建立后端 read model 与前端读取 module，集中批量查询、快照、polling 和错误状态。每一阶段先建立新 interface 和测试面，再迁移调用方，最后删除旧 interface 与重复测试。任何临时兼容只存在于相邻提交之间，不成为长期支持面。 ## Commits ### Phase 1 — Deepen the Channel module 1. **Add the complete-exchange outcome model and contract tests.** Introduce success and expected-failure outcomes covering send failure, reply timeout, polling failure, normalized text, latency, correlation identity and optional opaque diagnostic payload. Keep the existing transport primitives temporarily so the repository remains green. 2. **Add the send-confirmed persistence hook to the exchange orchestration.** Ensure the hook is awaited after a successful send and before polling starts. Add tests proving hook failure prevents polling and that no hook runs when sending fails. 3. **Add narrow persistence support for completing an already-recorded evaluation Turn.** Allow the evaluation ledger to record the sent fact before polling and attach reply, latency and diagnostic payload afterward without replacing unrelated fields. Verify both successful completion and timeout/failure paths. 4. **Migrate the evaluation engine to the complete-exchange interface.** Preserve Run, Case and Turn events, cancellation checkpoints, rule evaluation and error summaries. Replace engine tests that duplicate transport sequencing with tests of observable ledger and event outcomes. 5. **Migrate exploration sessions to the complete-exchange interface.** Preserve the rule that a successfully sent message immediately consumes budget and is durable before reply polling. Keep exploration-specific guardrails, typed errors and experience records outside Channel. 6. **Migrate intelligent-evaluation sessions to the complete-exchange interface.** Preserve session state checks and the two-transaction message ledger while removing local send/poll orchestration and reply-normalization duplication. 7. **Make adapter transport primitives private and remove the old public Channel interface.** Update Tutu, HTTP, OpenClaw and Mock adapters together, consolidate timeout ownership in Channel, and delete tests and helpers fully replaced by the new interface tests. Confirm no caller uses the old two-step protocol. ### Phase 2 — Make intelligent-evaluation writes atomic 8. **Introduce private compare-and-set persistence operations for intelligent evaluations.** Add narrow conditional updates keyed by expected status, typed conflict results and transaction failure tests without changing Router behavior yet. 9. **Make intelligent-evaluation creation atomic.** Validate that the Target exists, create directly in planning state with correct timestamps, and remove the intermediate committed draft state. Preserve the current rule that an inactive Target is not rejected at creation. 10. **Migrate approve and cancel to atomic lifecycle commands.** Make each command a single conditional transaction and verify concurrent or repeated transitions return a conflict without overwriting the winning state. 11. **Migrate plan submission, plan rejection and report submission to atomic lifecycle commands.** Update payload fields, feedback, status and timestamps together. Add failure injection proving no half-updated plan, feedback or report can survive. 12. **Move intelligent-evaluation session ownership and close invariants behind lifecycle.** Make session creation and closing use narrow writes, enforce evaluation ownership in the lifecycle interface, and remove command-side ownership checks from the HTTP adapter. 13. **Make intelligent-evaluation message ledger writes atomic at each durable point.** Commit user message plus turn count in one transaction from the Channel send-confirmed hook, then commit the assistant reply separately. Preserve sent messages after timeout and prevent stale session objects from overwriting concurrent changes. 14. **Close the public Repository write seam.** Restrict persistence implementation exports, migrate remaining callers to lifecycle, replace direct Repository state tests with lifecycle tests over in-memory SQLite, and leave Router tests responsible only for validation and HTTP error translation. ### Phase 3 — Deepen the durable Campaign lifecycle 15. **Add nullable child-Run plan identity to the storage model and migration.** Introduce plan-entry index and occurrence index with a Campaign-scoped uniqueness constraint. Keep all historical rows null, perform no backfill, and add upgrade/downgrade plus fresh-database parity tests. 16. **Add an idempotent child-Run claim operation.** Atomically create or retrieve the pending child Run for one Campaign plan occurrence, only while the Campaign remains running. Test repeated claims and competing sessions against the unique constraint. 17. **Migrate schedule advancement to durable claims.** Claim every due occurrence before execution, derive spawned progress from durable identities, and preserve per-entry error reporting. Add crash-point tests proving a restart cannot create a second Run for the same occurrence. 18. **Add child-Run recovery reconciliation.** Resume claimed pending Runs, mark process-orphaned running Runs failed with an explicit interruption reason, and never replay messages that may already have reached the Target. Verify ADR-0002 and ADR-0004 aggregation behavior remains unchanged. 19. **Move Campaign creation behind lifecycle.** Validate Target, Scenario, model and exploration inputs, then create the Campaign directly as running with its start timestamp in one transaction. Launch the in-process task only after commit so startup recovery can repair a crash between persistence and launch. 20. **Move Campaign cancellation behind lifecycle.** Atomically compare-and-set the Campaign to cancelled, write completion time and expire running exploration sessions before signaling the TaskRegistry. Add race tests proving no new claim succeeds after cancellation while already-started child Runs may finish. 21. **Move Campaign completion and exploration settlement behind lifecycle.** Atomically transition running to completed and expire dangling exploration sessions. Leave the Campaign running when settlement fails so a later tick can retry, and make completion idempotent under competing ticks. 22. **Make production-line automatic analysis durable.** Persist an analysis job before launching it, distinguish queued work from active generation, and resume unfinished jobs after restart. Preserve the rule that analysis never changes Campaign completion and may safely repeat an interrupted model call. 23. **Unify Campaign startup recovery through the lifecycle interface.** Reconcile running Campaigns, claimed child Runs and unfinished analysis jobs from database authority, while TaskRegistry remains an in-process mechanism. Remove Router and app-startup orchestration that bypasses lifecycle. 24. **Close the public Campaign write seam.** Route create, cancel, finalize and recovery through one lifecycle interface; keep the pure scheduler and persistence helpers as internal seams. Replace Router tests that spy on runner calls with lifecycle outcome tests using in-memory SQLite and a fake TaskRegistry. 25. **Record the durable Campaign decisions.** Amend the existing Campaign ADR with child-Run identity, pending-versus-running recovery, atomic exploration settlement and durable production-line analysis. State explicitly that cancelled Campaigns stop new claims but allow already-started Runs to finish. ### Phase 4 — Build the intelligent-evaluation read model 26. **Define typed list and detail projections.** Make backend response models the contract authority. Keep full session messages out of the detail projection and preserve the existing on-demand transcript interface. 27. **Implement the batched list projection.** Load all session counts in a bounded number of queries and attach them to evaluation summaries without per-row reads. Add query-count regression tests and empty-list coverage. 28. **Implement the consistent detail projection.** Read evaluation state, plan, report and session summaries from one database snapshot. Centralize evaluation/session ownership checks and return not-found for cross-evaluation access. 29. **Delegate existing read endpoints to the read model.** Preserve session, message, JSON report and Markdown interfaces used by OpenClaw and other consumers while removing duplicate Router aggregation. Add compatibility tests for the unchanged response contracts. 30. **Add the minimal frontend test foundation.** Install Vitest, React Testing Library and the DOM test environment with one passing smoke test, isolated from the production Vite build. Update the lockfile and contributor commands. 31. **Add the frontend intelligent-evaluation read adapter and state model.** Represent initial loading, ready snapshot, silent refresh and initial failure without coupling the state model to Ant Design renderers. Test transitions with a fake owned-backend adapter. 32. **Add status-aware polling to the frontend read module.** Use fake timers to verify five-second refreshes for active states, stopping in terminal states, retaining the last complete snapshot on silent failure and replacing snapshots only as a whole. 33. **Migrate the intelligent-evaluation page and detail renderer.** Consume list and detail projections, remove local session polling and request choreography, and preserve keep-alive behavior, Drawer navigation, actions and user feedback. 34. **Migrate the report renderer to the same detail snapshot.** Reuse report and session summaries already present in the projection, keep transcripts lazy per expanded session and preserve Markdown export. 35. **Remove obsolete frontend read orchestration and duplicated types.** Delete redundant setters, effects and request combinations only after the new module tests cover their behavior. Keep transport compatibility methods that are still used by OpenClaw. 36. **Run the complete regression and deployment build.** Execute the full backend suite, frontend tests, type checking and production build; verify Alembic upgrade/downgrade on a copied populated SQLite database; confirm the repository contains no unexpected generated or user-owned changes. ## Decision Document - The existing Channel module is deepened rather than wrapped by a parallel module. - Channel owns one complete technical exchange: send, polling, shared reply timeout, text normalization, latency and expected transport-failure classification. - Expected operational failures are returned as typed outcomes; programming and configuration errors remain exceptions. - Domain callers provide an awaited send-confirmed hook so ledger writes occur before reply polling. - Channel does not own Run, Turn, budget, state, scoring, WebSocket events, automatic send retry or immediate cancellation. - Success exposes normalized text, latency, correlation identity and an optional opaque diagnostic payload whose internal structure is not part of the contract. - Intelligent-evaluation lifecycle is the only public write authority; persistence is private implementation. - Intelligent-evaluation commands use narrow updates and expected-state compare-and-set. Ordinary commands are one transaction; message exchange commits once after send and once after reply. - Intelligent-evaluation read aggregation is deliberately deferred to the read-model phase. - SQLite is the sole Campaign state authority; TaskRegistry never determines durable status. - Campaign creation persists running before task launch. Cancellation and completion settle exploration state atomically before process-local follow-up. - New Campaign child Runs have a durable identity based on Campaign, plan entry and occurrence. Historical Runs remain null and are not backfilled. - Pending child Runs may resume; orphaned running child Runs fail rather than replay external messages. - Production-line analysis is durable and restartable. Cancelled Campaigns stop new child claims but allow already-started Runs to finish. - Intelligent-evaluation list and detail responses are stable projections from a single read snapshot. Transcripts stay lazy. - Existing read endpoints remain compatible and share the read-model implementation. - Backend response models are the contract authority; OpenAPI TypeScript generation is out of scope. - Frontend polling belongs to a dedicated read module, not page renderers. Silent failures retain the last complete snapshot. - ADR-0005 keep-alive routing, right-side Drawers and information architecture remain unchanged. ## Testing Decisions - Good tests cross the same public interface as production callers and assert observable outcomes, persisted state and emitted domain events. They do not reach through the interface to inspect private persistence objects, TaskRegistry dictionaries or adapter sequencing. - Channel contract tests cover success, every expected failure category, timeout ownership, hook ordering, normalized output and opaque diagnostics. Existing HTTP/OpenClaw adapter tests provide prior art. - Evaluation-engine, exploration and intelligent-evaluation tests retain only their domain ledger, guardrail, state and event assertions, using a fake Channel adapter. - Intelligent-evaluation lifecycle tests use in-memory SQLite and cover valid/invalid transitions, expected-state races, transaction failure injection, ownership and two-stage message durability. Existing integration lifecycle tests provide prior art. - Campaign lifecycle tests use in-memory SQLite and a fake TaskRegistry. Pure scheduler tests remain internal-seam tests. Existing scheduler-loop, narrow-update, settlement and auto-analysis tests provide prior art. - Migration tests cover upgrade, downgrade, fresh-schema parity, nullable legacy rows and uniqueness for new child identities. Existing Campaign analysis migration tests provide prior art. - Read-model tests assert projection content, ownership rules, snapshot consistency and bounded query count. - Frontend tests use Vitest, React Testing Library, fake timers and a fake owned-backend adapter. They test state and user-visible behavior rather than effect implementation. - Final acceptance requires the complete backend suite, frontend unit tests, TypeScript checking, production build and migration rehearsal against a copy of populated SQLite data. ## Out of Scope - Merging exploration sessions, intelligent-evaluation sessions or static evaluation Turns into one domain entity. - Changing pass-rate, availability, failure or cancellation aggregation defined by existing ADRs. - Adding automatic send retry, exponential backoff or immediate cancellation inside Channel. - Enforcing that a Target must be active when an intelligent evaluation is created. - Replaying a child Run that was already running when the process stopped. - Backfilling plan-entry identity for historical child Runs. - Removing or deprecating existing intelligent-evaluation read endpoints. - Introducing OpenAPI-generated frontend types. - Redesigning navigation, keep-alive behavior, Drawer layout or other ADR-0005 interactions. - Large-scale visual or screenshot testing. - Refactoring unrelated Repository modules solely because of file length. ## Further Notes - Baseline before planning: the complete backend suite passes 674 tests, and frontend TypeScript checking passes. - Backend validation must use the repository's Python 3.11 environment; the system Python 3.9 executable is not suitable. - Before deployment, rehearse the SQLite batch migration on a copy of the t480 database and take a recoverable backup. - Preserve the existing untracked campaign helper scripts in the scratch directory; they are user-owned and outside this refactor. - The original architecture report is stored in the operating-system temporary directory and is not a durable project artifact; this issue draft is the durable decision and commit record.

solahqb commented

2026-08-11 08:44:51 +00:00

已完成并验收。核心架构深化由提交 c896ab3 完成，生命周期一致性修正由 6248568 完成；最终随 v1.0.0（0326ec5）通过 744 项后端测试、15 项前端测试、Ruff、TypeScript 类型检查和生产构建，并已部署至 t480 开发线与 volcengine-102 正式线。

已完成并验收。核心架构深化由提交 c896ab3 完成，生命周期一致性修正由 6248568 完成；最终随 v1.0.0（0326ec5）通过 744 项后端测试、15 项前端测试、Ruff、TypeScript 类型检查和生产构建，并已部署至 t480 开发线与 volcengine-102 正式线。

solahqb closed this issue

2026-08-11 08:44:51 +00:00

Sign in to join this conversation.