12481cd1b8
## 核心变更
### 规则层全面异步化（DEBT-1）
- EvalRule.evaluate() 签名改为 async def，全量同步改造（无兼容层）
- LlmScoreRule._call_llm: requests.post → httpx.AsyncClient，彻底消除事件循环阻塞
- engine._save_rule_results: rule.evaluate() → await rule.evaluate()
### 工具函数去重（DEBT-2）
- 新建 agenteval/utils/llm.py，统一三个函数：
- extract_reply_text (原 5 处重复)
- extract_content_from_llm_response (原 2 处重复)
- parse_json_from_llm_text (统一 LLM 输出 JSON 解析)
- engine.py / llm_score.py / runs.py / report.py 全部切换到 utils.llm
### HTTP 通用通道（S1-3）
- 新建 channels/http.py (HttpChannel)
- 配置化 send_url / reply_url 模板 ({message}, {msg_id} 占位)
- dot-path 提取 msg_id 和 reply_text
- 可选 reply_ready_path 就绪标志
- 长连接 AsyncClient 复用
- ChannelFactory 注册 ChannelType.HTTP → HttpChannel
### 测试
- 新增 tests/unit/test_http_channel_and_rules.py (19 个测试)
- _get_path / health_check / send / poll_reply / 超时 / 就绪标志 / async 规则评估
- 测试总数：24 → 43，全部通过
Co-Authored-By: Claude <noreply@anthropic.com>
37 lines
1.3 KiB
Python
37 lines
1.3 KiB
Python
"""Response time evaluation rule."""
|
|
|
|
from agenteval.evaluation.rules.base import EvalRule, RuleResult, register_rule
|
|
from agenteval.models import Case, Turn
|
|
|
|
|
|
@register_rule
|
|
class ResponseTimeRule(EvalRule):
|
|
"""Check whether the reply latency is within the configured threshold."""
|
|
|
|
name = "response_time"
|
|
|
|
async def evaluate(self, case: Case, dialog: list[Turn]) -> RuleResult:
|
|
if not dialog:
|
|
return RuleResult(passed=False, reason="无回复记录")
|
|
|
|
threshold_ms = self.params.get("max_ms")
|
|
if threshold_ms is None:
|
|
threshold_ms = case.expectations.response_time_max_ms
|
|
if threshold_ms is None:
|
|
return RuleResult(passed=True, reason="未配置响应时间阈值")
|
|
|
|
last_turn = dialog[-1]
|
|
latency = last_turn.latency_ms
|
|
if latency is None:
|
|
return RuleResult(passed=False, reason="无法获取响应时间")
|
|
|
|
if latency > threshold_ms:
|
|
return RuleResult(
|
|
passed=False,
|
|
score=max(0.0, 1.0 - (latency - threshold_ms) / threshold_ms),
|
|
reason=f"响应时间 {latency}ms 超过阈值 {threshold_ms}ms",
|
|
)
|
|
|
|
score = 1.0 if latency <= 0 else min(1.0, threshold_ms / latency)
|
|
return RuleResult(passed=True, score=score, reason=f"响应时间 {latency}ms 通过")
|