12481cd1b8
## 核心变更
### 规则层全面异步化（DEBT-1）
- EvalRule.evaluate() 签名改为 async def，全量同步改造（无兼容层）
- LlmScoreRule._call_llm: requests.post → httpx.AsyncClient，彻底消除事件循环阻塞
- engine._save_rule_results: rule.evaluate() → await rule.evaluate()
### 工具函数去重（DEBT-2）
- 新建 agenteval/utils/llm.py，统一三个函数：
- extract_reply_text (原 5 处重复)
- extract_content_from_llm_response (原 2 处重复)
- parse_json_from_llm_text (统一 LLM 输出 JSON 解析)
- engine.py / llm_score.py / runs.py / report.py 全部切换到 utils.llm
### HTTP 通用通道（S1-3）
- 新建 channels/http.py (HttpChannel)
- 配置化 send_url / reply_url 模板 ({message}, {msg_id} 占位)
- dot-path 提取 msg_id 和 reply_text
- 可选 reply_ready_path 就绪标志
- 长连接 AsyncClient 复用
- ChannelFactory 注册 ChannelType.HTTP → HttpChannel
### 测试
- 新增 tests/unit/test_http_channel_and_rules.py (19 个测试)
- _get_path / health_check / send / poll_reply / 超时 / 就绪标志 / async 规则评估
- 测试总数：24 → 43，全部通过
Co-Authored-By: Claude <noreply@anthropic.com>
73 lines
2.3 KiB
Python
73 lines
2.3 KiB
Python
"""Shared utilities for LLM API interaction and message extraction.
|
|
|
|
Consolidates the duplicated _extract_text / _extract_reply_text pattern
|
|
(previously repeated in 5 places) and _extract_content_from_api_response
|
|
(previously duplicated in 2 places).
|
|
"""
|
|
|
|
from typing import Any
|
|
|
|
|
|
def extract_reply_text(reply: Any) -> str:
|
|
"""Extract plain text from a tutu-api reply object.
|
|
|
|
Handles: None | str | dict with msgBody.content or content key.
|
|
"""
|
|
if reply is None:
|
|
return ""
|
|
if isinstance(reply, str):
|
|
return reply
|
|
if isinstance(reply, dict):
|
|
body = reply.get("msgBody") or reply.get("content", "")
|
|
if isinstance(body, dict):
|
|
return body.get("content", "")
|
|
return str(body)
|
|
return str(reply)
|
|
|
|
|
|
def extract_content_from_llm_response(data: dict) -> str:
|
|
"""Extract text content from an LLM API response dict.
|
|
|
|
Handles:
|
|
- OpenAI format: choices[0].message.content as a plain string
|
|
- Anthropic-compatible format: choices[0].message.content as a list of
|
|
content blocks {type: "text", text: "..."} (non-text blocks are skipped)
|
|
"""
|
|
try:
|
|
content = data["choices"][0]["message"]["content"]
|
|
except (KeyError, IndexError, TypeError):
|
|
return ""
|
|
if isinstance(content, str):
|
|
return content
|
|
if isinstance(content, list):
|
|
parts = []
|
|
for block in content:
|
|
if not isinstance(block, dict):
|
|
continue
|
|
if block.get("type") == "text":
|
|
parts.append(block.get("text") or block.get("content") or "")
|
|
return "\n".join(parts)
|
|
return str(content)
|
|
|
|
|
|
def parse_json_from_llm_text(content: str) -> Any:
|
|
"""Parse JSON from LLM output, falling back to bracket-delimited substring.
|
|
|
|
Returns parsed JSON object, or raises json.JSONDecodeError if unparseable.
|
|
"""
|
|
import json
|
|
|
|
try:
|
|
return json.loads(content)
|
|
except json.JSONDecodeError:
|
|
pass
|
|
|
|
# Fallback: find the outermost JSON object or array
|
|
for open_char, close_char in [("{", "}"), ("[", "]")]:
|
|
start = content.find(open_char)
|
|
end = content.rfind(close_char)
|
|
if start != -1 and end != -1 and end > start:
|
|
return json.loads(content[start : end + 1])
|
|
|
|
raise ValueError(f"No JSON found in LLM output: {content[:200]}")
|