作者：Sravan Jayanthi
作者： Sravan Jayanthi、Joscha Koepke
自 ChatGPT 时代开启以来，我们 @ Connectly AI 一直深耕于构建销售与客户支持类对话式 AI。在生产环境中开发了多个 AI 智能体后，包括 Sofia Lite 和 Sofia V3，我们系统性地梳理出围绕幻觉与可靠性的关键痛点。这正是我们开发 Sofia QA 的起因——Connectly AI 最新的 AI 产品，它建立在生产环境中大量真实数据与经验积累的基础之上，以确保机器人具有高可靠性和高准确率。通过在产品设计与架构中投入大量精力去诊断和缓解幻觉问题，Sofia QA 已经获得了很强的采用率与用户黏性：在 2.5 万个用户会话中回答率达到 77%，客户反馈中设置体验评分为 5/5，问题解决效果和回答质量也均有提升。
在这篇博客中，我们试图精确定义我们观察到的失败模式，并阐述在设计与实现 Sofia QA 的过程中，我们用来评估和捕捉这些问题的策略。
在这一部分，我们将梳理 Sofia 机器人此前版本的主要短板，并对失败模式的性质进行分类。"幻觉"往往被当作一个过度泛化的笼统术语，用来描述各种各样的行为。我们将语言模型的幻觉定义为：模型做出了无法被验证或支持的行为或陈述。
Sofia V3 的智能体架构不受约束的特性，使其难以在从 RAG 形式的问答到基于图片的产品推荐等广泛任务中实现可靠且可预测的行为。这种缺乏约束的状态也导致生成指令和 LLM 执行策略缺乏可解释性，进而使得培训非专业人员理解机器人的能力以及如何微调其行为变得更加困难。
表 1：智能体失败模式分析
智能体失败模式分析
*上方相关链接*：AmiEnt、CLAMBER、mCoT、MAGICORE、Reflexion、PARC、HaluEval 2.0、CDQG、LLM Effective Negotiators、Hallucination Survey、PersuaBot、HiAgent、Sierra AI's Tau-bench、Claude 3.7、RuleBench
表 2：非专业用户配置难度分析
非专业用户配置智能体 AI 应用的难点
Sofia QA 的设计目标是应对上述所有主要挑战：提供基于事实、有知识依据的回答，减轻客服人员的工作负担，并提供清晰的分析数据以展示其影响。它可靠、易于设置、能够自我改进，并且不需要大量工程支持即可维护。
面向大语言模型（LLM）的归因技术旨在通过把生成文本与可验证的来源相关联，来提升事实可靠性。这类方法确保回答有据可依，从而降低产生幻觉的潜在风险。
我们探索了以下 3 种归因策略：
我们借助指标驱动开发方法，建立一个可衡量的基线数据集，用于设计和优化我们的系统。我们在一份包含 121 个西班牙语问题的客户样本数据集上进行了实验。构建这套 RAG 框架，需要针对多语言对话式 RAG 这一特定场景，找出最强的检索、引用和答案生成方法。
A. 检索质量
我们探索了 3 种检索方式：
实验 1：不同检索技术的检索质量
结论：使用重排序和过滤方法（对应的 LLM 调用）会带来检索时间的增加，这是一种权衡。最强的最终排序结果来自使用重排序器，但代价是响应时间大幅增加。
B. 引用质量
我们通过计算所选引用相对于真实引用的精确度，来衡量引用准确率。
实验 2：不同引用技术的引用精确度
结论：最精确的引用策略是引入一个归因步骤，其精确度略高于 CoT 或行内引用。值得注意的是，在我们的数据集上，带 CoT 的重排序器表现似乎不如可比的单次检索，这说明使用重排序器并未在引用质量上带来明显的下游提升。
C. 回答质量指标
我们将数据集划分为可回答（知识库中有对应答案的问题）和重定向（需要发送回退话术或转接人工客服的问题）两类。为了确定应采用哪种引用和检索技术组合，我们借助一个基于参考答案的 LLM 评判者，按以下提示词，以 1 到 4 分的量表对各技术打分：
LLM_AS_A_JUDGE_PROMPT = "Rate how well the generated response answers the user query in the conversation.
Provide reasoning in full sentences and then provide the rating.
Output just one number: 4 for strong, 3 for good, 2 for okay, or 1 for weak.
If the original response says it cannot find an answer or does not answer the user query and the generated response says {no_answer_string}, then both are in alignment and the rating should be 4.
If the conversation style is "Redirect", the generated response should say {no_answer_string} or redirect to the company contact information.
Conversation Style: {conversation_style}
Conversation: {conversation}
Assessement:
Lets think through step by step."
ONE_SHOT_RESPONSE_SYSTEM_PROMPT = "In this session, the model has access to search results and a user's question, your job is to answer the user's question using only information from the search results.
Remember to add a citation to the end of your response using markers like %[1]%, %[2]%, %[3]%, etc for the corresponding passage supports the response.
IMPORTANT: If the search results does not contain an answer to the question, you must state exactly "{default_fallback_message}".
IMPORTANT: If no passages support the response, do not cite any passages and state exactly "{default_fallback_message}"."
RESPONSE_USER_MESSAGE = "
{query}
Resource: Search Results: {search_results}"
COT_RESPONSE_USER_MESSAGE_1 = "
{query}
Think step-by-step."
COT_RESPONSE_USER_MESSAGE_2 = "
{query}
Please follow these steps:
1. {{Step 1}}
2. {{Step 2}}
…"
COT_RESPONSE_USER_MESSAGE_3 = "
{query}
Please follow these steps:
1. {{Step 1}}
2. {{Step 2}}
…
{{ User query}}
Think step by step first and then answer. Follow below format when responding
Response Schema:
<thinking>
( your thinking goes here )
</thinking>
<answer>
( your answer goes here )
</answer>"
COT_REFLECT_PROMPT = "You are an advanced reasoning agent that can improve based on self refection. You will be given a previous reasoning trial in which you were given a question to answer. You were unsuccessful in answering the question either because you guessed the wrong answer with Finish[<answer>] or there is a phrasing discrepancy with your provided answer and the answer key. In a few sentences, Diagnose a possible reason for failure or phrasing discrepancy and devise a new, concise, high level plan that aims to mitigate the same failure. Use complete sentences.
Previous trial:
{context}
Question: {question}
Reflection:"
COT_SIMPLE_AGENT_REFLECT_INSTRUCTION = "Solve a question answering task by having a Thought, then Finish with your answer. Thought can reason about the current situation. Finish[answer] returns the answer and finishes the task.
{context}
{reflections}
Question: {question}"
**【预过滤】LLM 重排序或过滤：**该提示词的目标是调整顺序，使最相关的上下文排在最前面。它也可以剔除不相关的上下文，从而减少发送给 LLM 的干扰信息。
LLM_RERANKING_PROMPT = "Reorder the following list of passages based on how well they answer the user's query.
Include the ID of the passage in the output along with a reason for its ordering. Make sure all passages are included, do not exclude any. Delimit the output as a numbered list with the ID and reason. (Ex: ID: 123 | Reason: The answer is relevant to the query)
User Query: {query}
Passages: {context}
Reordered Passages:"
**【后过滤】过滤掉响应句子：**给定 LLM 提供的一个回答，可以使用 LLM 过滤掉未在原文段落中得到支持的内容。
ATTRIBUTION_RESPONSE_CITATION_PROMPT = "Add in-line text citations for each of the given statements in <statement> using the content in <response_context>. Remove any statements not supported by the content in <response_context>.
The in-line text citation should use markers like %[1]%, %[2]%, %[3]%, etc for the corresponding passage that supports the response.
If the response_context does not contain information that supports any of the statements, you must state exactly "{default_fallback_message}".
<response_context>
{response_context}
</response_context>
<statement>
{statement}
</statement>
Filtered Statement with Citations: "
**波束搜索解码：**将解码方式从贪婪解码改为波束搜索，能显著提升输出质量。
将温度设为 0：将温度设为 0，可确保 LLM 行为具有确定性。
降低长度惩罚：如果生成内容过长或过短，可以调整这一参数，通常更短的回答会带来更准确的生成结果。
