发布于 由admin

解析 ‘ReAct’ 逻辑框架：LLM 是如何通过“思考-行动-观察”循环解决复杂多步问题的？

各位编程领域的专家、开发者同仁，大家好！

今天，我们将深入探讨一个在大型语言模型（LLM）领域中备受瞩目的逻辑框架——ReAct。ReAct，全称为“Reasoning and Acting”（思考与行动），它为LLM解决复杂多步问题提供了一种强大且直观的范式。它使得LLM不再仅仅是一个文本生成器，而能真正成为一个具备规划、执行和自我修正能力的智能代理。我们将以一个编程专家的视角，剖析其核心逻辑，并通过丰富的代码示例，理解LLM是如何通过“思考-行动-观察”循环，步步为营地解决现实世界中的挑战。

1. ReAct：LLM与现实世界的桥梁

大型语言模型在文本生成、摘要、翻译等任务上展现了惊人的能力。然而，它们在处理需要精确计算、实时信息查询、外部工具交互或长链式逻辑推理的复杂问题时，往往暴露出局限性：

  • 幻觉（Hallucination）： 模型可能编造事实，尤其是在知识库之外。
  • 缺乏实时性： 模型的知识截止于训练数据，无法获取最新信息。
  • 无法执行外部操作： 模型本身无法进行数学计算、代码执行或调用API。
  • 多步推理困难： 在需要分解问题、逐步解决并整合结果的场景中，纯文本生成模式难以胜任。

ReAct框架正是为了克服这些局限而生。它将LLM的强大推理能力与外部工具的精确执行能力相结合，形成一个闭环系统。其核心思想是让LLM不仅能够“思考”——进行内部推理和规划，还能够“行动”——调用外部工具，并通过“观察”——接收工具的执行结果，来指导下一步的思考和行动。这就像一个人类工程师解决问题：先思考（规划），然后使用工具（编程、搜索），再根据工具的反馈（编译错误、运行结果）调整思路，直至问题解决。

2. “思考-行动-观察”循环：ReAct的核心机制

ReAct框架的核心在于其迭代的“思考-行动-观察”（Thought-Action-Observation, T-A-O）循环。每一次循环，LLM都将扮演一个更智能的角色，不仅仅是回答问题，更是主动解决问题。

2.1 思考 (Thought)

“思考”是LLM的内部独白，是其进行推理、规划和问题分解的阶段。在这个阶段，LLM会：

  • 理解当前任务： 明确问题的目标和约束。
  • 规划下一步行动： 基于当前状态和可用的工具，决定最合理的下一步。
  • 分解复杂问题： 将一个大问题拆解成更小的、可管理的子问题。
  • 自我反思与修正： 根据之前的行动和观察结果，调整策略或纠正错误。

在实际实现中，“思考”通常表现为LLM生成的一段自然语言文本，它详细阐述了LLM的推理过程。这是ReAct框架中最具“智能”的部分，它使得整个过程变得可解释和可追溯。

Prompt工程的关键： 为了引导LLM进行有效的“思考”，我们需要在Prompt中明确指示它表达其思考过程。例如，使用“Thought:”作为前缀。

2.2 行动 (Action)

“行动”是LLM将内部思考转化为外部操作的阶段。在这个阶段，LLM会：

  • 选择工具： 根据当前的“思考”和任务需求，从一系列可用工具中选择一个最合适的。
  • 构造参数： 根据工具的API签名和当前上下文，生成调用工具所需的精确参数。
  • 执行工具： 将构造好的工具调用指令传递给外部执行器。

外部工具可以是任何能够与LLM交互并执行特定任务的模块，例如：

  • 搜索引擎API： 获取实时网络信息。
  • 计算器： 进行精确的数学运算。
  • 代码解释器： 执行编程语言代码，验证逻辑或处理数据。
  • 数据库查询工具： 检索或修改结构化数据。
  • 自定义API： 与特定业务系统交互。

Prompt工程的关键： 我们需要向LLM清晰地描述每个可用工具的功能、名称以及它们接受的参数。LLM需要被训练或被引导去以特定格式（如Action: tool_name 和 Action Input: arguments）输出工具调用指令，以便我们的系统能够解析并执行。

2.3 观察 (Observation)

“观察”是外部工具执行结果的反馈。在这个阶段，系统会将工具的输出（无论是成功结果、错误消息还是空值）捕获，并作为新的信息反馈给LLM。LLM会：

  • 解析观察结果： 理解工具返回的数据结构和内容。
  • 评估行动效果： 判断行动是否成功，是否达到了预期目标。
  • 更新内部状态： 将观察结果整合到其对当前问题的理解中。
  • 为下一步思考提供依据： 观察结果直接影响LLM在下一轮循环中的“思考”内容。

Prompt工程的关键： 我们需要将“Observation:”作为前缀，将工具的实际执行结果直接附加到LLM的上下文历史中，以便LLM能够读取并理解。

3. ReAct框架的详细机制与Prompt工程

要成功实现ReAct，精心设计的Prompt和鲁棒的解析逻辑至关重要。

3.1 Prompt结构设计

一个典型的ReAct Prompt会包含以下几个关键部分：

  1. 系统消息 (System Message): 定义LLM的角色、行为准则和整体目标。
  2. 可用工具描述 (Tools Description): 列出所有LLM可以调用的工具，包括它们的名称、详细功能描述和参数说明。这是LLM选择和使用工具的基础。
  3. 任务描述 (Task Description): 明确用户请求解决的问题。
  4. 示例 (Few-shot Examples): 最关键的部分，通过提供几个完整的“思考-行动-观察”循环的示例，向LLM展示ReAct的工作流程和期望的输出格式。这些示例是LLM学习如何进行推理和工具调用的“教程”。
  5. 当前上下文/历史 (Current Context/History): 包含之前所有的“思考-行动-观察”记录，让LLM了解当前进展。
  6. 指令 (Instruction): 明确告诉LLM如何继续，例如“请按照Thought, Action, Action Input, Observation的格式继续，直到给出最终答案。”

示例 Prompt 结构概览：

You are an AI assistant designed to solve complex problems by thinking step-by-step and using external tools.
You have access to the following tools:

[TOOLS_DESCRIPTION]

The user wants you to solve the following problem:
[USER_TASK]

To solve the problem, please use the following format:

Thought: You should always think about what to do.
Action: The action to take, should be one of the tools listed above.
Action Input: The input to the action.
Observation: The result of the action.
... (this Thought/Action/Action Input/Observation can repeat N times)
Thought: I have now solved the problem.
Final Answer: The final answer to the original question.

Here are some examples of how to interact:

Example 1:
User: What is the capital of France and what is its population?
Thought: The user is asking for two pieces of information: the capital of France and its population. I should first find the capital of France, then find its population. I can use the `search_web` tool for this.
Action: search_web
Action Input: capital of France
Observation: Paris
Thought: I have found that the capital of France is Paris. Now I need to find the population of Paris.
Action: search_web
Action Input: population of Paris
Observation: Paris has a population of approximately 2.1 million people.
Thought: I have found both pieces of information. I can now provide the final answer.
Final Answer: The capital of France is Paris, and its population is approximately 2.1 million people.

Example 2:
User: What is 123 * 456 + 789?
Thought: The user is asking for a mathematical calculation. I should use the `calculator` tool to perform this calculation.
Action: calculator
Action Input: 123 * 456 + 789
Observation: 56987
Thought: I have performed the calculation. I can now provide the final answer.
Final Answer: 123 * 456 + 789 = 56987.

Now, let's solve the original problem:
User: [USER_TASK]
[CURRENT_HISTORY_OF_T_A_O]
Thought:

3.2 工具定义与暴露

在Python中，我们可以将工具定义为函数，并为它们提供详细的描述和参数签名。

import json

class Tool:
    def __init__(self, name: str, description: str, func, input_schema: dict):
        self.name = name
        self.description = description
        self.func = func
        self.input_schema = input_schema # For structured input validation

    def __call__(self, *args, **kwargs):
        return self.func(*args, **kwargs)

# 示例工具：搜索引擎
def search_web_func(query: str) -> str:
    """
    Performs a web search for the given query and returns the top results.
    """
    print(f"[DEBUG] Executing search_web with query: '{query}'")
    # 模拟网络搜索结果
    if "World Series 2022 winner" in query:
        return "The Houston Astros won the World Series in 2022."
    elif "Python ReAct framework" in query:
        return "The ReAct framework combines reasoning and acting in LLMs. It was introduced in the paper 'ReAct: Synergizing Reasoning and Acting in Language Models' by Yao et al. (2022). It uses a Thought-Action-Observation loop."
    elif "capital of France" in query:
        return "Paris is the capital of France."
    elif "population of Paris" in query:
        return "The population of Paris is approximately 2.1 million people as of 2023."
    else:
        return f"No specific information found for '{query}'."

search_web_tool = Tool(
    name="search_web",
    description="Useful for answering questions about current events, facts, or anything that requires up-to-date information from the internet.",
    func=search_web_func,
    input_schema={"query": {"type": "string", "description": "The search query"}}
)

# 示例工具：计算器
def calculator_func(expression: str) -> str:
    """
    Evaluates a mathematical expression and returns the result.
    Example: '1 + 2 * 3' -> '7'
    """
    print(f"[DEBUG] Executing calculator with expression: '{expression}'")
    try:
        # 使用eval存在安全风险，生产环境应使用更安全的数学表达式解析器
        result = str(eval(expression))
        return result
    except Exception as e:
        return f"Error evaluating expression: {e}"

calculator_tool = Tool(
    name="calculator",
    description="Useful for performing mathematical calculations. Input should be a valid mathematical expression.",
    func=calculator_func,
    input_schema={"expression": {"type": "string", "description": "The mathematical expression to evaluate"}}
)

# 示例工具：Python代码解释器
def python_interpreter_func(code: str) -> str:
    """
    Executes Python code and returns the standard output.
    Useful for complex logic, data manipulation, or testing code snippets.
    """
    print(f"[DEBUG] Executing python_interpreter with code:n{code}")
    import io
    import sys
    old_stdout = sys.stdout
    redirected_output = io.StringIO()
    sys.stdout = redirected_output
    try:
        exec(code)
        output = redirected_output.getvalue()
        return output if output else "Execution successful, no output."
    except Exception as e:
        return f"Error executing Python code: {e}"
    finally:
        sys.stdout = old_stdout

python_interpreter_tool = Tool(
    name="python_interpreter",
    description="Executes Python code in a sandboxed environment and returns the output. Use 'print()' for output.",
    func=python_interpreter_func,
    input_schema={"code": {"type": "string", "description": "The Python code to execute"}}
)

AVAILABLE_TOOLS = {
    search_web_tool.name: search_web_tool,
    calculator_tool.name: calculator_tool,
    python_interpreter_tool.name: python_interpreter_tool
}

def get_tools_description_for_prompt(tools: dict[str, Tool]) -> str:
    """Generates the tool description string for the LLM prompt."""
    description_parts = []
    for tool_name, tool_obj in tools.items():
        description_parts.append(f"Tool Name: {tool_obj.name}nDescription: {tool_obj.description}nInput Schema: {json.dumps(tool_obj.input_schema, indent=2)}n")
    return "n".join(description_parts)

get_tools_description_for_prompt 函数将生成一个适合直接插入Prompt的工具描述字符串。

3.3 LLM输出解析

LLM的输出需要被精确解析，以提取Thought、Action、Action Input和Final Answer。这通常通过正则表达式或更复杂的基于状态的解析器来实现。

import re

def parse_llm_output(llm_output: str):
    """Parses the LLM's output to extract Thought, Action, Action Input, or Final Answer."""
    thought_match = re.search(r"Thought: (.*?)n", llm_output, re.DOTALL)
    action_match = re.search(r"Action: (.*?)nAction Input: (.*?)n", llm_output, re.DOTALL)
    final_answer_match = re.search(r"Final Answer: (.*?)$", llm_output, re.DOTALL)

    if final_answer_match:
        return {"type": "final_answer", "answer": final_answer_match.group(1).strip()}
    elif action_match:
        return {
            "type": "action",
            "thought": thought_match.group(1).strip() if thought_match else "",
            "action": action_match.group(1).strip(),
            "action_input": action_match.group(2).strip()
        }
    elif thought_match: # If only thought is present, but no action or final answer yet
        return {"type": "thought", "thought": thought_match.group(1).strip()}
    else:
        # This case might indicate an LLM error or deviation from format
        return {"type": "error", "message": "Could not parse LLM output. Expected Thought/Action/Final Answer format."}

4. 实现一个ReAct代理 (ReAct Agent)

现在，我们将这些组件组合起来，构建一个ReAct代理。

import os
from typing import Dict, Any, List

# 假设我们有一个LLM客户端，这里用一个模拟函数代替
# 实际项目中，这里会调用OpenAI, Anthropic, Google等模型的API
def mock_llm_inference(prompt: str, model_name: str = "gpt-4") -> str:
    """
    Mock LLM inference function. In a real scenario, this would call an actual LLM API.
    It tries to simulate ReAct behavior based on the prompt content for demonstration.
    """
    print(f"n--- LLM Call (Model: {model_name}) ---")
    print("Prompt Snippet (first 500 chars):n", prompt[:500], "...")
    print("-------------------------------------")

    # This is a very simplistic simulation. A real LLM would generate this dynamically.
    if "What is 123 * 456 + 789?" in prompt:
        return """Thought: The user is asking for a mathematical calculation. I should use the `calculator` tool to perform this calculation.
Action: calculator
Action Input: 123 * 456 + 789"""
    elif "World Series 2022 winner" in prompt and "search_web" in prompt:
        if "The Houston Astros won the World Series in 2022." in prompt: # After first observation
            return """Thought: I have found the winner of the World Series 2022. I can now provide the final answer.
Final Answer: The Houston Astros won the World Series in 2022."""
        else:
            return """Thought: The user is asking for the winner of the World Series in 2022. I should use the `search_web` tool to find this information.
Action: search_web
Action Input: World Series 2022 winner"""
    elif "Write a Python function to reverse a string and test it." in prompt:
        if "def reverse_string(s):" in prompt and "test_reverse_string" in prompt:
            return """Thought: The user asked to write a Python function to reverse a string and test it. I have already provided the code and executed it. The test results show that the function works correctly. I can now provide the final answer including the code and test output.
Final Answer: Here is the Python function to reverse a string and its test:
```python
def reverse_string(s):
    return s[::-1]

# Test the function
test_cases = [
    ("hello", "olleh"),
    ("Python", "nohtyP"),
    ("", ""),
    ("a", "a")
]

print("Testing reverse_string function:")
for input_str, expected_output in test_cases:
    actual_output = reverse_string(input_str)
    print(f"Input: '{input_str}', Expected: '{expected_output}', Got: '{actual_output}', Pass: {actual_output == expected_output}")

Test output:
Testing reverse_string function:
Input: ‘hello’, Expected: ‘olleh’, Got: ‘olleh’, Pass: True
Input: ‘Python’, Expected: ‘nohtyP’, Got: ‘nohtyP’, Pass: True
Input: ”, Expected: ”, Got: ”, Pass: True
Input: ‘a’, Expected: ‘a’, Got: ‘a’, Pass: True"""
else:
return """Thought: The user wants a Python function to reverse a string and a test for it. I should use the python_interpreter tool to write and test the code.
Action: python_interpreter
Action Input:
def reverse_string(s):
return s[::-1]

Test the function

test_cases = [
("hello", "olleh"),
("Python", "nohtyP"),
("", ""),
("a", "a")
]

print("Testing reverse_string function:")
for input_str, expected_output in test_cases:
actual_output = reverse_string(input_str)
print(f"Input: ‘{input_str}’, Expected: ‘{expected_output}’, Got: ‘{actual_output}’, Pass: {actual_output == expected_output}")
"""
elif "capital of France" in prompt and "search_web" in prompt:
if "Paris" in prompt and "population of Paris" in prompt:
return """Thought: I have found both the capital of France and its population. I can now provide the final answer.
Final Answer: The capital of France is Paris, and its population is approximately 2.1 million people as of 2023."""
elif "Paris" in prompt:
return """Thought: I have found that the capital of France is Paris. Now I need to find the population of Paris.
Action: search_web
Action Input: population of Paris"""
else:
return """Thought: The user is asking for two pieces of information: the capital of France and its population. I should first find the capital of France, then find its population. I can use the search_web tool for this.
Action: search_web
Action Input: capital of France"""

# Fallback for unexpected inputs or final answer if LLM thinks it's done
if "Final Answer:" in prompt:
    return prompt.split("Final Answer:", 1)[1].strip()
return "Thought: I am unable to proceed with this query given my current simulated capabilities. Please try a different query."

class ReActAgent:
def init(self, llm_inference_func, tools: Dict[str, Tool], max_iterations: int = 10):
self.llm_inference_func = llm_inference_func
self.tools = tools
self.max_iterations = max_iterations
self.conversation_history: List[str] = []

def _build_prompt(self, user_task: str) -> str:
    """Constructs the full prompt for the LLM."""
    system_message = "You are an AI assistant designed to solve complex problems by thinking step-by-step and using external tools."

    tools_description = get_tools_description_for_prompt(self.tools)

    prompt_template = f"""{system_message}

You have access to the following tools:

{tools_description}

The user wants you to solve the following problem:
{user_task}

To solve the problem, please use the following format:

Thought: You should always think about what to do.
Action: The action to take, should be one of the tools listed above.
Action Input: The input to the action.
Observation: The result of the action.
… (this Thought/Action/Action Input/Observation can repeat N times)
Thought: I have now solved the problem.
Final Answer: The final answer to the original question.

Here are some examples of how to interact:

Example 1:
User: What is the capital of France and what is its population?
Thought: The user is asking for two pieces of information: the capital of France and its population. I should first find the capital of France, then find its population. I can use the search_web tool for this.
Action: search_web
Action Input: capital of France
Observation: Paris
Thought: I have found that the capital of France is Paris. Now I need to find the population of Paris.
Action: search_web
Action Input: population of Paris
Observation: Paris has a population of approximately 2.1 million people as of 2023.
Thought: I have found both pieces of information. I can now provide the final answer.
Final Answer: The capital of France is Paris, and its population is approximately 2.1 million people.

Example 2:
User: What is 123 456 + 789?
Thought: The user is asking for a mathematical calculation. I should use the calculator tool to perform this calculation.
Action: calculator
Action Input: 123 456 + 789
Observation: 56987
Thought: I have performed the calculation. I can now provide the final answer.
Final Answer: 123 * 456 + 789 = 56987.

Now, let’s solve the original problem:
User: {user_task}
{os.linesep.join(self.conversation_history)}
Thought:
"""
return prompt_template

def run(self, user_task: str) -> str:
    self.conversation_history = [] # Reset history for a new run
    for i in range(self.max_iterations):
        current_prompt = self._build_prompt(user_task)
        llm_output = self.llm_inference_func(current_prompt)
        parsed_output = parse_llm_output(llm_output)

        print(f"n--- Iteration {i+1} ---")
        if parsed_output["type"] == "final_answer":
            print(f"Thought: {self.conversation_history[-1].replace('Thought: ', '') if self.conversation_history and 'Thought: ' in self.conversation_history[-1] else ''}") # Print last thought if available
            print(f"Final Answer: {parsed_output['answer']}")
            return parsed_output["answer"]
        elif parsed_output["type"] == "action":
            thought = parsed_output["thought"]
            action_name = parsed_output["action"]
            action_input = parsed_output["action_input"]

            self.conversation_history.append(f"Thought: {thought}")
            self.conversation_history.append(f"Action: {action_name}")
            self.conversation_history.append(f"Action Input: {action_input}")

            print(f"Thought: {thought}")
            print(f"Action: {action_name}")
            print(f"Action Input: {action_input}")

            if action_name in self.tools:
                tool_obj = self.tools[action_name]
                try:
                    # Attempt to parse action_input as JSON if tool expects structured input
                    if tool_obj.input_schema and 'object' in tool_obj.input_schema.get('type', ''):
                        args = json.loads(action_input)
                        observation = tool_obj(**args)
                    else: # Assume string input
                        observation = tool_obj(action_input)
                except json.JSONDecodeError:
                    observation = f"Error: Action Input '{action_input}' is not valid JSON for tool '{action_name}'. Please provide valid JSON."
                except Exception as e:
                    observation = f"Error executing tool '{action_name}': {e}"
            else:
                observation = f"Error: Unknown tool '{action_name}'."

            self.conversation_history.append(f"Observation: {observation}")
            print(f"Observation: {observation}")
        else:
            print(f"LLM output parsing error: {parsed_output.get('message', llm_output)}")
            self.conversation_history.append(f"Observation: LLM output parsing error or malformed output. Attempting to recover...")
            # Depending on robustness needs, one might try to re-prompt with an error message
            return f"Agent failed to parse LLM output: {llm_output}"

    print(f"nMax iterations ({self.max_iterations}) reached without a final answer.")
    return "Agent failed to find a final answer within the maximum number of iterations."

#### 4.1 示例 1: 简单的计算器代理

让我们使用这个代理来解决一个数学问题。

```python
# 初始化ReAct代理
agent = ReActAgent(llm_inference_func=mock_llm_inference, tools=AVAILABLE_TOOLS)

print("n--- Running Example 1: Simple Calculation ---")
result = agent.run("What is 123 * 456 + 789?")
print(f"nAgent Final Result: {result}")
# Expected Output: Agent Final Result: 123 * 456 + 789 = 56987.

运行轨迹分析：

  1. Prompt: 代理构建包含任务、工具描述和历史（初始为空）的Prompt。
  2. LLM Call: LLM（模拟）接收Prompt。
  3. LLM Output: LLM生成：“Thought: …计算… Action: calculator Action Input: 123 * 456 + 789”。
  4. Parse: 代理解析出Thought和Action。
  5. Execute Action: 代理调用calculator_tool，传入123 * 456 + 789。
  6. Observation: calculator_tool返回56987。
  7. Append History: Observation: 56987被添加到历史中。
  8. Loop: 代理再次构建Prompt，这次包含之前的T-A-O历史。
  9. LLM Call: LLM接收新的Prompt。
  10. LLM Output: LLM生成：“Thought: …已完成计算… Final Answer: 123 * 456 + 789 = 56987.”。
  11. Parse: 代理解析出Final Answer。
  12. Return: 代理返回最终答案。

4.2 示例 2: 网络搜索代理

获取实时信息。

print("n--- Running Example 2: Web Search ---")
result = agent.run("Who won the World Series in 2022?")
print(f"nAgent Final Result: {result}")
# Expected Output: Agent Final Result: The Houston Astros won the World Series in 2022.

运行轨迹分析：

  1. Prompt: 代理构建Prompt。
  2. LLM Output: LLM生成：“Thought: …搜索… Action: search_web Action Input: World Series 2022 winner”。
  3. Parse & Execute: 代理调用search_web_tool。
  4. Observation: search_web_tool返回“The Houston Astros won the World Series in 2022.”。
  5. Append History: Observation添加到历史。
  6. Loop: 代理再次构建Prompt。
  7. LLM Output: LLM生成：“Thought: …已找到… Final Answer: The Houston Astros won the World Series in 2022.”。
  8. Return: 返回最终答案。

4.3 示例 3: 结合工具解决多步问题

print("n--- Running Example 3: Multi-step Problem with Combined Tools ---")
result = agent.run("What is the capital of France and what is its population?")
print(f"nAgent Final Result: {result}")
# Expected Output: Agent Final Result: The capital of France is Paris, and its population is approximately 2.1 million people as of 2023.

运行轨迹分析：

  1. Prompt & Initial Thought: LLM规划，需要两步搜索。
  2. Action 1 (search_web): 查找“capital of France”。
  3. Observation 1: “Paris”。
  4. Thought 2: LLM根据观察结果，知道找到了首都，现在需要查找人口。
  5. Action 2 (search_web): 查找“population of Paris”。
  6. Observation 2: “Paris has a population of approximately 2.1 million people as of 2023.”。
  7. Thought 3 & Final Answer: LLM整合两次观察结果，给出最终答案。

4.4 示例 4: 代码解释器代理

执行并测试Python代码。

print("n--- Running Example 4: Python Interpreter ---")
result = agent.run("Write a Python function to reverse a string and test it.")
print(f"nAgent Final Result: {result}")
# Expected Output: Agent Final Result: ... (the code and test output)

运行轨迹分析：

  1. Prompt & Initial Thought: LLM规划，需要使用python_interpreter工具来编写和测试代码。
  2. Action 1 (python_interpreter): LLM生成Python代码，包括函数定义和测试用例，并指示执行。
  3. Observation 1: python_interpreter_tool执行代码并返回标准输出（测试结果）。
  4. Thought 2 & Final Answer: LLM根据观察结果，确认代码正确运行并测试通过，然后将代码和测试输出作为最终答案提供。

5. 高级概念与考量

ReAct框架虽然强大，但在实际应用中还需要考虑一些高级问题。

5.1 内存管理与上下文窗口

随着T-A-O循环的进行，conversation_history会不断增长。LLM的上下文窗口（Context Window）是有限的，过长的历史会导致：

  • 截断： 最早的历史信息被丢弃，导致LLM失去关键上下文。
  • 成本增加： 每次LLM调用传输的数据量增大。
  • 推理效率下降： LLM需要处理更多无关信息。

解决方案：

  • 历史摘要 (History Summarization): 定期让LLM或另一个摘要模型对旧的历史记录进行概括，保留关键信息。
  • 选择性召回 (Selective Recall): 使用RAG（Retrieval Augmented Generation）技术，根据当前任务从外部知识库或历史日志中检索最相关的T-A-O记录。
  • 滑动窗口 (Sliding Window): 始终只保留最近的N轮T-A-O历史。

5.2 自我修正与错误处理

ReAct自然地支持自我修正。当工具执行失败（Observation是错误消息）或结果不符合预期时，LLM会在下一轮“思考”中识别出问题，并尝试：

  • 重试： 使用相同的参数再次调用工具。
  • 修改参数： 调整工具输入，例如更换搜索关键词。
  • 更换工具： 尝试使用另一个功能相似的工具。
  • 寻求澄清： 如果错误难以理解，可能会尝试向用户提问。
  • 重新规划： 彻底改变解决问题的策略。

在Prompt中，可以明确指示LLM如何处理错误观察，例如：“如果Observation包含错误信息，请分析错误原因并尝试纠正。”

5.3 工具编排复杂性

随着可用工具数量的增加，LLM选择正确工具的难度也会增加。

  • 清晰的工具描述： 这是最基础也是最重要的。好的描述能帮助LLM理解工具的用途和限制。
  • 工具检索 (Tool Retrieval): 对于大量工具，可以使用向量数据库存储工具描述，然后根据LLM的Thought或用户查询进行语义搜索，只将最相关的少量工具提供给LLM。
  • 工具分层/组合： 将复杂任务分解为子任务，每个子任务对应一个更小的工具集。

5.4 人机协作 (Human-in-the-Loop)

在某些关键决策点或错误处理无法自动解决时，引入人工干预是必要的。

  • 确认敏感操作： 在执行可能影响实际系统的操作前，请求用户确认。
  • 错误分析协助： 当LLM无法理解某个复杂的错误信息时，将问题反馈给人类专家。
  • 最终结果审核： 在交付最终答案前，让人类审查其准确性和完整性。

5.5 评估 ReAct 代理

评估ReAct代理的性能比评估纯文本生成模型更复杂。

  • 任务成功率： 代理能否在给定限制内（如迭代次数）成功完成任务。
  • 效率： 完成任务所需的迭代次数、LLM调用次数、工具调用次数。
  • 答案质量： 最终答案的准确性、完整性和相关性。
  • 可解释性： Thought链是否逻辑清晰、易于理解。
  • 鲁棒性： 对各种输入和错误情况的处理能力。

6. ReAct 与其他方法的比较

  • Chain-of-Thought (CoT): CoT是让LLM在生成答案前先进行一步步的推理，从而提高复杂推理任务的性能。ReAct可以看作是CoT的扩展，它不仅包含内部推理（Thought），还包括外部行动（Action）和反馈（Observation），将CoT的推理能力与工具的执行能力相结合。

  • Tool-Augmented LLMs (例如 OpenAI Function Calling, LangChain Tools): 这些是实现ReAct的基础设施或库。OpenAI Function Calling 提供了一种结构化的方式让LLM生成函数调用，LangChain等框架则将工具抽象化并提供代理执行环境。ReAct是一种逻辑框架或模式，而这些是具体的实现技术。你可以使用Function Calling来让LLM生成ReAct中的Action和Action Input，然后由你的ReAct代理代码去执行这些函数。

7. ReAct 与 AI 智能体的未来

ReAct框架的出现，标志着LLM从被动的文本生成器向主动的智能代理迈出了关键一步。它使得LLM能够：

  • 突破知识边界： 通过工具获取实时和外部信息。
  • 执行复杂任务： 将多步推理与实际操作相结合。
  • 增强可靠性： 通过观察和自我修正降低幻觉和错误率。

未来，基于ReAct及其变体的AI代理将在更多领域发挥作用，例如：

  • 自动化客服： 不仅回答问题，还能查询订单、修改信息。
  • 代码开发助手： 编写、测试、调试代码，甚至与版本控制系统交互。
  • 科研辅助： 搜索文献、运行模拟、分析数据。
  • 教育领域： 提供个性化学习路径，解答复杂疑问，甚至批改作业。

ReAct及其所代表的代理范式，正在构建一个LLM能够真正理解、规划并与物理/数字世界交互的未来。

结语

ReAct框架通过“思考-行动-观察”的迭代循环，赋予了大型语言模型前所未有的问题解决能力。它将LLM的强大推理与外部工具的精确执行完美结合，为构建更智能、更自主的AI代理奠定了坚实基础。理解并掌握ReAct，对于我们编程专家而言，是打开LLM应用新篇章的关键。

发表回复

您的邮箱地址不会被公开。 必填项已用 * 标注