Introduction to Agents
Traditional LLMs like GPT & Llama operate in a stateless manner. This means that the output received for a query, is solely based on the prompt given to the LLM layer. The layer does not have any contextual awareness or memory. Consequently, this limits LLM layers to handle complex, robust & multi- reasoning tasks or adapting to the system positioned in. This is where agents come handy!
An agent can be understood as a simple application that aims to achieve a pre-defined goal. Generally, agents are assisted with ‘tools’ to help achieve the goal. A tool can range from a simple web search to a complex code interpreter, depending on the use case.
Agents are autonomous and when proper goals are defined they act independently of human intervention. They hold the capacity to reason & decide logically the next set(s) of steps to achieve the goal.
Agentic Frameworks
An agentic framework is a structured system designed to enable AI agents to function effectively by integrating reasoning, decision-making, and tool usage. These frameworks provide various mechanisms for defining agent objectives, managing interactions between agents and tools, and optimizing workflows for efficiency and accuracy. Moreover, they ensure that agents can autonomously break down tasks, adapt to new information, and refine their approach dynamically.
There are many agentic frameworks, each with its own strengths and weaknesses. Some commonly mentioned ones include Autogen and CrewAI. Based on our experience, LangGraph, a library under the LangChain framework, is the best option. It has a stable and growing community adopting the framework and strikes the right balance between low-level customizations and higher-level abstractions, and will therefore be the focus of this article.
LangGraph is designed to create a stateful, multi-agent system using LLM(s). It enables the creation of directed graphs (graphs in which edges have a direction) where each node represents a task, such as an interaction with an LLM or retrieving information from a tool. The connections between nodes define how they communicate and the order in which tasks are executed.
In LangGraph, a ‘persistence layer’ can be added, that ensures that the framework remembers past interactions like a conversation, and it also supports human involvement by allowing pauses for feedback. As information moves through the graph, the framework updates the state of the graph, which can include conversation history, context, and other relevant details.
While not mandatory, LangGraph integrates with LangChain for building agents seamlessly. On top of this, LangSmith can also be used for effective tracing of these agentic applications. LangGraph also comes with LangGraph Platform which is an infrastructure for deploying these agents and applications to production. These LangGraph applications can be visualized and debugged in the LangGraph Studio desktop app.
Importance of Prompt Optimization
These multi-agent systems can be built to do many complex tasks such as, Chatbots, RAG systems, code assistants, and many more. LangGraph provides a variety of agentic architectures namely, multi-agent systems, planning agents and reflection & critique systems. To demonstrate a simple use case, we will be using a multi-agent supervisor system, where a supervisor agent’s role is to assign sub tasks to different sub agents until a satisfactory answer is generated.
The supervisor agent and each sub agent have a system prompt of their own. While these multi-agent systems are very powerful, they are heavily dependent on the prompts they use during generation. Inefficient and crude prompts can lead to poor performance of these agents, hence the need for optimization.
Our current architecture for a chatbot system consists of the following:
• Supervisor Agent
- This agent is responsible for delegating tasks to sub agents and ending the process once all tasks are completed
• Response Generator Node
- This node generates the final response shown to the user
• Retrieval Agent
- This agent is connected to a vector database to retrieve documents similar to the query asked by the user
• Tools Agent - Web Search
- This agent can access the web to retrieve relevant documents concerned with the user query
• For our toy use case, the vector database has documents containing information about recipes and cooking instructions.
The test will be conducted between two systems, one with an unoptimized prompt and another with an optimized one.
The queries to be tested will be as follows:
• Can I cook pizzas easily at home? (Relevant to our database)
- The ideal flow should be:
- Supervisor → Retrieval Agent → Supervisor → Response Generator → Supervisor → END
• Who is the current president of the United States? (Irrelevant to our database)
- The ideal flow should be:
- Supervisor → Tool Agent → Web Search Tool → Supervisor → Response Generator → Supervisor → END
Unoptimized System:
supervisor_system_message = """
You are a world-class supervisor agent tasked with managing a conversation between the following workers:
["tool_agent", "retrieval_agent", "response_generator"]
Given the user request, you will decide which worker should act next.
Guidelines:
1. Each worker has specific tasks they can perform.
2. Analyze the user request and conversation context.
3. Decide the next appropriate worker to act, or decide if the task is complete (END).
Provide your decision clearly as one of: ["tool_agent", "retrieval_agent", "response_generator", "FINISH"].
"""
Optimized System:
supervisor_system_message = """
You are a world-class supervisor agent, who has great expertise in supervising tasks given by the user.
**Task:**
You are tasked with managing a conversation between the following workers:
["tool_agent", "retrieval_agent", "response_generator"]
### Objectives of the members:
- **retrieval_agent:** This agent retrieves relevant documents/output from a vector DB of cooking recipes.
- **tool_agent:** The tool agent is equipped with a set of tools designed to enhance the quality and accuracy of responses to user queries.
[web_search_tool]
- Information about the tools:
** web_search_tool **: The tool is used to search the web for a query
- **response_generator:** This node produces a well-structured and formatted output for the user.
### Steps to be followed:
1. When you receive a task, your primary responsibility is to analyze the task and the conversation text to determine the most appropriate next worker to invoke based on the objectives of the members defined above.
2. The flow involves **collecting relevant data from various agents (e.g., retrieval_agent, tool_agent)** before invoking the `response_generator` to generate the final output.
3. Once all subtasks have been completed and the overall task is addressed, ensure that the final response is structured and coherent.
4. Make sure a response is generated by the `response_generator` node to any query before marking the task as "FINISH".
5. After this, mark the task as "FINISH".
### Important Instructions:
- At each step, carefully analyze and delegate the appropriate worker. Always double-check before assigning tasks.
- The response_generator node must be visited at least once to ensure a response is generated.
"""
Conclusion
In both examples of the optimised prompt, it is clear that the supervisor follows the optimal path to deliver the final answer to the user. This is achieved by providing the supervisor with awareness of its environment, including the agents it interacts with, the tools available to those agents, and their respective objectives.
Additionally, the task assigned to the supervisor was more explicitly defined compared to the unoptimised prompt. The prompt maintained a structured flow and clarity, which the former version lacked, enabling a more efficient and accurate decision-making process.
Agentic systems are a powerful tool for reasoning and guiding information flow. However, to fully harness their potential, it is crucial to carefully design and optimize prompts to suit the specific use case, ensuring maximum efficiency and accuracy.
