Advent Calendar Day 4: How AI Agents Handle Multiple Product Requests in One Query

This December, I'm continuing to show how naive chatbots struggle with complex shopping requests.

We've seen them fail at clarifying questions, context shifts, and numerical requirements.

Today, let's see how they fail when a customer asks for more than one product in a single message. A naive chatbot will often treat the entire query as one block of text and return irrelevant results. An AI agent, however, can break the request down and return results for each item accurately.

Source Code

For a deeper dive into this example, check out my GitHub repository for a detailed, step-by-step implementation in Jupyter Notebooks, breaking down the code and methodology used by the AI agent in today's scenario.

Introducing SoleMates

SoleMates is our fictional online shoe store we'll use to illustrate these concepts:

SoleMates is our fictional online shoe store

Today's Challenge: Multiple Requests in One Query

Scenario

A customer asks our SoleMates chatbot:

Customer: "I need men's black formal shoes and women's red flats":

A customer initiates a chat with SoleMates and asks about men's black formal shoes and women's red flats

Naive Chatbot Response

The naive chatbot takes this combined query, turns it into a single vector, and searches once.

Because it's looking for something that matches the entire request as one phrase, it often fails to return correct products for both categories:

The naive chatbot takes this combined query, turns it into a single vector, and searches once

Instead of splitting the query into two separate searches (one for men's black formal shoes and one for women's red flats), the naive chatbot ends up failing to find matching products for both groups:

The naive chatbot ends up failing to find matching products for both groups

Naive Chatbot Says: "There are no men's black formal shoes or women's red flats available":

The naive chatbot ends replying: "There are no men's black formal shoes or women's red flats available"

But we know from our product database that these items do exist. The naive chatbot just isn't handling the multiple requests properly. It treats the whole sentence as one single, overly complex vector and fails.

Why Did the Naive Chatbot Fail?

• No Query Decomposition: Doesn't split the user's message into separate queries
• All-or-Nothing Matching: It searches for one big match, ignoring that the customer asked for two different product types
• Inflexible Search: The vector search doesn't know how to handle multiple requests in one prompt

Limitations Highlighted

• No Multi-Intent Handling: The naive chatbot can't break down the user request into multiple searches
• No Flexible Reasoning: It doesn't plan multiple steps or understand that the user wants two different categories of products

AI Agent Solution

The AI agent takes the user request and identifies that there are actually two distinct searches required:

1. Search for "men's black formal shoes"
2. Search for "women's red flats"

The agent then performs two separate metadata-filtered vector database searches, one for each product type.

This way, the agent ensures that it returns accurate matches for both requests:

The agent then performs two separate metadata-filtered vector database searches, one for each product type

Agent's Interaction:

Customer: "I need men's black formal shoes and women's red flats"

Agent: "Understood. Let me find men's black formal shoes first, and then I'll search for women's red flats"

1. The agent applies a metadata filter for men's black formal shoes
2. The agent then applies a metadata filter for women's red flats

Finally, the agent returns a list of both men's black formal shoes and women's red flats:

The agent returns a list of both men's black formal shoes and women's red flats

How Did the AI Agent Succeed?

• Query Decomposition: Splits the single user query into multiple targeted searches
• Metadata Filtering: Uses product attributes (gender, color, usage) to find matches
• Step-by-Step Reasoning: Executes multiple search steps instead of a single combined query

Key Takeaways

Naive Chatbot Limitation

• Treats multiple requests as one complex query and fails to find correct matches for each segment

AI Agent Advantages

• Handles Multiple Requests: Breaks down the user's request into several parts
• Accurate Product Returns: Finds relevant items for each part of the request
• Flexible, Step-by-Step Approach: Queries the vector database multiple times as needed

Conclusion

By handling multiple requests in a single query, the AI agent shows how flexible and powerful it can be compared to a naive chatbot. Instead of failing at the complexity, it adapts by splitting the request and delivering accurate results for each category.

This scenario highlights yet another way that AI agents excel over naive chatbots. Stay tuned for tomorrow's example in this December series.

About This Series

Each day in December, I show a different scenario where naive chatbots fall short and how AI agents overcome those limitations. By following along, you'll learn what makes AI agents more versatile and reliable. If you're interested in building your own advanced chatbot, read on.

Learn to Build Your Own AI Agent Chatbot

I'm preparing a course to teach you how to build and deploy your own AI agent chatbot. Sign up here!

---

Behind the Scenes: Code Snippets

Here's a simplified illustration of how the AI agent processes the query.

We're giving the agent access to two tools it can use freely:

1. Vector database metadata filtering
2. Vector database query

1. Vector database metadata filtering tool

This tool lets the agent break down user queries into multiple filters.

• For "men's black formal shoes," it creates filters for gender = men, color = black, and usage = formal
• For "women's red flats," it creates filters for gender = women and color = red

def create_metadata_filter(filter_string):
    # Example filter string: "men's black formal shoes"
    # Parses and returns:
    # [{"key": "gender", "value": "men", "operator": "=="}, {"key": "color", "value": "black", "operator": "=="}, {"key": "usage", "value": "formal", "operator": "=="}]

    # Then another filter string: "women's red flats"
    # Parses and returns:
    # [{"key": "gender", "value": "women", "operator": "=="}, {"key": "color", "value": "red", "operator": "=="}]

    filters = parse_filters(filter_string)
    return filters

2. Vector database query

The agent runs the query once for men's black formal shoes, then again for women's red flats. This two-step approach ensures accurate matches for each request:

def search_footwear_database(query_str, filters_json):
    embedded_query = embed_query_aws_titan(query_str)
    results = vector_db.search(embedded_query, filters=filters_json)
    return results

AWS Titan Multimodal Embeddings in Action

I use AWS Titan, a multimodal embedding model that converts both product texts and images into vectors, integrated as the function embed_query_aws_titan into the AI agent tool search_footwear_database.

This means the AI agent can process a query like "red heels" and match it to not only product descriptions but also actual images of red heels in the database.

By combining text and image data, the model helps the AI agent provide more relevant and visually aligned recommendations based on the customer's input.

Agent workflow

Construct agent worker with access to the two tools and initiate the agent:

from llama_index.core.agent import AgentRunner, FunctionCallingAgentWorker

agent_worker = FunctionCallingAgentWorker.from_tools(
    [
        create_metadata_filters_tool,
        query_vector_database_tool,
    ])

agent = AgentRunner(
    agent_worker,
)

Initiate conversation with agent:

agent.chat("I need men's black formal shoes and women's red flats")

Agent:

1. Calls create_metadata_filter("men's black formal shoes")
2. Returns filters:

[
    {"key": "gender", "value": "men", "operator": "=="}, 
    {"key": "color", "value": "black", "operator": "=="}, 
    {"key": "usage", "value": "formal", "operator": "=="}
]

3. Queries vector DB for men's black formal shoes → finds matches
4. Calls create_metadata_filter("women's red flats")
5. Returns filters:

[
    {"key": "gender", "value": "women", "operator": "=="}, 
    {"key": "color", "value": "red", "operator": "=="}
]

6. Queries vector DB for women's red flats → finds matches
7. Returns both sets of results to the user

The agent decided to apply two separate filters and query the vector database twice to get the results

Here's the full agent output:

Added user message to memory: I need men's black formal shoes and women's red flats
=== Calling Function ===
Calling function: create_metadata_filter with args: {"filter_string": "men's black formal shoes"}
=== Function Output ===
[{"key": "gender", "value": "men", "operator": "=="}, {"key": "color", "value": "black", "operator": "=="}, {"key": "usage", "value": "formal", "operator": "=="}]
=== Calling Function ===
Calling function: create_metadata_filter with args: {"filter_string": "women's red flats"}
=== Function Output ===
[{"key": "gender", "value": "women", "operator": "=="}, {"key": "color", "value": "red", "operator": "=="}]
=== Calling Function ===
Calling function: search_footwear_database with args: {"query_str": "shoes", "filters_json": [{"key": "gender", "value": "men", "operator": "=="}, {"key": "color", "value": "black", "operator": "=="}, {"key": "usage", "value": "formal", "operator": "=="}]}
=== Function Output ===
[{'color': 'black', 'text': 'Arrow men formal black shoe', 'gender': 'men', 'product_type': 'formal shoes', 'product_id': 6, 'usage': 'formal', 'price': 180, 'similarity_score': 1.18878198}, {'color': 'black', 'text': 'Lee cooper men black shoe', 'gender': 'men', 'product_type': 'formal shoes', 'product_id': 5, 'usage': 'formal', 'price': 155, 'similarity_score': 1.18653321}]
=== Calling Function ===
Calling function: search_footwear_database with args: {"query_str": "shoes", "filters_json": [{"key": "gender", "value": "women", "operator": "=="}, {"key": "color", "value": "red", "operator": "=="}]}
=== Function Output ===
[{'color': 'red', 'text': 'Catwalk women red shoes', 'gender': 'women', 'product_type': 'casual shoes', 'product_id': 54, 'usage': 'casual', 'price': 75, 'similarity_score': 1.7024461}, {'color': 'red', 'text': 'Carlton london women casual red casual shoes', 'gender': 'women', 'product_type': 'casual shoes', 'product_id': 53, 'usage': 'casual', 'price': 90, 'similarity_score': 1.67705441}, {'color': 'red', 'text': 'Catwalk women red flats', 'gender': 'women', 'product_type': 'flats', 'product_id': 51, 'usage': 'casual', 'price': 170, 'similarity_score': 1.17914069}, {'color': 'red', 'text': 'Carlton london women red & black heels', 'gender': 'women', 'product_type': 'heels', 'product_id': 57, 'usage': 'casual', 'price': 95, 'similarity_score': 1.17578256}, {'color': 'red', 'text': 'Carlton london women casual red stilettos', 'gender': 'women', 'product_type': 'heels', 'product_id': 56, 'usage': 'casual', 'price': 175, 'similarity_score': 1.17057896}, {'color': 'red', 'text': 'Clarks women silvi spice raspberry leather red flats', 'gender': 'women', 'product_type': 'flats', 'product_id': 55, 'usage': 'casual', 'price': 100, 'similarity_score': 1.16590405}]
=== LLM Response ===
Here are some options for men's black formal shoes and women's red flats:

### Men's Black Formal Shoes:
1. **Arrow Men Formal Black Shoe**
   - Price: $180

2. **Lee Cooper Men Black Shoe**
   - Price: $155

### Women's Red Flats:
1. **Catwalk Women Red Flats**
   - Price: $170

2. **Clarks Women Silvi Spice Raspberry Leather Red Flats**
   - Price: $100

If you need more information or have any other preferences, feel free to ask!

Additional Resources

For a deeper dive into this example, check out my GitHub repository where I break down the code and methodology used by the AI agent in today's scenario.

Learn to Build Your Own AI Agent Chatbot

I'm preparing a course to teach you how to build and deploy your own AI agent chatbot. Sign up here!