Advent Calendar Day 13: How AI Agents Handle Multiple Product Requests in One Query

This December, I'm highlighting how naive chatbots fail at numeric filters and how AI agents get it right.

We've seen naive chatbots fail at clarifying questions, context shifts, numerical requirements, multiple requests in one query, price filters, style suggestions, unavailable colors, negations, new color requirements, multi-color requests and special occasions.

In yesterday's issue, we explored numeric filters for lower heel heights.

Today, we look at a new scenario: a user wants "red heels" and "blue men's sneakers" in the same message.

A naive chatbot may only focus on one part of the request or return random mismatched products. An AI agent, however, will parse both requests and provide accurate matches for each.

Source Code

For a deeper dive into this example, check out my GitHub repository for a detailed, step-by-step implementation in Jupyter Notebooks, breaking down the code and methodology used by the AI agent in today's scenario.

Introducing SoleMates

SoleMates is our fictional online shoe store we'll use to illustrate these concepts:

SoleMates is our fictional online shoe store

Today's Challenge: Handling Two Product Requests at Once

Scenario

Customer: "I'm looking for red heels and blue men's sneakers":

A customer initiates a chat with SoleMates and asks about red heels and blue men's sneakers

Naive Chatbot Response

The naive chatbot vectorizes the entire query as a single text block. It might find:

• Men's sneakers, but no red heels
• Random products that partially match

The naive chatbot tries to pull shoes that match the request

It finds some blue sneakers but fails to pull red heels:

The naive chatbot only found some blue sneakers

It fails to search for women's red heels separately, leaving part of the request unsatisfied.

Naive Chatbot: "There are no red heels available. However, we have some blue men's sneakers"

The naive chatbot only found some blue sneakers

Why Did the Naive Chatbot Fail?

• Only focuses on one subset of the request
• No step-by-step approach to process each product type
• Ends up ignoring or missing key parts of the query

Limitations Highlighted

• No multi-intent parsing
• Inconsistent or incomplete results
• Cannot handle separate product categories in one conversation

AI Agent Solution

The AI agent recognizes two different product categories in a single query:

1. Red heels
2. Blue men's sneakers

It applies filters for each request separately:

• For red heels:

  • color=red
  • product_type=heels

• For blue men's sneakers:

  • color=blue
  • product_type=sneakers
  • gender=men

AI Agent: "Here are some options for red heels and blue men's sneakers":

AI agent applies filters for each request and pulls suitable products for the whole request

How Did the AI Agent Succeed?

• Splits the query into two sub-requests
• Applies filters for each request
• Returns a well-organized list of relevant products for both

Key Takeaways

Naive Chatbot Limitation

• Confuses multiple requests and ends up ignoring one

AI Agent Advantages

• Handles multi-intent queries by breaking them down
• Applies different filters for each request
• Gives users a complete set of product recommendations

Conclusion

When a single message contains multiple product requests, naive chatbots fail to handle the complexity. AI agents excel by parsing each request, applying separate filters, and returning results for both. This leads to a better user experience and higher satisfaction.

About This Series

In this series, we explore how naive chatbots fall short, and how AI agents handle real-world shopping queries. Stay tuned for tomorrow's issue, where we dive into another scenario that naive chatbots can't handle as well as AI agents.

Follow along as we continue to explore scenarios where AI agents outperform naive chatbots.

Learn to Build Your Own AI Agent Chatbot

I'm preparing a course to teach you how to build and deploy your own AI agent chatbot. Sign up here!

---

Behind the Scenes: Code Snippets

Here's a simplified illustration of how the AI agent processes the query.

We're giving the agent access to two tools it can use freely:

1. Vector database metadata filtering
2. Vector database query

1. Vector database metadata filtering tool

def create_metadata_filter(filter_string):
    # Splits and extracts relevant filters
    # "red heels" -> color=red, product_type=heels
    # "blue men's sneakers" -> color=blue, gender=men, product_type=sneakers
    # The agent calls this tool multiple times if needed
    filters = parse_filters(filter_string)
    return filters

2. Vector database query

def search_footwear_database(query_str, filters_json):
    embedded_query = embed_query_aws_titan(query_str)
    results = vector_db.search(embedded_query, filters=filters_json)
    return results

AWS Titan Multimodal Embeddings in Action

I use AWS Titan, a multimodal embedding model that converts both product texts and images into vectors, integrated as the function embed_query_aws_titan into the AI agent tool search_footwear_database.

This means the AI agent can process a query like "red heels" and match it to not only product descriptions but also actual images of red heels in the database.

By combining text and image data, the model helps the AI agent provide more relevant and visually aligned recommendations based on the customer's input.

Agent workflow

Construct agent worker with access to the two tools and initiate the agent:

from llama_index.core.agent import AgentRunner, FunctionCallingAgentWorker

agent_worker = FunctionCallingAgentWorker.from_tools(
    [
        create_metadata_filters_tool,
        query_vector_database_tool,
    ])

agent = AgentRunner(
    agent_worker,
)

Initiate conversation with agent:

agent.chat("I'm looking for red heels and blue men's sneakers")

Agent workflow:

Agent:

1. Detect multiple requests
2. Filter and search for red heels
3. Gets filter:

[
    {"key": "color", "value": "red", "operator": "=="}
]

4. Filter and search for blue men's sneakers
5. Gets filter:

[
    {"key": "gender", "value": "men", "operator": "=="}, 
    {"key": "color", "value": "blue", "operator": "=="}
]

6. Return both sets of results in one response

AI agent applies filters for each request and pulls suitable products for the whole request

Here's the full agent output:

Added user message to memory: I'm looking for red heels and blue men's sneakers
=== Calling Function ===
Calling function: create_metadata_filter with args: {"filter_string": "red heels"}
=== Function Output ===
[{"key": "color", "value": "red", "operator": "=="}]
=== Calling Function ===
Calling function: create_metadata_filter with args: {"filter_string": "blue men's sneakers"}
=== Function Output ===
[{"key": "gender", "value": "men", "operator": "=="}, {"key": "color", "value": "blue", "operator": "=="}]
=== Calling Function ===
Calling function: search_footwear_database with args: {"query_str": "heels", "filters_json": [{"key": "color", "value": "red", "operator": "=="}]}
=== Function Output ===
[{'color': 'red', 'text': 'Carlton london women red & black heels', 'gender': 'women', 'product_type': 'heels', 'product_id': 57, 'usage': 'casual', 'price': 95, 'heels_height': 2, 'similarity_score': 1.69652748}, {'color': 'red', 'text': "Nike men's mercurial gli red pink shoe", 'gender': 'men', 'product_type': 'sports shoes', 'product_id': 50, 'usage': 'sports', 'price': 100, 'heels_height': None, 'similarity_score': 1.19959509}, {'color': 'red', 'text': 'Catwalk women red shoes', 'gender': 'women', 'product_type': 'casual shoes', 'product_id': 54, 'usage': 'casual', 'price': 75, 'heels_height': None, 'similarity_score': 1.19956303}, {'color': 'red', 'text': 'Carlton london women casual red stilettos', 'gender': 'women', 'product_type': 'heels', 'product_id': 56, 'usage': 'casual', 'price': 175, 'heels_height': 3, 'similarity_score': 1.18981349}, {'color': 'red', 'text': 'Basics men red casual shoes', 'gender': 'men', 'product_type': 'casual shoes', 'product_id': 47, 'usage': 'casual', 'price': 105, 'heels_height': None, 'similarity_score': 1.18097079}, {'color': 'red', 'text': 'Nike men jordan fly wade red sports shoes', 'gender': 'men', 'product_type': 'sports shoes', 'product_id': 45, 'usage': 'sports', 'price': 55, 'heels_height': None, 'similarity_score': 1.17897511}, {'color': 'red', 'text': 'Fila men leonard red shoes', 'gender': 'men', 'product_type': 'casual shoes', 'product_id': 48, 'usage': 'casual', 'price': 90, 'heels_height': None, 'similarity_score': 1.17849505}, {'color': 'red', 'text': 'Nike men air quick handle red sports shoe', 'gender': 'men', 'product_type': 'sports shoes', 'product_id': 49, 'usage': 'sports', 'price': 160, 'heels_height': None, 'similarity_score': 1.17772675}, {'color': 'red', 'text': 'Catwalk women red flats', 'gender': 'women', 'product_type': 'flats', 'product_id': 51, 'usage': 'casual', 'price': 170, 'heels_height': None, 'similarity_score': 1.17658234}, {'color': 'red', 'text': 'Carlton london women casual red casual shoes', 'gender': 'women', 'product_type': 'casual shoes', 'product_id': 53, 'usage': 'casual', 'price': 90, 'heels_height': None, 'similarity_score': 1.16239476}]
=== Calling Function ===
Calling function: search_footwear_database with args: {"query_str": "sneakers", "filters_json": [{"key": "gender", "value": "men", "operator": "=="}, {"key": "color", "value": "blue", "operator": "=="}]}
=== Function Output ===
[{'color': 'blue', 'text': 'Skechers men blue shoes', 'gender': 'men', 'product_type': 'casual shoes', 'product_id': 34, 'usage': 'casual', 'price': 195, 'heels_height': None, 'similarity_score': 1.23348832}, {'color': 'blue', 'text': "Adidas men's canon dialect blue yellow shoe", 'gender': 'men', 'product_type': 'sports shoes', 'product_id': 31, 'usage': 'sports', 'price': 55, 'heels_height': None, 'similarity_score': 1.22590578}, {'color': 'blue', 'text': 'Puma men faas 300 blue sports shoes', 'gender': 'men', 'product_type': 'sports shoes', 'product_id': 32, 'usage': 'sports', 'price': 200, 'heels_height': None, 'similarity_score': 1.18794334}, {'color': 'blue', 'text': "Lee cooper men's lc casual denim blue shoe", 'gender': 'men', 'product_type': 'casual shoes', 'product_id': 33, 'usage': 'casual', 'price': 165, 'heels_height': None, 'similarity_score': 1.17678}, {'color': 'blue', 'text': 'Asics men gel ds sky speed 2 running blue sports shoes', 'gender': 'men', 'product_type': 'sports shoes', 'product_id': 35, 'usage': 'sports', 'price': 180, 'heels_height': None, 'similarity_score': 1.17514169}, {'color': 'blue', 'text': 'Adidas men blue & red f10 sports shoes', 'gender': 'men', 'product_type': 'sports shoes', 'product_id': 36, 'usage': 'sports', 'price': 65, 'heels_height': None, 'similarity_score': 1.16689789}]
=== LLM Response ===
Here are some options for red heels and blue men's sneakers:

### Red Heels:
1. **Carlton London Women Red & Black Heels**
   - Price: $95
   - Heels Height: 2 inches
   - Usage: Casual

2. **Carlton London Women Casual Red Stilettos**
   - Price: $175
   - Heels Height: 3 inches
   - Usage: Casual

### Blue Men's Sneakers:
1. **Skechers Men Blue Shoes**
   - Price: $195
   - Usage: Casual

2. **Adidas Men's Canon Dialect Blue Yellow Shoe**
   - Price: $55
   - Usage: Sports

3. **Puma Men Faas 300 Blue Sports Shoes**
   - Price: $200
   - Usage: Sports

If you need more information or have specific preferences, feel free to ask!

Additional Resources

For a deeper dive into this example, check out my GitHub repository where I break down the code and methodology used by the AI agent in today's scenario.

Upcoming AI agent course

Sign up here for my upcoming course to learn how to build your own AI agent chatbot

Why use AI agents instead of simple chatbots?

Users often make multiple product requests in a single chat message, such as "red heels and blue men's sneakers".

AI agents split the query into sub-requests, applying metadata filters for each, and deliver results for both.

This multi-intent approach enhances user satisfaction and ensures the chatbot covers all parts of the inquiry.