This December, I'm showing how naive chatbots fail and how AI agents make shopping easier.
We've seen naive chatbots fail at clarifying questions, context shifts, numerical requirements, multiple requests in one query, price filters, style suggestions, unavailable colors and negations.
Today, we focus on remembering color requirements while keeping the original product constraints intact.
When a customer first asks for men's running shoes and then follows up with a color preference, a naive chatbot might forget the original request. An AI agent, however, remembers both the category and the color, delivering results that match the entire conversation.
Source Code
For a deeper dive into this example, check out my GitHub repository for a detailed, step-by-step implementation in Jupyter Notebooks, breaking down the code and methodology used by the AI agent in today's scenario.
Introducing SoleMates
SoleMates is our fictional online shoe store we'll use to illustrate these concepts:
SoleMates is our fictional online shoe store
Today's Challenge: Failing to Remember Context When Color is Added
Scenario
Customer: "I'm looking for men's running shoes":
A customer initiates a chat with SoleMates and asks about men's casual shoes, but not in black
The naive chatbot vectorizes the customer inquiry and correctly pulls spots shoes from the vector database.
Sudden context shift
But a moment later, the customer says "Do you have them in red?":
Customer: "Do you have them in red?":
The customer likes the selection but like to see the sports shoes in red
Naive Chatbot Response
The naive chatbot treats every new message as a standalone query. When asked "Do you have them in red?" it forgets the original focus on "men's running shoes" and just searches for red products.
Naive Chatbot: "Yes, there are several products in red:
A naive RAG chatbot processes each user message independently and replies with irrelevant shoes
The chatbot reply mixes women's shoes, casual shoes, and various irrelevant options.
The chatbot fails to remember we need men's running shoes.
Why Did the Naive Chatbot Fail?
The naive chatbot does not remember the initial product category (men's running shoes) when handling the follow-up request. It processes each new question independently.
Limitations Highlighted
- No conversation memory
- No consistent filtering criteria across the conversation
- Confused results that fail to meet the user's needs
AI Agent Solution
The AI agent uses conversation memory and metadata filtering.
When the customer says, "I'm looking for men's running shoes," it applies filters: gender = men.
It shows men's running shoes:
The AI agent applies a filter and finds men's sports shoes
When the user then asks, "Do you have them in red?" the AI agent doesn't start over.
It adjusts the query to men's running shoes + color = red:
The AI agent doesn't forget the original focus on men's sports shoes
How Did the AI Agent Succeed?
- Conversation Memory: It remembers the initial query
- Dynamic Filtering: It adds the new color filter without losing the original gender and product type filters
- Accurate Results: The agent finds red men's running shoes that actually match what the customer wants
Key Takeaways
Naive Chatbot Limitation
- Forgets the original query and returns irrelevant suggestions when a new preference is added
AI Agent Advantages
- Maintains a chain of context
- Adjusts and refines queries without losing track of previous filters
- Delivers accurate, context-aware product recommendations
Conclusion
When customers refine their search with color or any other attribute, naive chatbots often fail by ignoring earlier details. AI agents, on the other hand, build upon the entire conversation, ensuring every new detail refines the results rather than starting from scratch.
About This Series
This series highlights how naive chatbots stumble in realistic shopping scenarios and how AI agents excel by maintaining context, applying filters, and delivering truly helpful answers.
Coming Up: Understanding Context Shifts
In tomorrow's issue, we'll explore How AI Agents Handle Multi-Color Requirements
I'm preparing a course to teach you how to build and deploy your own AI agent chatbot. Sign up here!
Behind the Scenes: Code Snippets
Here's a simplified illustration of how the AI agent processes the query.
We're giving the agent access to two tools it can use freely:
- Vector database metadata filtering
- Vector database query
1. Vector database metadata filtering tool
def create_metadata_filter(filter_string):
# For "men's casual shoes -black":
# Returns:
# [{"key": "gender", "value": "men", "operator": "=="},
# {"key": "color", "value": "red", "operator": "=="}]
filters = parse_filters(filter_string)
return filters
2. Vector database query
def search_footwear_database(query_str, filters_json):
embedded_query = embed_query_aws_titan(query_str)
results = vector_db.search(embedded_query, filters=filters_json)
return results
I use AWS Titan, a multimodal embedding model that converts both product texts and images into vectors, integrated as the function embed_query_aws_titan into the AI agent tool search_footwear_database.
This means the AI agent can process a query like "red heels" and match it to not only product descriptions but also actual images of red heels in the database.
By combining text and image data, the model helps the AI agent provide more relevant and visually aligned recommendations based on the customer's input.
Agent workflow
Construct agent worker with access to the two tools and initiate the agent:
from llama_index.core.agent import AgentRunner, FunctionCallingAgentWorker
agent_worker = FunctionCallingAgentWorker.from_tools(
[
create_metadata_filters_tool,
query_vector_database_tool,
])
agent = AgentRunner(
agent_worker,
)
Initiate conversation with agent:
agent.chat("I'm looking for men's running shoes")
Agent workflow:
- Customer: "I'm looking for men's running shoes"
- Agent calls
create_metadata_filter("men's running shoes") - Gets filter:
[{"key": "gender", "value": "men", "operator": "=="}]
- Returns men's running shoes
- Customer: "Do you have them in red?"
- Agent calls
create_metadata_filter("red men's running shoes") - Returns red men's running shoes:
The AI agent doesn't forget the original focus on men's sports shoes
Here's the full agent workflow output:
Added user message to memory: I'm looking for men's running shoes
=== Calling Function ===
Calling function: create_metadata_filter with args: {"filter_string": "men's running shoes"}
=== Function Output ===
[{"key": "gender", "value": "men", "operator": "=="}]
=== Calling Function ===
Calling function: search_footwear_database with args: {"query_str": "running shoes", "filters_json": [{"key": "gender", "value": "men", "operator": "=="}]}
=== Function Output ===
[{'color': 'blue', 'text': 'Asics men gel ds sky speed 2 running blue sports shoes', 'gender': 'men', 'product_type': 'sports shoes', 'product_id': 35, 'usage': 'sports', 'price': 180, 'similarity_score': 2.21155381}, {'color': 'blue', 'text': 'Skechers men blue shoes', 'gender': 'men', 'product_type': 'casual shoes', 'product_id': 34, 'usage': 'casual', 'price': 195, 'similarity_score': 1.7302072}, {'color': 'brown', 'text': 'Force 10 men brown shoes', 'gender': 'men', 'product_type': 'casual shoes', 'product_id': 70, 'usage': 'sports', 'price': 55, 'similarity_score': 1.72068143}, {'color': 'blue', 'text': 'Puma men faas 300 blue sports shoes', 'gender': 'men', 'product_type': 'sports shoes', 'product_id': 32, 'usage': 'sports', 'price': 200, 'similarity_score': 1.71378744}, {'color': 'white', 'text': 'Fila men fila boston white sports shoes', 'gender': 'men', 'product_type': 'sports shoes', 'product_id': 18, 'usage': 'sports', 'price': 185, 'similarity_score': 1.70925343}, {'color': 'red', 'text': 'Fila men leonard red shoes', 'gender': 'men', 'product_type': 'casual shoes', 'product_id': 48, 'usage': 'casual', 'price': 90, 'similarity_score': 1.70791}, {'color': 'red', 'text': 'Nike men jordan fly wade red sports shoes', 'gender': 'men', 'product_type': 'sports shoes', 'product_id': 45, 'usage': 'sports', 'price': 55, 'similarity_score': 1.70653546}, {'color': 'black', 'text': 'Adidas men eqt nitro fashion black sports shoes', 'gender': 'men', 'product_type': 'sports shoes', 'product_id': 1, 'usage': 'sports', 'price': 120, 'similarity_score': 1.70371079}, {'color': 'red', 'text': 'Basics men red casual shoes', 'gender': 'men', 'product_type': 'casual shoes', 'product_id': 47, 'usage': 'casual', 'price': 105, 'similarity_score': 1.69492245}, {'color': 'green', 'text': 'Flying machine men casual green casual shoes', 'gender': 'men', 'product_type': 'casual shoes', 'product_id': 60, 'usage': 'casual', 'price': 100, 'similarity_score': 1.6935122}]
=== LLM Response ===
Here are some men's running shoes you might be interested in:
1. **Asics Gel DS Sky Speed 2** - Blue sports shoes, priced at $180.
2. **Puma Faas 300** - Blue sports shoes, priced at $200.
3. **Fila Boston** - White sports shoes, priced at $185.
4. **Nike Jordan Fly Wade** - Red sports shoes, priced at $55.
5. **Adidas EQT Nitro Fashion** - Black sports shoes, priced at $120.
If you need more options or specific features, feel free to let me know!
Added user message to memory: Do you have them in red?
=== Calling Function ===
Calling function: create_metadata_filter with args: {"filter_string": "red men's running shoes"}
=== Function Output ===
[{"key": "gender", "value": "men", "operator": "=="}, {"key": "color", "value": "red", "operator": "=="}]
=== Calling Function ===
Calling function: search_footwear_database with args: {"query_str": "running shoes", "filters_json": [{"key": "gender", "value": "men", "operator": "=="}, {"key": "color", "value": "red", "operator": "=="}]}
=== Function Output ===
[{'color': 'red', 'text': 'Fila men leonard red shoes', 'gender': 'men', 'product_type': 'casual shoes', 'product_id': 48, 'usage': 'casual', 'price': 90, 'similarity_score': 1.70791}, {'color': 'red', 'text': 'Nike men jordan fly wade red sports shoes', 'gender': 'men', 'product_type': 'sports shoes', 'product_id': 45, 'usage': 'sports', 'price': 55, 'similarity_score': 1.70653546}, {'color': 'red', 'text': 'Basics men red casual shoes', 'gender': 'men', 'product_type': 'casual shoes', 'product_id': 47, 'usage': 'casual', 'price': 105, 'similarity_score': 1.69492245}, {'color': 'red', 'text': "Nike men's mercurial gli red pink shoe", 'gender': 'men', 'product_type': 'sports shoes', 'product_id': 50, 'usage': 'sports', 'price': 100, 'similarity_score': 1.21902752}, {'color': 'red', 'text': 'Nike men air quick handle red sports shoe', 'gender': 'men', 'product_type': 'sports shoes', 'product_id': 49, 'usage': 'sports', 'price': 160, 'similarity_score': 1.21318638}]
=== LLM Response ===
Here are some red men's running shoes available:
1. **Nike Jordan Fly Wade** - Red sports shoes, priced at $55.
2. **Nike Mercurial Gli** - Red and pink sports shoes, priced at $100.
3. **Nike Air Quick Handle** - Red sports shoes, priced at $160.
Let me know if you need more information or other options!
Additional Resources
For a deeper dive into this example, check out my GitHub repository where I break down the code and methodology used by the AI agent in today's scenario.
Want to build an AI agent that respects user preferences and exclusions? Sign up here and create smarter chatbots.
Why use AI agents instead of simple chatbots in e-commerce?
Customers often search for specific products like "men's running shoes in red" online.
AI agents use advanced search capabilities and memory to find the perfect match.
Unlike simple chatbots, AI agents filter results by color, product type, and gender all at once.
These advanced techniques help online shoppers quickly find what they need, enhancing user satisfaction and boosting conversions.