Our chatbot challenge: handling multi-color queries

Here's the challenge we're looking to solve by building an AI agent.

Scenario

Imagine this scenario: A customer visits your store, excited to buy shoes. They ask your simple RAG system chatbot:

Customer: "I need women's black shoes with red details":

A customer initiates a chat with SoleMates and asks about women's black shoes with red details

Your chatbot responds confidently:

The LLM correctly identifies that black shoes are requested but it fails to pull multiple black shoes with red details

Why did the chatbot only recommend one pair of shoes?

The chatbot pulls a list of products from the vector database, most of which don't meet the customer's criteria.

Here's the vector database pull:

http://localhost:3000/notebooks/jupyter_notebook.ipynb

Nike women's transform iii in black pink shoe

Fila men's agony black red shoe

Fila men's passion black red canvas shoe

Red tape black men's semi casual shoe

Converse men's chuck taylor big check red black canvas shoe

Red tape men's black formal shoe

Nike women's double team lite black shoe

Red tape men's black casual shoe

Our challenge

Our naive chatbot made two key mistakes:

1. Incorrect Product Selection: It failed to correctly match "women's black shoes with red details" to the actual products
2. Misleading Response: It confidently recommended an irrelevant product (black with pink details)

This is a failure of basic retrieval systems. Simple RAG systems struggle when faced with nuanced queries requiring multiple conditions, like color combinations or gender-specific filters.

Why did the simple chatbot fail?

The simple retrieval system:

• Treated the query as a single vector, prioritizing some words while ignoring others (e.g., "black" over "red").
• Couldn't handle multi-color filtering, where multiple metadata attributes need to align.

Despite having six shoes in the database that perfectly meet the customer's query, the chatbot pulled mostly irrelevant results.

Your mission

We're going to fix this.

In this course, you'll learn to:

• Build a metadata-aware AI agent capable of understanding complex, multi-attribute queries.
• Design a chatbot that can retrieve exact matches from a vector database.
• Overcome the limitations of naive RAG systems.

By the end of this course, you'll create a chatbot that confidently delivers the six perfect matches for this query:

The AI agent looks through the retrieved vector database results and provide a response with recommended shoes

Let's start by building a simple RAG system together and see how it handles this query in the next lesson.