Showcasing scores and product recommendations

To present the product recommendations and their similarity scores, we'll use IPython for rendering HTML directly in the Jupyter Notebook. Follow these steps to set up an HTML string that includes the query, images, and their scores:

# Initialize an HTML string
html_str = f"<h3>Query: '{customer_query}'</h3><table><tr>"

Iterate over the sorted scores, appending each product image and its score to the HTML string:

# Loop through sorted scores and images
for filename, score in sorted_scores.items():
    image_path = os.path.join('data', 'bags', filename)
    
    # Adding each image and its details to the HTML string
    html_str += f"<td style='text-align:center'><img src='{image_path}' width='100'><br>{filename}<br>Score: {score:.3f}</td>"

After looping through all relevant products, close off the HTML table and use IPython to display it:

html_str += "</tr></table>"

# Display the HTML
display(HTML(html_str))

If you run the cell, the output should look something like this:

Similarity scores displayed

There you have it, a recommendation system that compares the customer inquiry with your product catalog and gives the customer the most similar product.

But what happens if a customer asks for feature not shown in a product photo?

The described method with image embeddings works great when a customer asks for visible features, like color and shape. But what happens when a customer asks about a specific brand or material that might not be visible in a product photo?

For these scenarios, you can input both text and image data into the AWS Titan model to generate a single vector that captures all relevant information. This way, the AI model can handle queries about non-visible features like brand or material effectively.

I'm working on a detailed course where we get to build a chatbot agent that can handle both cases.

Join the waitlist to learn more about integrating advanced AI capabilities into your applications.

In the following section, we'll review the overall system we've developed.