What does the next generation of AI eCom look like?

This is a submission for the Wix Studio Challenge .

What I Built

For this challenge I chose to explore a future of AI driven eCommerce. I tried to take AI innovation beyond the typical chatbot or content generation use case and break the shopping experience free of what has become the standard for the past 10+ years.

By tapping into the design capabilities of Wix Studio, the Velo APIs of Wix's eCommerce business solution Wix Store, and OpenAIs completion models I created an immersive shopping experience like nothing you have seen before (I hope 😅).

In this shop instead of browsing and interacting alone by clicking around, you are accompanied by an AI companion who shows your products, adds them to your cart, and even helps take you to the checkout when your done.

While this MVP only accommodates text based interactions - the concept opens up a world of more interactive and accessible shopping experiences using voice and vision capabilities of modern AI models.

Demo

Try it out yourself here: https://eitanwaxman.wixstudio.io/ai-store

Development Journey

I started out by building an 'agent' that can create Vector embeddings using the product data and store it in a Wix Collection. This agent runs whenever a product is added or updated in the Wix Store. These embeddings serve as the foundation for the semantic search capabilities of the shop.
Note: In an ideal world I would utilize a Vector database for this but opted to stay in Wix to save time.

Next I moved on to building out the brain of the shop which essentially follows this process:

1. Receives a query from the user and gets the intent from the 'intent agent'. This intent is one of many actions the user might want to be taking in the shop such as browsing products, adding something to their cart, or checking out.
2. Executes an action by interacting with the Velo APIs via a dedicated agent. These agents use AI to get a JSON object of parameters that are then passed to the API.
3. Generates a plain text response based on: The companion personality, the conversation history, the action taken, and the data returned.

After that it was a matter of tweaking, debugging, and elaborating on the above.

Some challenges I encountered (some still not completely resolved 🙃):

1. Handling situations where multiple chain actions need to be taken based on a single user query.
2. Handling special scenarios like product variants.
3. Getting the AI not to confuse conversation history with facts and data.

Which APIs and Libraries did you utilize?

wix-stores-backend: Creating embeddings onProductCreated and onProductUpdated events. Also for retrieving product variants.
wix-ecom-backend: Adding products to cart, other cart interactions and generating a checkout link.
wix-data: Storing and retrieving embeddings.
wix-fetch & wix-secrets-backend: Fetching responses from OpenAI APIs.
wix-location-frontend: Navigation to the checkout link.
wix-storage: Storing the companion selection in session.

Thanks for reading!