Elizabeth Fuentes L for AWS

Posted on Feb 2 • Originally published at community.aws

Elevating Customer Support With a LLM Whatsapp Travel Assistant.

#python #langchain #aws #genai

🇻🇪🇨🇱 Dev.to Linkedin GitHub Twitter Instagram Youtube
Linktr

This article was written with Guillermo Ruiz

What was an exciting trip to Las Vegas for a presentation as speaker on re:Invent 2023 turned into an unexpected journey to an unknown destination. A booking mishap resulted in the speaker gaining an airline ticket to Las Vegas, New Mexico instead of the well-known Las Vegas. This hilarious error set the stage for investigating how advanced technologies like generative AI and Retrieval Augmented Generation(RAG) can revolutionize traditional support channel models, turning a complicated ticket change into a quick solution through fluid conversation.

This blog will guide you through building a Whatsapp Travel Assistant Application that uses an LLM assistant. It can understand and communicate in multiple languages, both written and spoken, with the goal of providing self-service assistance through natural conversations and remembering previous interactions to solve common travel issues. Capable of checking the status of passenger flights, as well as the data related to their trip, using your reservation number or passenger identification.

✅ The Whatsapp Travel Assistant Application is ready to deploy using AWS Cloud Development Kit. Find the code in Elevating Customer Support With Rag Langchain Agent Bedrock Dynamodb And Kendra
github repo.

How The Travel Assistant Work?

To show how the travel assistant works, let's break it down into three main blocks:

1. Message Input and Initial Processing:

A user sends either a text or voice message via WhatsApp, the message hits the Amazon API Gateway, then a whastapp_in AWS Lambda Function is executed to process new WhatsApp messages, extract relevant details, and write them in Amazon DynamoDB Streams.

2. Message Processing Based on Format:

The process_stream Lambda Function is trigger by events in DynamoDB Streams and identify the format of the message (text or audio):

If Message is Text Format: a Lambda Function named API_bedrock_agents is triggered, the heart and brain of the Travel Assistant. We’ll explain it later.
If Message is Audio Format: The star_transcibe_job Lambda Function is triggered. This Lambda Function downloads the WhatsApp audio from the link in the message in an Amazon S3 bucket, using authentication, then converts the audio to text using the Amazon Transcribe start_transcription_job API, which leaves the transcript file in an Output Amazon S3 bucket.

Function that invokes start_transcription_jobb looks like this:

def start_job_transciptor (jobName,s3Path_in,OutputKey,codec):
    response = transcribe_client.start_transcription_job(
            TranscriptionJobName=jobName,
            IdentifyLanguage=True,
            MediaFormat=codec,
            Media={
            'MediaFileUri': s3Path_in
            },
            OutputBucketName = BucketName,
            OutputKey=OutputKey 
            )

Note: Notice that the IdentifyLanguage parameter is configured to True. Amazon Transcribe can determine the primary language in the audio.

The transcribe_done Lambda Function is triggered once the Transcribe Job is complete. It extracts the transcript from the Output S3 bucket and sends it to the agent.

3. LLM Processing and Response:

Here we explain the heart ❤️ and brain 🧠 of the Travel Assistant.

The Travel Assistant is managed by a Langchain Agent, a framework for developing LLM applications, who uses the Amazon Bedrock API to understand and respond through natural language by invoking a foundational models. This Travel Assistant employs Anthropic Claude, from which the assistant gains multilingual capabilities.

By using Retrieval Augmented Generation (RAG), the assistant can extract passenger details from an Amazon DynamoDB table and answer questions about how to resolve specific cases to a knowledge base in Amazon Kendra.

In order to have seamless conversations that recall past messages, we use the Langchain feature for memory management. This feature stores conversation content in a DynamoDB Table called SessionTable. To handle session duration, we use a DynamoDB table named user_metadata, this table stores the user’s metadata and session start, which is queried and compared with a define max session duration during each interaction. Change session duration here.

📚 Tip: Kenton Blacutt, an AWS Associate Cloud App Developer, collaborated with Langchain, creating the Amazon Dynamodb based memory class that allows us to store the history of a langchain agent in an Amazon DynamoDB.

The query_table_passanger lambda function is invoked by the agent when it needs to know the passenger's information or query user_metadata in DynamoDB table.

When the agent finishes assembling the response, they respond to Whatapp through the whatsapp_out Lambda Function.

Let's Build The Travel Assistant

Step 0: Activate WhatsApp account Facebook Developers

1- Get Started with the New WhatsApp Business Platform

2- How To Generate a Permanent Access Token — WhatsApp API

3- Get started with the Messenger API for Instagram

Step 1: Previous Configuration

✅ Clone the repo

git clone https://github.com/build-on-aws/elevating-customer-support-with-rag-langchain-agent-bedrock-dynamodb-and-kendra.git

✅ Go to:

cd customer-support-bot

✅ Create The Virtual Environment: by following the steps in the README

python3 -m venv .venv

source .venv/bin/activate

for windows:

.venv\Scripts\activate.bat

✅ Install The Requirements:

pip install -r requirements.txt

✅ Set Values:

In customer_support_bot_stack.py edit this line with the whatsapp Facebook Developer app number:

DISPLAY_PHONE_NUMBER = 'YOU-NUMBER'

This agent maintains the history of the conversation, which is stored in the session_tabble Amazon DynamoDB table, also have control session management in the session_active_tabble Amazon DynamoDB table, and sets the time here in this line:

if diferencia > 300: #session time in seg

Step 2: Deploy The App With CDK.

Follow steps here

✅ Synthesize The Cloudformation Template With The Following Command:

cdk synth

✅🚀 The Deployment:

cdk deploy

✅ Review what is deployed in the stack:

Go to the AWS Cloudformation console, select the region where you deployed and click on CustomerSupportBotStack:

Then go to the resources tab and explore what's deployed:

✅ Wait a few minutes:

This stack automatically creates an Amazon Kendra Index with the data source that contains the Q&A database of the airline "La inventada", you must wait a few minutes for all the data to be synchronized.

Step 3: Activate WhatsApp Messaging In The App

Go to AWS Secrets Manager and edit the WhatsApp settings and replace them with Facebook Developer settings.

Step 4: Configure Webhook In Facebook Developer Application

Let´s try!

✅ Q&A:

You can start asking for customer service information as if it were an airline customer service line.

✅ Passenger information:

The CDK stack creates the dynamoDB table named Passenger_ID with the sample passenger dataset from Kaggle. Select one and request information regarding it. What if I now change the language and ask the AI in Spanish?

The multilanguage function depends on the LLM you use.

✅ Send it voice notes:

Amazon Transcribe is able to automatically identify the languages spoken in your media without you having to specify a language code.

🚀 Keep testing the agent, play with the prompt in this Amazon Lambda function and adjust it to your need.

Conclusion

While the speaker's trip to the wrong city of Las Vegas began as a comedy of errors, it also highlighted an important opportunity to reimagine customer service. Whatsapp Travel Assistant is the application that the speaker imagined, an application capable of delivering a self-service experience for travelers through natural conversations.

Whatsapp Travel Assistant can:

Understand conversations in any language, both written and spoken, and response in the same language.
Query a knowledge database in Amazon Kendra and an Amazon DynamoDB Table using RAG.
Deliver sophisticated answers according to the query using RAG, querying knowledge databases in Amazon Kendra, and tables in Amazon DynamoDB.
Manage conversation memory and store it in an Amazon DynamoDB table.
Managing session time through a Amazon Dynamodb Table.

We invite you to build this application, play with it, improve it and tell us how it went for you.

Thanks! 🧔🏻‍♂️👩🏻

🚀 Some links for you to continue learning and building:

🇻🇪🇨🇱 Dev.to Linkedin GitHub Twitter Instagram Youtube
Linktr

DEV Community