Fabio Beoni

Posted on Sep 29 • Edited on Oct 10

Part 2 - Deploy a Chat App, Bedrock Agent and Actions to Ask Questions about Live Data

This article is part of the series "Generative-AI-driven Chatbot for the Factory Plant" implementing a P.O.C. about an environmental pollution monitoring (PM2.5/PM10) to safeguard workers in a production plant.

Disclaimer

The information provided in this document is for informational purposes only and is subject to change. While efforts have been made to ensure the accuracy and reliability of the content, no guarantees are made regarding the completeness, reliability, or suitability of the information.
The author shall not be liable for any losses, injuries, or damages arising from the use or reliance on the information contained herein.

The AWS resources provisioned following this article can generate costs. Make sure to delete all the resources created at the end of the exercise.

Overview

The main purpose of this series of articles is to test out Amazon Bedrock Studio (no-code editor for Gen-AI driven apps), at the time I am writing the Studio is still in preview and actually after a quick try I encountered a number of bugs that prevent me to complete this P.O.C.

To get around of these issues, I decided to go ahead with P.O.C. by implementing my self the missing part, keeping a low-code approach and favoring configuration of Bedrock components over imperative programming as much as possible. I will then come back to Studio as soon as it reaches a stable release.

For those of you who had a read to "Part 1 - Gathering pollution data from the edge, pushing them to AWS IoT Core and DynamoDB", this new article adopts Terraform to provision all the resources, including those you have created before (IoT Core, Certificates, IoT Rules, DynamoDB Table, etc...).

The tutorial also runs a simulation of the IoT edge device (RaspberryPI+Nova Laser Sensor), so you can start from scratch without any physical hardware.

Preview of the App

Bedrock Components

LLM Claude 3 Sonnet

Gen-AI layer carrying out:

user question understanding, semantic meaning, intent recognition and parameters extraction (if any),
understanding of sensor data domain as well as data specific pollution values and metadata,
generation a contextual answer including a selection of most relevant data, according to the business rules provided to the LLM instructions.

Ref.: https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids.html

Bedrock Actions Group

Layer providing to the LLM a text description of "actions" available to the Agent to carry out work, by invoking lambda functions. In this use-case the Agent can invoke the action query-pm-values to get near real-time pollution data from the sensor.

Ref.: https://docs.aws.amazon.com/bedrock/latest/userguide/agents-action-create.html

Lambda Functions

Business logic layer implementing read/write of user/agent chat messages, as well as reading of pollution data.

Bedrock Knowledge-base

Technical and procedural knowledge layer to further instruct workers with company's specific know-how. Included later, on the next article.
Ref.: https://docs.aws.amazon.com/bedrock/latest/userguide/kb-how-it-works.html

Bedrock Agent

Logical coordinator of LLM, Actions and Knowledge-base. It is the "glue" making Bedrock a smart tool to build Gen-AI services.
Ref.: https://docs.aws.amazon.com/bedrock/latest/userguide/agents-how.html

An overview of how Agent/LLM/Actions work together:

Ref.: https://docs.aws.amazon.com/images/bedrock/latest/userguide/images/agents/agents-runtime.png

_{Source image: AWS Documentation}

Other Cloud Resources Included

AWS
- Cognito (IDP, user authentication, login UI)
- API Gateway (http interface layer chat-app/services)
- DynamoDB (db data storage of chat messages and pollution data)
- OpenSearch Serverless (knowledge-base documents, from the next article)
- IoT Core components (to interact and gather data from the edge device)
- IAM roles/policies
Surge.sh
- CDN domain (to host the chat-app UI)

Architecture Recap

Click to zoom.

Note: this diagram shows the components of the Knowledge-base module that is still coming soon.

Project Structure

Source code repo: https://bitbucket.org/fabiobeoni/bedrock-studio-chatbot/src/v2/ (note v2)

Modules

/cloud (Terraform/Bash/JS):
- /ai-module: Amazon Bedrock components
- /chat-app-module: AWS resources serving the mobile chat app ui
- /pm-monitor-module: AWS IoT Core serving the edge device
/edge
- /chat-app (TS/Ionic/Vue3)
- /pm-monitor-service (NodeJS) Pollution reader service. This module designed to run on a RaspberryPI with Nova PM Laser sensor attached, by default runs locally emulating the IoT device (./device_config.json mockSensor=true).

Build & Deploy

Repository (version 2)

https://bitbucket.org/fabiobeoni/bedrock-studio-chatbot/src/v2/ (note v2)

Setup / Requirements

In AWS Console / Amazon Bedrock / Base Models : you have to request access to the model anthropic.claude-3-sonnet-20240229-v1:0. You can use a different one, in that case remember to override the default variable llm_id in ./variables.tf.
AWS CLI
Terraform
NodeJS
OpenSSL

Build Lambda Functions

echo ">> Build Lambda Functions"

echo ">> AI module"
(cd "cloud/ai-module/functions" && /bin/bash package.sh query-pm-values-fnc)

echo ">> Chat App module"
(cd "cloud/chat-app-module/functions" && /bin/bash package.sh chat-write-user-message-fnc)
(cd "cloud/chat-app-module/functions" && /bin/bash package.sh chat-read-user-messages-fnc)
(cd "cloud/chat-app-module/functions" && /bin/bash package.sh ask-bot)

Create and Register the Device Certificate

echo '>> Provisioning self signed certificate'

# set a variable to host the name
thing_name="gateway_pm_monitor"

# create certificate and key pairs, 
# assign the $thing_name as subject 
openssl req -x509  -newkey rsa:2048 -sha256  -days  365  -nodes \
    -keyout edge/pm-monitor-service/gateway.private_key.pem \
    -out edge/pm-monitor-service/gateway.certificate.pem \
    -subj "/CN=$thing_name"

 #download AWS CA
 curl -O https://www.amazontrust.com/repository/AmazonRootCA1.pem
 mv AmazonRootCA1.pem edge/pm-monitor-service

Deploy Cloud Resources to AWS

echo ">> Deploy cloud infra"
terraform init
terraform apply

# save cloud endpoints and IDs to the edge/chat-app to instruct the chat app
terraform output -json > edge/chat-app/src/config.json

# save iot endpoint and cognito client ID to the edge/pm-monitor-service to instruct the edge device
terraform output -json  | jq '{cognito_client_id: .cognito_client_id, iot_endpoint_address: .iot_endpoint_address}' \
    > edge/pm-monitor-service/cloud_config.json

Build Chat App

echo ">> Build chat app"
HOSTING_URL=$(terraform output -raw hosting_url)
npm --prefix edge/chat-app/ run setup
npm --prefix edge/chat-app/ run build

Deploy Chat App to Surge.sh CDN or Run in localhost

echo ">> Deploying the chat app to the CDN: ${HOSTING_URL}"

# Note: the first time Surge asks you to provide an email/pw to register an account 
HOSTING_URL=${HOSTING_URL} npm --prefix edge/chat-app/ run deploy

echo ">> You can now open the app to: ${HOSTING_URL} and login with user/pw provided to Terraform/Cognito"

Note: for free accounts sometimes Surge.sh websites do not respond, in that case you can test the chat-app in http://localhost:3000, by:

# terminal window 1
(cd edge/chat-app && npm run dev)

# terminal window 2
(cd edge/chat-app && npm run serve)

echo ">> finally open browser to http://localhost:3000 and login with user/pw provided to Terraform/Cognito"

Run PM Monitor Service (emulated locally)

Some sample pm values are available already into the DynamoDB, just start this service if you want to test the overall process with live data from IoT Core.

echo ">> Running the pm monitor service emulating the edge device"

npm --prefix edge/pm-monitor-service/ install --omit=dev
npm --prefix edge/pm-monitor-service/ run start

Clean Up Cloud Resources

terraform destroy -auto-approve

surge list
surge teardown <output_from_list>

Final Notes

Steps Functions over Lambda

Bedrock Agent's Action wants a Lambda target. To embrace a full no-code approach with configuration over programming you might want to consider using lambda to proxy all the action executions requests to AWS Step Functions. In that way you can define you business logic declaratively with YAML/JSON, or even draw them with the visual editor provided by Step Functions. Keep in mind that you will deal with sync execution and timeouts, because the Bedrock Agent does require Lambda to return a result.

Sensor Input Validation

The P.O.C. does not implement any validation logic for data coming from the IoT edge device. For the purpose of a demo/P.O.C. that's not a problem, but keep in mind if you use this repo to start a new project.

Part 3 - Providing workers guidance with Amazon Bedrock Knowledge-base and Gen-AI

DEV Community