Developer's Guide: Crafting API-Driven Apps with MongoDB Atlas Using AWS CDK, API Gateway, and Lambda

Author:

Deepti Chugh (Sr Partner Success SA at AWS)
Bharath S (Senior Partner Solutions Architect ISVs at AWS)

Contributor:
Babu Srinivasan (Senior Partner Solutions Architect at MongoDB)

Welcome to our technical blog, where we unveil a step-by-step guide to deploying a robust REST API powered by Lambda functions, expertly bridging the gap between MongoDB Atlas and AWS, all with the added convenience of automation through the AWS Cloud Development Kit (CDK). Our mission is to empower developers like you to seamlessly integrate MongoDB Atlas with AWS API Gateway, all while implementing authentication via Cognito User Pools. If you're ready to embark on a journey that not only streamlines the process of building modern API-driven applications but also leverages the power of automation, you're in the right place. Let's dive into the details and unlock the potential of this dynamic integration.

What Will You Build?
This solution comprises the following AWS services which get deployed using CDK (Cloud Development Kit):

• In the MongoDB SaaS account:
  o A MongoDB cluster
  o A MongoDB project
  o A MongoDB database user

• In the AWS customer account:
  o Amazon Cognito UserPool - user directory for authentication and authorization
  o AWS Secrets Manager – for keeping MongoDB Database Credentials
  o Application Programming Interface (API) Gateway – acts as the "front door" for applications to access data, business logic, or functionality from your backend services
  o Lambda function – connects to the Mongo DB database using PyMongo which is the Python driver for MongoDB

Reference architecture

In the above figure, the users call the API gateway endpoint to access MongoDB Atlas by invoking the AWS Lambda function. The user is authenticated by Amazon Cognito services. The credentials are stored in AWS Secrets Manager and the entire setup can be automated using the AWS CDK. MongoDB Atlas resides in a distinct Atlas VPC, fully administered by MongoDB. It is accessed securely through a private link for enhanced security.

Implementation Steps

This solution uses AWS CDK to deploy the solution on AWS. The first step involves creating a MongoDB cluster and database and then deploying AWS.

Prerequisites:

• AWS CDK
• NPM
• MongoDB Atlas Account
• AWS Account and AWS CLI Installed and Configured
• Activate MongoDB Atlas CloudFormation resources in your AWS account with sufficient permissions
• Store MongoDB Atlas programmatic API Keys in AWS Secrets Manager. You can find more details about these in MongoDB's tutorial or the GitHub repository

Step 0: Initialize the CDK Project

1. Open the IDE of your choice — Cloud9, VS Code, etc.
2. Execute the below commands to initialize the environment.

#Get the application code
    git clone https://github.com/mongodb-partners/Microservice_Application_with_MongoDBAtlas_AWSCDK_APIGW_Lambda.git
    cd aws_mongodb_sample_dir

# If you DONT have cdk installed
    npm install -g aws-cdk

# Make sure you in root directory
    python3 -m venv .venv
    source .venv/bin/activate
    pip3 install -r requirements.txt

Step 1: Deploy MongoDB Atlas and AWS resources

1. Set up the AWS CLI and connect to the session.

2. Copy the [MongoDB Atlas Organization ID] and this will be used in the next step.(https://www.mongodb.com/docs/atlas/access/orgs-create-view-edit-delete/#view-organizations)

3. Run the below commands to install the Python dependencies included with this sample.

#Install Dependencies for Lambda Function
    cd aws_mongodb_sample
    pip install --target ./dependencies pymongo
    cd ..

# Set Environment Variables
    export ORG_ID="<ORG_ID>"
    export MONGODB_USER="<MONGODB_USER>"
    export MONGODB_PASSWORD="<MONGODB_PASSWORD>"

    cdk bootstrap aws://<ACCOUNT_NUMBER>/<AWS-REGION>

1. Run the below commands to deploy the CDK template.

cdk synth
cdk deploy --all

Copy the API gateway output endpoint from the terminal as you will need this while testing the API gateway. Alternatively, you can copy it from the stack output from the cloud formation in the console.

Step 3: Explore the Deployed Resources

Once the CDK is deployed, go to the AWS Console and verify the resources
1) MongoDB::Atlas::Cluster
2) MongoDB::Atlas::Project
3) MongoDB::Atlas::DatabaseUser
4) MongoDB::Atlas::ProjectIpAccessList
5) Secret for storing ATLAS DB URI
6) Cognito User Pool
7) Lambda
8) API Gateway

Step 4: Test the Resources
1) Navigate to the Cognito user pool and copy the user pool ID and client ID (in the App Integration tab) from the Cognito user pool

2) Open Cloud Shell and create a user with the command below:

aws cognito-idp admin-create-user --user-pool-id  <YOUR_USER_POOL_ID>  --username apigwtest

3) Once you’ve created the user, since it’s created by an admin, we will have to force change the password by running the below command:

aws cognito-idp admin-set-user-password --user-pool-id <YOUR_USER_POOL_ID>  --username apigwtest  --password <YOUR_PASSWORD> --permanent

4) Replace the user pool ID and client ID copied in the above step. Also, replace the password of the user created above.

aws cognito-idp admin-initiate-auth --user-pool-id <YOUR_USER_POOL_ID> --client-id <CLIENT_ID>  --auth-flow ADMIN_NO_SRP_AUTH --auth-parameters USERNAME=apigwtest,PASSWORD=<YOUR_PASSWORD>

5) Copy the ID token created from the above step and run the below command to test the API. Copy the API_GATEWAY_ENDPOINT from the API gateway console --> API Gateway: APIs: ApiGateway (xxxxxx) :Stages

curl --location --request GET 'https://<API_GATEWAY_ENDPOINT>.execute-api.us-east-1.amazonaws.com/dev' --header 'Content-Type: application/json' --header 'Authorization: <ID_TOKEN>'

Conclusion

As we wrap up our journey into the world of modern API-driven applications, we hope this blog has illuminated the path to seamless integration. With AWS CDK, MongoDB Atlas, Cognito, and Lambda at your disposal, you're armed with the tools to craft dynamic, efficient, and scalable applications. The power of these technologies lies in your hands, and we encourage you to roll up your sleeves, dig into the code, and embark on your development adventure. The possibilities are boundless, and your next innovative application could be just a few lines of code away. So, go ahead and explore, experiment, and turn your ideas into reality with the combination of AWS CDK, MongoDB Atlas, Cognito, and Lambda. Your journey is just beginning, and the future of application development is at your fingertips.

Try Out
AWS CDK for MongoDB Atlas,
Amazon Cognito, and
AWS Lambda

Rollback

cdk destroy --all

Cost and Licenses
There is no cost to use this Partner Solution, but you will be billed for any AWS services or resources that this Partner Solution deploys. For more information, refer to the AWS Partner Solution General Information Guide.
This Partner Solution deploys MongoDB Atlas resources with the latest stable MongoDB enterprise version, which is licensed and distributed under the Server Side Public License (SSPL).