DEV Community

Cover image for 💥 AWS CDK 101 - 💫 Dynamodb Local setup and integrating with sam invoke
AWS Community Builders profile image Aravind V
Aravind V for AWS Community Builders

Posted on • Originally published at devpost.hashnode.dev

💥 AWS CDK 101 - 💫 Dynamodb Local setup and integrating with sam invoke

#typescript #aws #serverless #awscdk

🔰 Beginners new to AWS CDK, please do look at my previous articles one by one in this series.

If in case missed my previous article, do find it with the below links.

🔁 Original post at 🔗 Dev Post aws-cdk-101-sam-local-to-test-and-debug-lambda-function

🔁 Reposted at 🔗 dev to @aravindvcyber aws-cdk-101-sam-local-to-test-and-debug-lambda-function-1afj

Also, we have started to develop an open source project which we would be using to play around with refracting the architecture as well as learn CDK stuff at the same time we will provide something useful for our community. Find more about this discussed in the article below.

arch

🔁 Original project post at 🔗 Dev Post aws-cdk-101-projects-cdk-stackresourcedrift-events-forwarded-to-cool-slack-posts-event-forwarder

🔁 Reposted project post at 🔗 dev to aws-cdk-101-projects-cdk-stackresourcedrift-events-forwarded-to-cool-slack-posts-event-forwarder-1m0m

event-forwarder Github repo

Dynamodb local with sam invoke 🍡

Earlier in our article, we have seen how to set up local integration testing of the lambda using sam invoke. Dynamodb local will not only be a cool integration setup to inspect in locally, but also it can help to understand the dynamodb data model using NoSQL workbench.

So why do we need now ❔

In our previous article, we mentioned that we have sam local to test the function locally. But we don't have the local dynamodb. We set up that brings that into the same docker network in this article to fix the below error.

Image description

Thank you fully the error slack hook post the exception clearly to my slack

Image description

Docker composes dynamodb local 🌟

Let us use docker compose to spin up dynamodb local

Dynamodb local container 🤡

I prefer to use the docker mode of setting up dynamodb local since it was very much predictable and can also use docker compose yml file alongside.

Here in this snippet of yaml file below, you can find a simple dynamodb docker container bound to cloud network and local volume ddb-data which has been already pre-created.

version: "3.5"

services:
  dynamo:
    container_name: local-ddb
    image: amazon/dynamodb-local
    networks:
      - cloud
    ports:
      - "8000:8000"
    volumes:
      - ddb-data:/home/dynamodblocal
    working_dir: /home/dynamodblocal
    command: "-jar DynamoDBLocal.jar -sharedDb -dbPath ."

networks:
  cloud:
    external: true

volumes:
  ddb-data:
    external: true
Enter fullscreen mode Exit fullscreen mode

Create the docker network 💦

Here let us define a docker network to be shared by our local cloud containers.

docker network create -d bridge cloud
Enter fullscreen mode Exit fullscreen mode

Create the docker volume 🐟

Here let us create a new docker volume and mount it to our local folder

docker volume create \
        --driver local \
        --opt type=none \
        --opt o=bind \
        --opt device=*******Your Local Volume folder*********** \
        ddb-data
Enter fullscreen mode Exit fullscreen mode

Start and Stop dynamodb local 🌱

Navigate to the directory you have defined the docker compose file. Open the terminal and run the below command to start it.

docker-compose up -d
Enter fullscreen mode Exit fullscreen mode

And to stop the container run the below command

docker-compose down
Enter fullscreen mode Exit fullscreen mode

Validate the ddb connection local 🌴

Run the below command in the local terminal to make sure the port is correct as per your docker-compose you have defined above.

aws dynamodb list-tables --endpoint-url http://localhost:8000
Enter fullscreen mode Exit fullscreen mode

You may get some results like the below.

{
  "TableNames": [

  ]
}
Enter fullscreen mode Exit fullscreen mode

Create local table 🌾

Before that if the table already exists you can delete it as shown below.

aws dynamodb delete-table --table-name eventStores9 --endpoint-url http://localhost:8000
Enter fullscreen mode Exit fullscreen mode

Net us create the table by copying the schema from the cloud table by running the below describe-table command

aws dynamodb describe-table --table-name eventStores9 --profile av > eventStores9.json
Enter fullscreen mode Exit fullscreen mode

Note in the above step, I have not used the endpoint but rather used my AWS named profile av to connect. In your case, you can have your own and write the output to the file eventStores9.json

Now run the below command using the file created above (note it is expected to throw error)

aws dynamodb create-table --cli-input-json file://eventStores9.json --endpoint-url http://localhost:8000
Enter fullscreen mode Exit fullscreen mode

ddb local table creation error

When you get the above error, you simply have to remove those nodes from the JSON file you are using and the cleanup version of the file is already checkin to the local of this project to help you identify.

And then you can run the same command again, if it does not throw any error it should create the table and will give the description of the table as output.

aws dynamodb create-table --cli-input-json file://eventStores9.json --endpoint-url http://localhost:8000
Enter fullscreen mode Exit fullscreen mode

Validate by listing the tables locally 🌸 

aws dynamodb list-tables --endpoint-url http://localhost:8000
Enter fullscreen mode Exit fullscreen mode

You may get some results like the below.

{
  "TableNames": [
    "eventStores9"
  ]
}
Enter fullscreen mode Exit fullscreen mode

And there you have it your local dynamodb table is ready to help you.

NoSQL Workbench 🐲

https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/workbench.settingup.html

You may also try this tool which can help you model dynamodb much faster and even push it to the cloud or local connection. However, this tool does not support LSI in the data modeling stage. Hope it is available soon. If it supports LSI, we would have directly imported the cloud formation template without parameters to create the schema in the tool itself. I tried it but is ignoring the LSI. So that is why I used the AWS CLI to generate the table.

Still, this tool can help us with the Operation Builder feature which will connect to the dynamodb tables directly and help with a range of other features and even help with some code snippets as well to design.

Create a new database connection 🐐

local connection setup

Inspect the metadata and LSI and GSI definitions

local connected table metadata

With this interface, you can directly query, inspect and manipulate the local table data during testing and debugging.

Further, you can also use LSI and GSI in your operations seamlessly.

Code changes to the function 🍃

Code changes to the function to enable local ddb connection. Just like how we identified AWS_SAM_LOCAL, we can do the same here as well in the dynamodb-util.ts file.

Take note of the name of the host it is the container name we defined in docker-compose yaml

if(process.env.AWS_SAM_LOCAL) {
  options.endpoint = 'http://local-ddb:8000'
}

const dynamo = AWSXRay.captureAWSClient(new DynamoDB(options));
Enter fullscreen mode Exit fullscreen mode

Run script changes 🌲

Here we will readily know that sam CLI scripts are part of the run scripts as we demonstrated in the previous article. Here we need to include the docker network while using sam local invoke to be able to connect to used other

{
  "sam:i": "sam local invoke $npm_config_fname --docker-network cloud -e $npm_config_event | jq .",
    "sam:i:debug": "sam local invoke $npm_config_fname --docker-network cloud -d 9999 -e $npm_config_event  --log-file logs/sam-debug-logs.txt --debug | jq .",
}
Enter fullscreen mode Exit fullscreen mode

This should help to connect the function with local dynamodb as expected.

Post testing events results in dynamodb 🌰

I have run all the test events I have prepared in the last article in events and I can see the results in the local table.

npm run t:1:1 -- --event=events/event_stack_create_complete.json
npm run t:1:1 -- --event=events/event_stack_delete_complete.json
npm run t:1:1 -- --event=events/event_stack_update_complete.json
npm run t:1:1 -- --event=events/event_resouce_create_complete.json
npm run t:1:1 -- --event=events/event_drift_detection_complete.json
Enter fullscreen mode Exit fullscreen mode

Scan dynamodb items

dynamodb items local

Normal Query

normal query

Query using LSI_STATUS

lsi status query

Debugging the data in vscode 🌿 

npm run d:1:func1 -- --event=events/event_stack_create_complete.json
Enter fullscreen mode Exit fullscreen mode

debugger started

Verify that it is passing through the sam local script section when it is true and it resets the dynamodb endpoint.

sam local verification and local ddb

With this approach, we will be able to monitor data moving in and out of the dynamodb table in local
inspect data moving and out of ddb

If you don't like to use js debugging when using typescript with webpack you can add the below to your launch config attach configuration sourceMapPathOverrides as discussed in the previous article.

{
"sourceMapPathOverrides": {
    "meteor://💻ap{workspaceFolder}/*",
    "webpack:///./{workspaceFolder}/node_*",
    "webpack://?:{workspaceFolder}/*"
}
Enter fullscreen mode Exit fullscreen mode

source map overrides

Then directly you can place break points in the typescript files and you can cleanly debug them as follows.

Sample typescript debugging screenshots

typescript debug 1

Sample typescript debugging screenshots with a specific point of interest

typescript debug 2

Conclusion ⛲

This will be extremely useful when you repeatedly iterate and make code changes to the processor and you could use the test event to test it swiftly using local dynamodb setup and integration.

We will be talking about more similar engineering concepts as we refactor and refine the event forwarder project. Keep following for similar posts on engineering with IaC primarily using AWS CDK and Serverless.

Also, feel free to contribute to the progress of the below solution with your comments, and issues, maybe you can also do a pr if you feel it can help our community.

event-forwarder Github repo

arch

🔁 Original project post at 🔗 Dev Post aws-cdk-101-projects-cdk-stackresourcedrift-events-forwarded-to-cool-slack-posts-event-forwarder

🔁 Reposted project post at 🔗 dev to aws-cdk-101-projects-cdk-stackresourcedrift-events-forwarded-to-cool-slack-posts-event-forwarder-1m0m

⏭ We have our next article in serverless and IaC, do check out

🎉 Thanks for supporting! 🙏

Would be great if you like to ☕ Buy Me a Coffee, to help boost my efforts 😍.

Buy Me a Coffee at ko-fi.com

🔁 Original post at 🔗 Dev Post aws-cdk-101-dynamodb-local-setup-and-integrating-with-sam-invoke

🔁 Reposted at 🔗 dev to @aravindvcyber aws-cdk-101-dynamodb-local-setup-and-integrating-with-sam-invoke-527f

Top comments (0)

Read next

pmbanugo profile image

How We Moved from AWS to Fly.io

Peter Mbanugo -

kungfoosian profile image

How I built the infrastructure for my cloud resume

Anthony Vo -

fady_nabil10 profile image

New features in Amazon Bedrock Prompt Management

Fady Nabil -

fady_nabil10 profile image

New Feature: Amazon CloudFront no longer charges (No Billing) for requests blocked by AWS WAF

Fady Nabil -