In an engineering team with matured devops practices, it is often required to know all the DynamoDBs running in various accounts. This list of tables could then be used for:
- Performing standard compliance checks on your table names and configuration.
- Build alarm systems that monitors list of tables in a production account to make sure that every single table you need exists.
- Creating backups and restoring data in DynamoDB tables.
- Managing and organizing your DynamoDB tables.
- Monitoring and optimizing performance of your DynamoDB tables.
- Automating and integrating your application with other AWS services that use DynamoDB tables as input or output.
- Auditing and tracking usage of your DynamoDB tables.
In this article, we'll be exploring how to retrieve all the DynamoDB table names using AWS SDK for JavaScript v3.
Before you start
I'd recommend using the latest version of NodeJS supported by Lambda Functions. It's also helpful to have nvm in your machine to easily switch between NodeJS versions.
# Install Node 18 to use the latest code-base
nvm install 18
node --version
# Switch to NodeJS 18
nvm use 18
You will have to install the JavaScript AWS SDKs v3 for DynamoDB. If you are using NPM, use the following commands to install the SDKs.
# If you use NPM
npm install @aws-sdk/client-dynamodb
# Install only if you want to use
# named profiles inside your dev machine
npm install @aws-sdk/credential-providers
If you need to use yarn:
# If you use yarn
yarn add @aws-sdk/client-dynamodb
# Install only if you want to use inside your dev machine
yarn add @aws-sdk/credential-providers
The Source Code
By design, ListTablesCommand can only return a maximum of 100 DynamoDB names from a single AWS region (see the code below).
import { DynamoDBClient, ListTablesCommand } from "@aws-sdk/client-dynamodb";
import { fromIni } from "@aws-sdk/credential-providers";
const client = new DynamoDBClient({ region: "ap-southeast-1" });
// Use this code if you need named profiles
// const client = new DynamoDBClient({
// credentials: fromIni({ profile: "my-poc-profile" }),
// region: "ap-southeast-1",
// });
const command = new ListTablesCommand({});
const response = await client.send(command);
console.log(response);
Which would result to the following JSON response. The response below contains all the DynamoDB tables in a single region (In our case ap-southeast-1):
{
'$metadata': {
httpStatusCode: 200,
requestId: 'LKJEWJOI3290923902302310219',
extendedRequestId: undefined,
cfId: undefined,
attempts: 1,
totalRetryDelay: 0
},
LastEvaluatedTableName: undefined,
TableNames: [
'dynamo-table-a',
'dynamo-table-b',
'dynamo-table-c'
]
}
The page limitation of 100 items per response is probably placed by AWS to gracefully handle accounts with large number of DynamoDB tables in them.
If you really want to retrieve all the tables in a single region of a single account. You can use the following code instead:
import { DynamoDBClient, ListTablesCommand } from "@aws-sdk/client-dynamodb";
import { fromIni } from "@aws-sdk/credential-providers";
const client = new DynamoDBClient({
credentials: fromIni({ profile: "my-poc-profile" }),
region: "ap-southeast-1",
});
let startTableName = "";
let hasResults = false;
let tableNames = [];
do {
hasResults = false;
const searchInput = {
Limit: 100,
};
if (startTableName) {
searchInput.ExclusiveStartTableName = startTableName;
}
const command = new ListTablesCommand(searchInput);
const response = await client.send(command);
if (response.TableNames && response.TableNames.length > 0) {
startTableName = response.TableNames[response.TableNames.length - 1];
tableNames = [...tableNames, ...response.TableNames];
hasResults = true;
}
} while (hasResults);
console.log(tableNames);
Which would result to:
[ 'dynamo-table-a', 'dynamo-table-b', 'dynamo-table-c' ]
To improve the re-usability of this code, we can export it into a modular function that looks like:
import { DynamoDBClient, ListTablesCommand } from "@aws-sdk/client-dynamodb";
import { fromIni } from "@aws-sdk/credential-providers";
export async function getAllDynamoDBTableNamesV3(profile, region) {
const clientConfig = {};
if (profile) {
clientConfig.credentials = fromIni({ profile });
}
if (region) {
clientConfig.region = region;
}
const client = new DynamoDBClient(clientConfig);
let startTableName = "";
let hasResults = false;
let tableNames = [];
do {
hasResults = false;
const searchInput = {
Limit: 100,
};
if (startTableName) {
searchInput.ExclusiveStartTableName = startTableName;
}
const command = new ListTablesCommand(searchInput);
const response = await client.send(command);
if (response.TableNames && response.TableNames.length > 0) {
startTableName = response.TableNames[response.TableNames.length - 1];
tableNames = [...tableNames, ...response.TableNames];
hasResults = true;
}
} while (hasResults);
return tableNames;
}
Now that our query could be re-used, its now easier to retrieve all Dynamo DB tables from various regions inside a single AWS account.
Single Account Multi-region Search
Let's try something that seeks all tables from different active regions:
import { getAllDynamoDBTableNamesV3 } from "./get-all-dynamodb-table-names.js";
const regions = [
"ap-northeast-1",
"ap-northeast-2",
"ap-northeast-3",
"ap-south-1",
"ap-southeast-1",
"ap-southeast-2",
"ca-central-1",
"eu-central-1",
"eu-north-1",
"eu-west-1",
"eu-west-2",
"eu-west-3",
"sa-east-1",
"us-east-1",
"us-east-2",
"us-west-1",
"us-west-2",
];
const accountTables = await regions.reduce(async (memo, region) => {
const regionTables = await getAllDynamoDBTableNamesV3(
"my-poc-profile",
region
);
const regionTablePair = regionTables.map((tableName) => {
return {
region,
tableName,
};
});
return [...(await memo), ...regionTablePair];
}, []);
console.log(accountTables);
Which should result to the following table-region key pairs:
[
{
region: 'ap-southeast-1',
tableName: 'dynamodb-table-a'
},
{
region: 'ap-southeast-1',
tableName: 'dynamodb-table-b'
},
{
region: 'ap-southeast-1',
tableName: 'dynamodb-table-c'
},
{
region: 'ap-southeast-2',
tableName: 'dynamodb-table-d'
},
]
How about multi-account query for all DynamoDBs in an AWS organization?
Well, I'll be leaving that for you to enjoy a bit of coding using ES6 and AWS SDK v3. To give a bit of hint, you can wrap the single AWS account query into its own module and make the profile name "Configurable" so that you can implement an parallelised search across various AWS accounts.
This approach should be helpful in monitoring several accounts in an entire organization.
Top comments (0)