DynamoDB OneTable (OneTable) is an access library for DynamoDB applications that use one-table design patterns with NodeJS. OneTable makes dealing with DynamoDB and one-table design patterns dramatically easier while still providing easy access to the full DynamoDB API.
OneTable is used by the SenseDeep Serverless Troubleshooter for all DynamoDB access. OneTable is provided open source (MIT license) from GitHub OneTable or NPM OneTable.
History and Credits
After watching the famous Rick Houlihan DynamoDB ReInvent Video, we changed how we used DynamoDB for our SenseDeep serverless troubleshooter to use one-table design patterns. However, we found the going tough and thus this library was created to make our one-table patterns less tedious, more natural and a joy with DynamoDB.
A big thank you to Alex DeBrie and his excellent DynamoDB Book. Highly recommended. And thanks also to Jeremy Daly for his Off by None Blog blog, posts and his DynamoDB Toolbox which pointed out a better way for us to do a number of things.
OneTable Overview
OneTable is not an ORM. Rather it provides a convenience API over the DynamoDB APIs. It offers a flexible high-level API that supports one-table design patterns and eases the tedium of working with the standard, unadorned DynamoDB API.
OneTable can invoke DynamoDB APIs or it can be used as a generator to create DynamoDB API parameters that you can save or execute yourself.
OneTable is not opinionated (as much as possible) and provides hooks for you to customize requests and responses to suit your exact needs.
Here are some of the key features of OneTable.
- Schema supported one-table access to DynamoDB APIs.
- Efficient storage and access of multiple entities in a single DynamoDB table.
- High level API with type marshaling, validations, and extended query capability for get/delete/update operations.
- Bidirectional conversion of DynamoDB types to Javascript types.
- Option to invoke DynamoDB or simply generate API parameters.
- Generation of Conditional, Filter, Key and Update expressions.
- Schema item definitions for attribute types, default values, enums and validations.
- Powerful field level validations with required and transactional unique attributes.
- Easy parameterization of filter and conditional queries.
- Multi-page response aggregation.
- Compound and templated key management.
- Encrypted fields.
- Support for Batch, Transactions, GSI, LSI indexes.
- Intercept hooks to modify DynamoDB requests and responses.
- Controllable logging to see exact parameter, data and responses.
- Simple, easy to read source to modify (< 1000 lines).
- Safety options to prevent "rm -fr *".
- No module dependencies.
- Support for the AWS SDK v3
Installation
npm i dynamodb-onetable
Quick Tour
Import the OneTable library. If you are not using ES modules or Typescript, use require to import the libraries.
import {Table} from 'dynamodb-onetable'
If you are using the AWS SDK V2, import the AWS DynamoDB class and create a DocumentClient instance.
import DynamoDB from 'aws-sdk/clients/dynamodb'
const client = new DynamoDB.DocumentClient(params)
This version includes prototype support for the AWS SDK v3.
If you are using the AWS SDK v3, import the AWS v3 DynamoDBClient class and the OneTable Dynamo helper. Then create a DynamoDBClient instance and Dynamo wrapper instance.
import {DynamoDBClient} from '@aws-sdk/client-dynamodb'
import Dynamo from 'dynamodb-onetable/Dynamo'
const client = new Dynamo({client: new DynamoDBClient(params)})
Initialize your your OneTable Table instance and define your models via a schema.
const table = new Table({
client: client,
name: 'MyTable',
schema: MySchema,
})
This will initialize your your OneTable Table instance and define your models via a schema.
Schemas
Schemas define your models (entities), keys, indexes and attributes. Schemas look like this:
const MySchema = {
indexes: {
primary: { hash: 'pk', sort: 'sk' }
gs1: { hash: 'gs1pk', sort: 'gs1sk' }
},
models: {
Account: {
pk: { value: 'account:${name}' },
sk: { value: 'account:' },
id: { type: String, uuid: true, validate: /^[0-9A-F]{32}$/i, },
name: { type: String, required: true, }
status: { type: String, default: 'active' },
zip: { type: String },
},
User: {
pk: { value: 'account:${accountName}' },
sk: { value: 'user:${email}', validate: EmailRegExp },
id: { type: String },
accountName: { type: String },
email: { type: String, required: true },
firstName: { type: String, required: true },
lastName: { type: String, required: true },
username: { type: String, required: true },
role: { type: String, enum: ['user', 'admin'], required: true, default: 'user' }
balance: { type: Number, default: 0 },
gs1pk: { value: 'user-email:${email}' },
gs1sk: { value: 'user:' },
}
}
}
Schemas define your models and their attributes. Keys (pk, gs1pk) can derive their values from other attributes via templating.
Alternatively, you can define models one by one:
const Card = new Model(table, {
name: 'Card',
fields: {
pk: { value: 'card:${number}'}
number: { type: String },
...
}
})
To create an item:
let account = await Account.create({
id: '8e7bbe6a-4afc-4117-9218-67081afc935b',
name: 'Acme Airplanes'
})
This will write the following to DynamoDB:
{
pk: 'account:8e7bbe6a-4afc-4117-9218-67081afc935b',
sk: 'account:98034',
id: '8e7bbe6a-4afc-4117-9218-67081afc935b',
name: 'Acme Airplanes',
status: 'active',
zip: 98034,
created: 1610347305510,
updated: 1610347305510,
}
Get an item:
let account = await Account.get({
id: '8e7bbe6a-4afc-4117-9218-67081afc935b',
zip: 98034,
})
which will return:
{
id: '8e7bbe6a-4afc-4117-9218-67081afc935b',
name: 'Acme Airplanes',
status: 'active',
zip: 98034,
}
To use a secondary index:
let user = await User.get({email: 'user@example.com'}, {index: 'gs1'})
To find a set of items:
let users = await User.find({accountId: account.id})
let adminUsers = await User.find({accountId: account.id, role: 'admin'})
let adminUsers = await User.find({accountId: account.id}, {
where: '${balance} > {100.00}'
})
To update an item:
await User.update({id: userId, balance: 50})
await User.update({id: userId}, {add: {balance: 10.00}})
To do a transactional update:
let transaction = {}
await Account.update({id: account.id, status: 'active'}, {transaction})
await User.update({id: user.id, role: 'user'}, {transaction})
await table.transaction('write', transaction)
Why OneTable?
DynamoDB is a great NoSQL database that comes with a steep learning curve. Folks migrating from SQL often have a hard time adjusting to the NoSQL paradigm and especially to DynamoDB which offers exceptional scalability but with a fairly low-level API.
The standard DynamoDB API requires a lot of boiler-plate syntax and expressions. This is tedious to use and can unfortunately can be error prone at times. I doubt that creating complex attribute type expressions, key, filter, condition and update expressions are anyone's idea of a good time.
Net/Net: it is not easy to write terse, clear, robust Dynamo code for one-table patterns.
Our goal with OneTable for DynamoDB was to keep all the good parts of DynamoDB and to remove the tedium and provide a more natural, "Javascripty" way to interact with DynamoDB without obscuring any of the power of DynamoDB itself.
More?
You can read more in the detailed documentation at: GitHub OneTable or NPM OneTable.
Future
We'll be releasing additional components that we use in-house like a migration tool and DynamodDB maintenance scripts to remove orphaned items and legacy attributes. If you have suggestions or ideas for how to improve OneTable please let us know. All feedback, contributions and bug reports are very welcome.
SenseDeep with OneTable
At SenseDeep, we've used the OneTable module extensively with our SenseDeep serverless troubleshooter. All data is stored in a single DynamoDB table and we extensively use one-table design patterns. We could not be more satisfied with DynamoDB implementation. Our storage and database access costs are insanely low and access/response times are excellent.
Contact
You can contact me (Michael O'Brien) on Twitter at: @SenseDeepCloud, or email and ready my Blog.
To learn more about SenseDeep and how to use our troubleshooter, please visit https://www.sensedeep.com/.
Top comments (1)
Now with AWS SDK V3 support. github.com/sensedeep/dynamodb-onet...