DEV Community

loading...

DynamoDb stream lambda trigger

konsta_ivlev profile image Konstantin ・3 min read

Assignment

Let's take a look on a typical scenario, once it is required to subscribe to DynamoDb stream events and trigger AWS lambda function. In theory and in tutorials the task looks quite simple:

private subscribeToDynamoDbStream(
    lambdaRole: Role,
    layer: LayerVersion,
    applicationName: string,
    targetTable: ITable) {
    const lambda = new Function(this, `${applicationName}-lambda`, 
    {
          code: Code.fromAsset('functions/custom-trigger/src'),
          functionName: `${applicationName}-lambda`,
          handler: handler,
          layers: [layer],
          role: lambdaRole,
          runtime: Runtime.NODEJS_12_X,
     });

     lambda.addEventSource(new DynamoEventSource(targetTable, {
     startingPosition: StartingPosition.TRIM_HORIZON, }));
}
Enter fullscreen mode Exit fullscreen mode

This scenario works well in case of the single repository, once lambda, dynamodb and dynamodb stream are provisioned in the same stack.

Issue

In more complex layout the dynamodb itself is created and corresponding dynamodb stream is provisioned in a different stack. The dynamodb arn is saved to the SSM parameter store, or the name is known.
So, first import table:

private importTargetTable(environment: string, tableArn: string) {
    const table = Table.fromTableArn(this, 'ImportedTable', tableArn);
    return table;
  }
Enter fullscreen mode Exit fullscreen mode

Then call subscribeToDynamoDbStream function:

const importedTable = importTargetTable(this.environment, this.tableArn);
this.subscribeToDynamoDbStream(lambdaRole, layer, applicationName, importedTable);
Enter fullscreen mode Exit fullscreen mode

The result is unexpected: DynamoDB Streams must be enabled on the table [tablename]. The AWS console clearly shows that the Dynamodb stream is enabled. Quick googling recommends the most obvious thing: enable already enabled feature. CDK source code inspection reveals that during fromTableArn invocation the stream arn most probably has not been loaded.

Solution

First of all there should be DynamoDb stream arn exported to SSM during DynamoDB provisioning:

const targetTable = new Table(this, `targeTableName`, {
   partitionKey: { ... },
   sortKey: { ... },
   ...
   stream: StreamViewType.NEW_AND_OLD_IMAGES,
   tableName: `targetTableName`
    });
   // Export target table stream arn to SSM
   new StringParameter(this.ctx, `targetTable-stream-arn`, {
      description: "Exported stream arn",
      parameterName: `/targeTable-stream-arn`,
      stringValue: targetTable.tableStreamArn!
    });  
Enter fullscreen mode Exit fullscreen mode

In a stack where a lambda is created and subscription to the dynamodb stream is setup, tableStreamArn is read from SSM.

Option 1

Change the way how ITable is constructed:

private importTargetTable(environment: string, tableArn: string, tableStreamArn: string) {
    const table = Table.fromTableAttributes(this, 'ImportedTable', {
        tableArn: tableArn,
        tableStreamArn: tableStreamArn
    return table;
  }
Enter fullscreen mode Exit fullscreen mode

Option 2

Technically speaking AWS lambda with DynamoDb stream trigger does not have to be aware of DynamoDb table (and it's arn). Option 2 could be considered as an improvement of the code above: 'Invent and Simplify'.
Once the lambda is created, EventSourceMapping object could be used to enable DynamoDb stream trigger for the lambda usin tableStreamArn only:

const lambda = new Function(this, ...);
new EventSourceMapping(this, `lambda-trigger-mapping`, {
    eventSourceArn: tableStreamArn,
    startingPosition: StartingPosition.TRIM_HORIZON,
    // any necessary options of a choice, e.g. batchSize, retryAttempt, etc.
    target: lambda
});
Enter fullscreen mode Exit fullscreen mode

I hope that this post would help you in your CDK learning journey.
Best wishes..

Discussion (0)

pic
Editor guide