The AWS SAM CLI team has published a new feature for SAM connectors called multi-destination connector. This makes it easier to define multiple integrations to several AWS services on the same invoking resource (Source). Its primary goal is to reduce the complexities associated with maintaining and writing IAM permissions inside SAM templates.
In this article, we will be exploring:
- What challenges do SAM connectors try to address.
- How to define SAM multi-destination connectors.
- How they improve developer experience compared to policy templates and the original variant of SAM connectors.
- Some opportunities that SAM team have to improve this current version.
A little bit of context
A few months ago, SAM connectors were introduced with the following goals in mind:
- Simplifying the definition of IAM permissions: With SAM connectors, developers can define IAM permissions more easily and in a more granular way.
- Reducing complexity in writing and maintaining serverless applications: SAM connectors provide an abstraction layer that makes it easier to write and maintain serverless applications. They help to simplify the process of integrating multiple AWS services and reduce the need for writing complex IAM code.
- Improving productivity: By simplifying the definition of IAM permissions and reducing the complexity of writing custom IAM code. This directly results to improved productivity.
However, some improvement opportunities were spotted by the developer community which included:
One connector for each source to destination pair.
The first version of SAM connector requires a single connector definition between a source and destination pair. Take the code example below which includes three dynamo db definitions, a lambda function and three SAM connectors:
Transform: AWS::Serverless-2016-10-31
Resources:
UsersTable:
Type: AWS::Serverless::SimpleTable
GroupsTable:
Type: AWS::Serverless::SimpleTable
PoliciesTable:
Type: AWS::Serverless::SimpleTable
#
# PUT Function
#
PutFunction:
Type: AWS::Serverless::Function
Properties:
Runtime: nodejs18.x
Handler: index.handler
CodeUri: put-endpoint/
Environment:
Variables:
USERS_TABLE_NAME: !Ref UsersTable
GROUPS_TABLE_NAME: !Ref GroupsTable
POLICIES_TABLE_NAME: !Ref PoliciesTable
Events:
PutResource:
Type: "Api"
Properties:
Path: "/user/put"
Method: "PUT"
# First connector seems tolerable
PutUserToUserDBConnector:
Type: AWS::Serverless::Connector
Properties:
Source:
Id: PutFunction
Destination:
Id: UsersTable
Permissions:
- Write
# Until you need another one!
# (Shout out to multi-table design users)
PutUserToPolicyDBConnector:
Type: AWS::Serverless::Connector
Properties:
Source:
Id: PutFunction
Destination:
Id: PoliciesTable
Permissions:
- Write
# And yet another one
PutUserToGroupDBConnector:
Type: AWS::Serverless::Connector
Properties:
Source:
Id: PutFunction
Destination:
Id: GroupsTable
Permissions:
- Write
With SAM multi-destination connectors, all connectors can now be grouped within their respective main resource (Original source / lambda function):
PutFunction:
Type: AWS::Serverless::Function
Connectors:
RWAccessConnector:
Properties:
Destination:
- Id: UsersTable
- Id: PoliciesTable
- Id: GroupsTable
Permissions:
- Write
- Read
Properties:
Runtime: nodejs18.x
Handler: index.handler
CodeUri: put-endpoint/
Environment:
Variables:
USERS_TABLE_NAME: !Ref UsersTable
GROUPS_TABLE_NAME: !Ref GroupsTable
POLICIES_TABLE_NAME: !Ref PoliciesTable
Events:
PutResource:
Type: "Api"
Properties:
Path: "/user/put"
Method: "PUT"
More lines of code compared to Sam Policy Templates
When the first version was released, it required us to write more lines of code (9 lines for read or write connector, 10 for read and write connector) than what SAM policy templates require us to write (3 lines).
PutUserToGroupDBConnector:
Type: AWS::Serverless::Connector
Properties:
Source:
Id: PutFunction
Destination:
Id: GroupsTable
Permissions:
- Write
- Read
# What it was trying to beat prior to multi-destination connector
PutUserToGroupDBConnector:
Type: AWS::Serverless::Connector
Properties:
Policies:
- DynamoDBCrudPolicy:
TableName: !Ref GroupsTable
Thanks to AWS SAM connectors, the overhead code of the function scaffold can now be seen as a valuable investment. As you link more resources to the main resource, the code line savings you get from using SAM connectors scales up, resulting in increased efficiency and reduced development time.
Connectors:
RWAccessConnector:
Properties:
Destination:
- Id: UsersTable
- Id: PoliciesTable
- Id: GroupsTable
- Id: UserIdentificationCardBucket # S3
- Id: PaymentSettlementQueue # SQS
- Id: InvoicingTopic # SNS (Diverse resource definition on unified syntax)
Permissions:
- Write
- Read
Another important advantage of AWS SAM connectors is their ability to support multiple types of destination resources using a single definition tree. This feature makes SAM connectors much more user-friendly compared to SAM policy templates.
With SAM connectors, there is no need to remember the specific property names (TableName, BucketName, QueueName, etc.) needed to link different resources, as the new syntax allows for a simplified and intuitive approach to defining resource connections
When multiple source-destination connectors are defined in separate areas of a large SAM file, it can be difficult to maintain and use them effectively.
Prior to the release of multi-destination connectors, if engineers incrementally link several resources to a function using SAM connectors, chances that they'd do it on different sections of a large SAM file are high.
Imagine engineers merging conflicting changes on a large SAM file? Sound familiar eh?
PutUserToUserDBConnector:
Type: AWS::Serverless::Connector
Properties:
Source:
Id: PutFunction
Destination:
Id: UsersTable
Permissions:
- Write
# Imagine 200 lines in-between here
PutUserToPolicyDBConnector:
Type: AWS::Serverless::Connector
Properties:
Source:
Id: PutFunction
Destination:
Id: PoliciesTable
Permissions:
- Write
# Imagine 1,532 lines in-between here
PutUserToGroupDBConnector:
Type: AWS::Serverless::Connector
Properties:
Source:
Id: PutFunction
Destination:
Id: GroupsTable
Permissions:
- Write
# At this point, we can't remember
# where they link and what is their purpose don't we?
Unwanted incidents resulting from scattered resource definitions can lead to challenges in code reading for team members over the long run. To avoid these issues, it is important to establish proper standards and quality gates in the development process.
Multi-destination connectors provide a convenient solution for managing destination-specific integration code by allowing developers to place all the code inside a unified YAML node (Source Resource).
This not only streamlines the development process, but also allows for greater ease of use and more efficient resource management, making it easier to be kind to our future selves.
PutFunction:
Type: AWS::Serverless::Function
Connectors:
RWAccessConnector:
Properties:
Destination:
- Id: UsersTable
- Id: PoliciesTable
- Id: GroupsTable
- Id: UserIdentificationCardBucket
- Id: PaymentSettlementQueue
- Id: InvoicingTopic
Permissions:
- Write
- Read
# Code trimmed for brevity
What features can SAM CLI team still improve?
Although SAM connectors offer support for a wide range of resources, they still do not cover all the resources that are available in SAM Policy Templates. It would be particularly beneficial to have connectors that support AWS Secrets Manager and AWS SSM Parameter Store, as these are two of the most commonly used features when building integrations with non-AWS resources in serverless stacks.
In my humble opinion, there is still room for improvement in SAM connectors. By implementing a more concise syntax, we can reduce the amount of boilerplate code required. For example, the following syntax could be used to define multiple connectors for a single function
PutFunction:
Type: AWS::Serverless::Function
WriteConnectors:
- Id: SampleTableA
- Id: SampleTableB
- Arn: arn:aws:secretsmanager:::secret:sample-credential-Lw92LOwe
This would make it easier to manage multiple connectors and simplify the development process even further.
- It would be really cool if the SAM CLI team could introduce a linting rule that enforces the usage of multi-destination connectors over 1-1 destination source pairs. This would encourage developers to adopt the more efficient and streamlined approach offered by multi-destination connectors, leading to more consistent code and better resource management.
Summary
- Using SAM Multi-destination connectors can improve the readability and maintainability of large SAM templates, making it easier to manage integrations between resources.
- SAM Multi-destination connectors reduce integration code length, leading to more efficient code and faster development times.
- SAM Multi-destination connectors group connectors by the source, leading to a more organized integration code.
- We are no longer constrained to use one-to-one source to destination pair connector syntax.
- There are still some opportunities to improve SAM connector syntax after this.
Wrapping Up!
I'm eager to hear your thoughts on AWS SAM Multi-destination connectors. What do you think could be improved, and how would you improve it if you had the chance to provide feedback to the SAM CLI team? Share your comments below!
Oldest comments (0)