Table of content
- Background - What brought us here?
- cfn-diagram - CloudFormation visualizer
- evb-cli - EventBridge pattern generator and debugging suite
- cfn-resource-actions - VS Code extension that lets you interact with deployed resources from the template
- sam-policies-cli - CLI UI for browsing and injecting SAM Policy Templates
- iam-policies-cli - CLI UI for building complex IAM policies
- cwlogs-cli - Tool to quickly launch CloudWatch Logs Insights with multiple log groups pre-selected
A background
When we in January 2017 made the decision to rewrite Sweden's leading online grocery store, MatHem.se, from scratch into a serverless microservices architecture we found ourselves in an extremely promising yet premature tech segment with a lack of tooling due to a yet quite small community.
This led us to adopting a mindset of 'when tasks get repetitive, find a tool for it or write one yourself'.
Much of our tooling solves bespoke in-house procedural bottlenecks, but when we find solutions to tasks that can benefit the community we make sure to publish it in the open.
This post is a summary of the AWS/serverless related productivity tools we have authored.
cfn-diagram
NPM: https://www.npmjs.com/package/@mhlabs/cfn-diagram
GitHub: https://github.com/mhlabs/cfn-diagram
Installation
npm i -g @mhlabs/cfn-diagram
Features
Parses CloudFormation/SAM JSON or YAML and renders a diagram in either draw.io or vis.js network format.
Examples
Reason for existing
Our application is made up of hundreds of microservices, each defined with its own CloudFormation/SAM template. No engineer knows about every stack. Our engineering team has been going through a rapid growth phase which has required onboarding of new developers to be efficient.
To ease the getting to know a stack, we had an initiative that every repository was required to have an up-to-date diagram over the service's resources in the readme file. No matter how good the intentions were, those sort of 'musts' will never be followed and these diagram will be something that 'might be true' and therefore never trusted.
With cfn-diagram any developer new to a stack can quickly pull up a visualisation of how it all hangs together. It's also useful to render in planning meetings instead of drawing on the whiteboard
evb-cli
NPM: https://www.npmjs.com/package/@mhlabs/evb-cli
GitHub: https://github.com/mhlabs/evb-cli
Installation
npm i -g @mhlabs/evb-cli
Features
Pattern generator and debugging tool for EventBridge with features including:
- Event pattern builder
- Input transformation builder
- Event rules browser
- Generate a diagram over the events flow in a region
- Local debugging of events where actual events are sent to the console over websockets. These can be forwarded into sam-local for some advanced debugging.
Examples
Pattern generation
Events flow diagram generator
Local debugging
Reason for existing
Historically we have been heavily invested in pub/sub using SNS/SQS, but we were always annoyed with the tight coupling this created between services and teams, so when EventBridge was announced we got very excited.
One of the most powerful features of EventBridge is its content based filtering where you only let through events matching a part of the payload that interests the consumer of the event. Together with EventBridge's Schema Registry an engineer can build event patterns without knowing anything about the producing service.
These event patterns often get quite complex and we found ourselves spending a lot of time first writing, then deploying, then debugging them. Typos were a common mistake both when composing event patterns and input transformations.
Our developers also reported that they find it cumbersome to debug their rules as well as finding it difficult to see the whole picture of how all events hang together, so from that feedback we added local debugging features
cfn-resource-actions
Marketplace: https://marketplace.visualstudio.com/items?itemName=ljacobsson.cfn-resource-actions
GitHub: https://github.com/mhlabs/cfn-resource-actions
Installation
ext install ljacobsson.cfn-resource-actions
Features
Interact with your deployed CloudFormation/SAM templates directly from the template. Turns your template in to an interface into the AWS console
- Tail lambda log
- Query DynamoDB tables
- Copy Physical Id of any resource
- Consume EventBridge events directly in vs code
- Send to and Poll SQS
- Visualise template
- etc
Examples
Invoke lambda + tail its logs
CTRL+click from template to Lambda handler
Query DynamoDB
Visualise stack requires Draw.io Integration extension
Reason for existing
Context switching is a huge cost in productivity loss. Leaving VS Code to go and do stuff in the AWS console is a distraction we want to avoid
sam-policies-cli
NPM: https://www.npmjs.com/package/@mhlabs/sam-policies-cli
GitHub: https://github.com/mhlabs/sam-policies-cli
Installation
npm i -g @mhlabs/sam-policies-cli
Features
CLI UI to quickly inject SAM policy templates into your AWS::Serverless::Function resources
Example
Reason for existing
We strive to follow the Principle of Least Privilege and finding the correct policy templates requires googling
iam-policies-cli
NPM: https://www.npmjs.com/package/@mhlabs/iam-policies-cli
GitHub: https://github.com/mhlabs/iam-policies-cli
Installation
npm i -g @mhlabs/iam-policies-cli
Features
UI to quickly build simple to complex IAM policies based on resources in a CloudFormation template
Example
Reason for existing
We strive to follow the Principle of Least Privilege and composing granular IAM policies is time consuming and prone to human errors
cwlogs-cli
NPM: https://www.npmjs.com/package/@mhlabs/cwlogs-cli
GitHub: https://github.com/mhlabs/cwlogs-cli
Installation
npm i -g @mhlabs/cwlogs-cli
Features
Create groups of log groups from resource tags or log group prefixes and launch CloudWatch Logs Insights with those groups predefined
Example
Reason for existing
CloudWatch Insights Logs is a great way to search your logs. Often you want to search across many log groups to, for example, tracing a correlation id across multiple services. CloudWatch Logs Insights lets you search across up to 20 log groups in one query, but adding log groups from the dropdown is a bit cumbersome.
Top comments (0)