On 7th April 2021 I was stunned to find out I had won the 2020 Liberty Mutual Chairman's award. To put this into context, "The Chairman’s Award is our most prestigious award handed out once a year and recognises employees whose significant contributions, effort and performance go above and beyond". If you are ever in Boston, my name should soon be permanently displayed somewhere in the Foyer of the Liberty Mutual HQ. This is only the second time someone from Liberty IT has won the award so I am truly humbled by it.
The award description:
This seemed like a perfect opportunity to reflect on the past 12 to 18 months, share what I have learned, what mistakes I made and how I ended up here.
Why AWS CDK?
I have used a lot of technologies in my career, why have I pushed so hard to make this one easy for our engineers to embrace? Why not just pull out the old "it depends" when asked about Infrastructure as Code tools?
2 years ago, I was promoted into a different role within Liberty. I went from driving the technical direction of a product area, working with a specific set of teams to being an enabling function across ~240 developers in Belfast/Dublin or thousands globally within the wider organisation. I was given the freedom and autonomy to do this my own way but how could I have any kind of impact on a group this large?
Honestly, on day one I wasn't sure this was possible but I had some down time and my colleague Mark McCann had said to me "Have you seen AWS CDK? It's pretty cool". The LIT technical strategy set by Dave Anderson was that we wanted to be serverless first. My new area in Liberty Mutual also had a serverless first north star. I sat and thought about the biggest blockers to serverless developer productivity from my previous team. Two things came to mind:
- CloudFormation
- API Gateway
The configuration we needed to use for API Gateway was so specific that a typical CloudFormation Template (CFT) with just a gateway in it was approx 1500 lines long. Developers facing that reality coming from SpringBoot where you just annotate a method was grim. You also had no local validation of the CFT, developers needed to deploy and hope it worked (it frequently didn't so we used to sit around at tea telling our horror stories). Dedicated Infra engineers will tell me that pure YML for CFT is the best method but for everyday engineers to move at the speed we wanted it was causing considerable drag and stress.
Since Mark had put AWS CDK in my head, I thought "can it help here?". The way I evaluated it was to build an AWS CDK construct for our API Gateway configuration, this construct came with a full set of unit tests and reduced the 1500 lines of YML mentioned above to about 15 lines of TypeScript
I had never seen a product with so much potential to get us where we needed to go through developer empowerment before, it even allowed us to bring our engineering excellence standards from previous platforms. The problem was AWS CDK had basically only gone Generally Available(GA), it was changing rapidly to meet the needs of early adopters and as such examples of people using it were scarce. I also had to factor in that the serverless team at AWS were mostly focused on their new product at the time AWS SAM. We needed to use the product right there and then, not wait 2 years - To me, this seemed like the right level for an enabling architect to operate.
Introducing The Software Accelerator
At this exact moment in time, a Senior Architect in Liberty Mutual (Rajesh Kannan) was creating the Software Accelerator. This was going to be our developer enablement platform where everyone could share working patterns and practices for rapid deployment. To see this platform in action you can sign up for CDK Day on 30th April 2021 (Accelerator CDK Day).
I excitedly phoned Rajesh and said "hey, this is a game changer" he agreed and had already luckily decided the accelerator would be based on AWS CDK. This left me in a fortunate position, I had a technology I could see the potential in and a currently unreleased platform, still in dev that could make this technology easy to distribute. What was missing? The working patterns, I wanted to make sure that by the time the Accelerator launched our engineers had working patterns to deploy with it. Developers have deadlines to meet and one bad experience with a technology causes inertia to try it again so this was mission critical to our serverless first journey in my head.
Finding The Patterns
There was a decision point where as an Architect I could have gone to my happy place and created super complex, opinionated patterns backed by my ideas alone. That plan would not have worked, what we needed was for groups of engineers to reuse / contribute back to the same patterns without all the bloat I would inevitably add due to personal preferences. The easiest way to create these patterns was to find someone in the industry who is already recognised and respected so that when a developer disagrees with the implementation you can say "well, this well known industry expert says otherwise, want to get them on a call and discuss it? Before we set that up, here is all the evidence for why it works and here are all the companies doing it in production. Still disagree?"
Jeremy Daly was that industry expert, an AWS Serverless Hero who had a famous blog post dedicated to serverless patterns. The intention here was never to take away credit from Jeremy but to open up a new audience of developers by coding his patterns in AWS CDK and linking it all back to him.
Open Source or Inner Source?
The next big decision was if I should do this just for Liberty or since the industry as a whole was lacking reliable code examples for AWS CDK if I should create my first ever proper open source project. In December 2019 I attended AWS re:Invent where I spoke to other developers plus watched some of the talks where I confirmed this was an industry issue.
On Jan 11th 2020 after some holiday time off, I made my decision:
Was thinking “be the change you want to see in the world” today so I purchased cdkpatterns.com so that developers can have a place to grab the official #cdk version of aws #serverless archs. Wanna help contribute some patterns? I’d be ecstatic! Mvp will be a github repo.20:48 PM - 11 Jan 2020
Marketing
I now had a new problem, my social media presence was non existent so how would I get the word out? I had no budget to spend on advertisements or on a marketing team. I decided to start small and created the @CdkPatterns twitter handle then every time I created a new pattern I would tweet about it and tag the pattern's original creator, this way if what I was doing was valuable they might retweet it and growth would be organic. I also created a dev.to account and started blogging about each of the patterns as they were released. Finally I created a YouTube channel where I talked through the theory of some of the patterns. I will show later how the marketing worked out but we have some steps in the story to discuss first.
Building a Product
The plan was to make a developer productivity play through some kind of product called CDK Patterns with open source at the core.
How would I judge the success of my product? It wasn't profits or usage in production since I was focused on education over reusable libraries. I did not want to own the code liability for the whole CDK community, I wanted to empower them to use the standard AWS constructs without custom building. Ideally, people wouldn't even know my name after using the product.
The North Star of CDK Patterns was to increase serverless adoption of AWS CDK in the industry but at that point in time the community was very small and we didn't meet. I decided that for the first while I would judge success on unique visitors to the repo, clones, GitHub stars and I would also take an informal check of serverless AWS CDK noise in the industry (blog posts, YouTube videos, conference presentations etc).
It is important to note that we are talking about the behaviour of a global developer community here which is a rich tapestry of interactions, motivations and reactions so I knew I could contribute to the outcome I wanted but I am not in any way claiming that I owned this whole north star.
I had a second North Star that was voluntary serverless AWS CDK adoption in Liberty Mutual. This was something I could easily measure through Accelerator usage metrics. This one was arguably more important but I knew it would come as a side effect of the industry adoption.
The Language Problem
It quickly became apparent that since AWS CDK supported multiple programming languages (TypeScript, Python, Java, .Net at the time), I needed a way to not isolate the majority of the community. I did a quick poll on Twitter to ask what languages people were using and at the time Python/TS covered 87% of users with TypeScript the majority
CDK Patterns@cdkpatterns
Pythonista Friends! I see you, so today I added a python version of every pattern. Clone, build, deploy and learn today at cdkpatterns.com. #serverless #ServerlessForEveryone #python18:49 PM - 08 Feb 2020
I came up with a flow where all new patterns get added in TS/Python with the other languages something the community could contribute later as needed.
The Monorepo Problem
In another poll I asked the community if they would rather have a repo per pattern or keep them all together in all available languages. The resounding answer was to keep them all together. Whilst on paper this seems simpler and easier for users, it presents an information discovery problem. There is only so much you can do with a GitHub readme file (believe me I pushed the limits).
I created two solutions to try and solve this problem, the first was a cli tool that allowed developers to clone any single pattern in their desired language with one command
CDK Patterns@cdkpatternsI’m very excited to share with you a new way of learning cdk patterns! Now you can clone and deploy any pattern in 3 commands - npmjs.com/package/cdkp This is a beta solution for now so be kind but I hope you all enjoy using it. #ServerlessForEveryone10:05 AM - 08 Mar 2020
The second was to create the website cdkpatterns.com as an information radiator for the patterns available in the repo. I allowed you to browse all the patterns or filter the patterns by the AWS component contained within (e.g. Lambda, SNS, SQS)
The Content Problem
Once I had CDK Patterns established a new problem emerged - how do I help people choose the right pattern for their situation? If I just kept adding patterns with no guidelines then it would be overwhelming with choice. The solution I came up with was to pair the patterns with the AWS Well Architected Framework, the idea was that developers could do a well architected review of their application and then based on the outcome of that report they could find CDK Patterns to help fill the gaps. Heitor Lessa helped me significantly with refining this idea. If you read the recent AWS case study on Liberty Mutual you will see this mirrored our internal strategy as well
🥳🥳 Announcing 🥳🥳
cdk patterns alter ego:
serverless-patterns.co.uk
look inside the well architected section to see cdk patterns matched with serverless well architected best practices. @heitor_lessa has helped me so much with this.
Now, let's fill in the blanks13:58 PM - 19 May 2020
Bringing The Community Together
Remember my North star was to increase serverless adoption of AWS CDK in the industry but I had no way of judging this since we didn't meet properly. I had this idea to create a new conference called CDK Day where the whole community could come together for one day and showcase the brightest and best of CDK (by this point we had 3 CDKs - AWS CDK, CDK for Terraform and cdk8s for Kubernetes)
I wrote a full article on this idea which goes into who the amazing group were that brought this together and the timelines we faced:
as well as my learnings:
The short version is that over 3000 people signed up to attend with over 1300 tuning in live for the keynote.
People signed up from nearly everywhere, this still astonishes me:
Most of the talks were about serverless implementations so I was ecstatic to see the community developing. Part of the discussions for CDK Day were about how do we keep people talking and Sebastian Korfmann had purchased the cdk.dev domain so we agreed to create a cdk.dev Slack space which would be used for attendees to chat during the day and hopefully they would stay there afterwards. As of April 2021 there are over 2000 members in that slack community with constant daily interactions.
AWS Summit Americas
In one of the most surreal experiences of my career, the day before CDK Day Werner Vogels as CTO of Amazon quoted me in his keynote where he tells the origin story of AWS CDK. He also took the time to talk about CDK Patterns and deep dive into how they apply the Well Architected Framework
Aaaaaaagh @Werner talking about me, @Liberty_IT / @LibertyMutual and serverless, well-architected @CdkPatterns!!!
Highlights are the CloudWatch Dashboard and the Lambda Power Tuner
Checkout the keynote here - pages.awscloud.com/aws-summit-onl…
also remember cdkpatterns.com 😃15:03 PM - 29 Sep 2020
You can watch the full video here:
A couple of days later he also said these very kind words
@NIDeveloper Matt you played a very important role in the evolution of the CDK, so you should take big credit for its success.16:46 PM - 01 Oct 2020
Published Articles With AWS
AWS were graceful enough to ask me to co author a couple of blog posts with them as well
Speaking At Events (and others speaking about CDK Patterns)
Another great avenue to spread the word about CDK Patterns was to appear at other people's events. There is a massive amount of work in preparing to speak at an event, crafting the slides/narrative but it definitely brings traction!
Serverless Bash
NI Dev Conf
AWS Community Days Dublin
What a great setup at #AWSCommunityDay Dublin (@AWSUserGroups) today!
If you missed my talk representing @Liberty_IT you can find the slides here:
speakerdeck.com/nideveloper/bu…13:25 PM - 13 Oct 2020
Serverless Chats Podcast
Serverless Chats Podcast@serverlesschatsOn the next episode, Jeremy chats with Matt Coulter (@NIDeveloper) about why he built CDKPatterns.com, how he used it to help @Liberty_IT choose the CDK, how they've used these patterns to implement Well-Architected #serverless solutions, and much more. Don't miss it!16:12 PM - 21 Nov 2020
Marcia Villalba What to Watch at re:Invent
Dave Anderson at re:Invent
19:05 PM - 08 Dec 2020
Jason Fulghum at re:Invent
I’m just catching up with some of the re:Invent talks, i’m up to “AWS CDK what’s new and what’s next”
thanks @jason_fulghum for mentioning my wee project @CdkPatterns 🙌08:09 AM - 19 Dec 2020
The Results
In December 2020 I gave this presentation showing the social media growth and repo stars trend
As of April 2021, the CDK Patterns Twitter account has 2600 followers, the repo has over 1100 GitHub Stars, hundreds of visitors/clones per week. The interesting thing about monitoring the repo usage is that visitors are always higher than clones but that is because developers open the repo, find the few lines they need, copy it into their IDE and leave. Given my original mission was an education play this is the perfect balance that I always wanted. The YouTube channel has ~600 subscribers (I haven't put enough time into it to have serious growth)
All of this led to me being announced as one of the first AWS DevTools Heroes (I can't embed any more tweets so the image will have to do with the link to the blog underneath)
What Have I Learned? (In no particular order)
- The AWS Community are awesome! So many welcoming people willing to just pitch in and help
- Don't underestimate the amount of time you will spend doing marketing vs coding if you build a product
- Developers genuinely love AWS CDK
- Organising a conference is a serious amount of work, even virtually. It only works when you can pull together an awesome team of volunteers
- Managing Pull Requests and maintaining an open source repo is basically a full time job
How Can You Get More Involved?
If you are excited about everything you have read and serverless architectures deployed through AWS CDK then you can:
- Follow @CdkPatterns on Twitter
- Sign up for and share CDK Day with everyone you know
- Join the cdk.dev slack
- Open a Pull Request with a new CDK Pattern on cdkpatterns.com
Top comments (1)
This is an awesome write-up, Matt! And a huge congratulations, I don't know how you manage to fit it all in 😂 You definitely took the "be the change" quote to heart!
I'm excited to see where you take things in 2021! 🚀🚀🚀