DEV Community

Ricardo Sueiras for AWS

Posted on • Originally published at blog.beachgeek.co.uk

AWS open source newsletter - 2022 in review

The AWS open source newsletter - review of 2022

I wanted to kick off 2023 by sharing some data points and things of interest that came up in 2022 as part of writing and putting together this open source newsletter. Given the nature of the newsletter, and that transparency and openness is core to open source, I hope these might be interesting to some of you.

In 2022, I published 45 newsletters over the course of the year. These were enjoyed (I hope) by over 100K readers from across the world and clicked on an average of five of the posts I share each week. I actually post across two sites, dev.to and my personal blog. Readers came from all over the world, with over 110 countries accessing the newsletter. The top five countries where U.S.A, India, United Kingdom, Australia, and Germany.

Over 150K words were written, with nearly 3K (2920) open source related articles shared across code, blog posts, videos, and more. We featured over a thousand authors (actual number is 1211), including many first time authors, AWS Heroes, AWS Community Builders, and the broader AWS and open source community. See below for the top ten posts of 2022.

In 2022, the newsletter introduced you to 931 new open source projects (with 187K clicks throughs) so I hope you were able to check and use a few of these. See below for the top 15 projects.

I transitioned to using Hugo to build my personal blog, and introduced tagging as a way of being able to better find the favourite open source technologies you like. Over 200 different tags were used to cover the various open source projects.

Most viewed projects

What were the most popular projects that readers found interesting in this newsletter? I thought I would share what have been the most popular projects this year (measured by folks that have clicked on the links). These are just too good to miss, so check this list out and let me know what you think. What were your favourite projects of 2022? What is missing that you are surprised about?

  • querypal the most viewed project, provides a nice WebUI for Amazon Athena
  • cfn-diagram is CLI tool to visualise CloudFormation/SAM/CDK stacks as visjs networks, draw.io or ascii-art diagrams
  • driftctl helps you detect, track and alert on infrastructure drift
  • infracost shows cloud cost estimates for Terraform
  • aws-sdk-client-mock provides AWS JavaScript SDK v3 mocks for easy unit testing
  • cfn_nag is a linting tool for CloudFormation templates
  • steampipe ise SQL to instantly query AWS resources across regions and accounts
  • keycloak is an open source Identity and Access Management solution
  • gnuradio is a free & open-source software development toolkit that provides signal processing blocks to implement software radios
  • eventcatalog helps you discover, explore and document your Event Driven Architectures powered by Markdown.
  • ddb_local provides a Python wrapper for DynamoDB Local
  • cloudquery-policies/aws is an open-source cloud asset inventory powered by SQL
  • kronicle is an open source tool and dashboard for documenting and visualising a tech stack
  • memq is an efficient, scalable cloud native PubSub system from Pintrest

Most viewed blog posts and articles

It was not just code that readers were interested in. Here is a list of the most viewed blog posts of 2022. If you missed them the first time around, here is your chance to catch up.

Feedback and looking forward to 2023

Thanks for reading this short post to recap 2022. As I look forward to 2023, I am thinking about what themes to cover, what open source technologies to focus on, and ensure that I work back from our customers.

To help I have a very short survey that I would like you to complete. The survey link is here, and completing the survey will give you the chance to win some SWAG in the Raffle. After completing the survey, you will get a Raffle ticket code. Please ignore the link below this saying you can redeem this in the AWS Console. These are not AWS Credit codes.

KEEP THE CODE SAVED

I will publish the winning code in a future edition of the AWS Open Source newsletter.

Top comments (1)

Collapse
 
adriens profile image
adriens

Hi @094459 I'm actively working on migrating from ELK to OpenSearch. Yesterday I had to to face the following request from the project Team :

❔ [Question] Please help going away from ELK "Ingest pipelines" by using Open Search 🏃 #5740

Context

  • We have migrated from ELK to OpenSearch for logs management
  • Have started to migrate to OpenSearch for data and dashboard projects

☝️ Still for a new project , the team came to use with a case which seems to be only implemetable with ELK, called Ingest pipelines

They could use this feature to strip multiple indexes on the fly into a new one...thanks to a common key.

To make short, with the folllowing two indices :

Index_1 :

customerId name
1 My name is John and my phoneNumber is
2 Your name is Bob and my phoneNumber is

Index_2 :

customerId phone
1 +687 00.00.00
2 +687 12.34.56

Then after operation ingestion (let's call that operator X), then...

Index_3 = Index_1 X Index_2 where Index_3 becomes :

customerId name phone
1 My name is John and my phoneNumber is +687 00.00.00
2 Your name is Bob and my phoneNumber is +687 12.34.56

🙏 Solution I would like

  • Know if this kind of feature is available in Open Search (it not, when)
  • If the feture exists : how is it called and how can this be achieved

😓 Current alternative you've considered

We currently load the data with Logstash into ELK

ℹ️ Additional context

Our main pattern is to :

  1. Consume data from Kafka
  2. Load it into ELK
  3. Perform live aggregation with ELK

Any idea on how to achieve this ?
Thanks you in advance for your help.

Best Regards... and have a great WE ;-p
Adrien