Background processing in .NET Core: Azure Functions

In this article I'll talk about my experience with Azure Functions. Instead of showing you how to do a "hello world in Azure Functions" , I'm going to talk about some scenarios where I used them and what I learned from this. So in this article I'll tell you how I created a small project one night that gained hundreds of thousands of users by next morning. We'll see if Azure Functions lived up to the challenge and if they are as cheap as they are advertised.

Bogdan Bujdea

@thewindev

Took 30 minutes to write and deploy an #AzureFunctions project that processes partial results for Romanian elections...now let's see how it will hold up with the flow of people eager to see the results #europarlamentare2019 bit.ly/rezultateprovi…

22:39 PM - 26 May 2019

9 51

I'll break this down in the 3 parts, so it's easier for you to navigate:

1. The story - I'll talk a bit about the project so we'll understand what it does and why I did some things in a certain way
2. Azure Functions
3. Conclusion - we'll see what I learned and what would I change

The story

It was May 26th, 2019, and in Romania it was election day for the European Parliament. After 10 PM the results started to appear on the official government website, but they were stored in csv files. The process was like this:

• each polling station sends their results to the central election committee
• they add this as a line in a CSV file
• people could download this CSV file and see the latest results

This means that if you wanted to see the stats for the counted votes up to a certain time, you would have to download the CSV and sum up the columns for each candidate. Then you had to do this each time a polling station sent in their votes so you would have a live update of the results. Of course, I did this once, I did it again after 10 minutes... but then I said to myself "why the hell am I manually downloading a file when this could be done in a few lines of code?". So I started working on this.

First, there was a console app. Each time I run it, the CSV is downloaded, the results are aggregated and displayed in descending order. Great, now I have this tool for myself, but there are 18 million voters out there who would be interested in the results as well as I am. So how could I share this with them in the easiest way possible?

Azure Functions

In one of the talks I've given two years ago, I said that Azure Functions are great for prototyping. You could just create a function with the piece of code that you need and deploy it in Azure, so this is what I did. I took the code from my console app and put it in a function. Then, I just formatted the results as HTML and returned them from each HTTP call. Here's how it worked:

This is actually the latest version. When I first deployed this I had only the HTTP function that would download the CSV file for every request. This is not a good idea when you have this many users, so I created another function that would download the file every 5 minutes and store the results in Azure Table Storage. Then, for each request I got from the users I would take the latest results from the table and sent them back as HTML.

I know, the code is ugly, the UI was ugly as well, but it got the job done and this is what mattered.

As soon as the function was live and functional, I posted the link on Facebook.
The word got out and soon I was having users hitting the endpoint and seeing the results. After a while I got from a few users to a few thousand users per second, and this continued for about a day.

PS: In case you're curious and you really want to see that page, the function is still up at this address, and this is what the project evolved into

What went well

Easy to publish

I liked how easy it was to deploy my code from a console app in an Azure Function. I just created the function, modified the code so it will return an HTML file with the results, and then I used the Visual Studio option for publishing a project. From there you're a few clicks away from seeing your function live.

Auto-scaling

The function scaled pretty fast when people started to use it and I enjoyed the fact that I didn't had to do anything special to make this work. This was just the built-in behavior and I just sat there and watched as it created more servers or killed them according to the number of users.

Free

The function didn't charge me anything because I was under 4 million executions, and back then this was the limit.

What went bad

Slow deployment

A few hours after users started pouring in, I did a deploy to change the name of a candidate. The deploy is usually very fast but now I had tens of thousands of users hitting F5 in their browsers at the same time, so Azure couldn't create new servers that fast. It took ~10-15 minutes until the function managed to deploy enough servers (over 100 from what I remember) and I stopped seeing errors.

Not really free

The function was free, but logging and storage are not. I enabled Application Insights to see more stats about the function and in a few days the cost was about ~40 $ for insights and ~10 $ for Azure Table Storage. So yes, the functions are cheap, even free, but additional services will incur costs and you have to pay attention if you need them. Microsoft is not actually selling Azure Functions, they're selling additional services that go along with them, remember this :)

What would I change?

Well, I would be more careful on what services I decide to use with my functions. Application Insights provided a lot of metrics but it was too much for my needs, so if I had to cut costs, I would start with this. Also, I kept every snapshot of the results(meaning a csv file) in Azure Table Storage. It would've been easier to keep only the latest version, that would incur less costs.

Overall, I think this was a perfect scenario for using Azure Functions. I needed something fast, easy to deploy, and cheap. Azure Functions checked every item on the list and I would use this again if I had to.

This doesn't mean you have to use Azure Functions everywhere you can, but only everywhere you think it's a match. Sometimes developers have the tendency to use every new technology as a hammer and every requirement as a nail. Don't be like that! There are alternatives to Azure Functions like scheduling libraries( I just wrote about Hangfire , Quartz.NET), or even hosted services from .NET Core. The last two I'll cover in my next articles from this series, so if you want to get notified just subscribe to my newsfeed or follow me on Twitter

PS: If you want to see something cool done with Azure Functions, I would recommend the presentation below made by Sander Molenkamp at dotnetdays.

The post Background processing in .NET Core: Azure Functions appeared first on thewindev.net.