How I almost maxed out my 10GB monthly limit of downloads from my database in 2 days

TL;DR: Too many downloads were being made from my database (due to ignorance and many bad decisions) which almost maxed out my free monthly limit of 10gb in only 2 days.

Hello world! This is my first post so I'm a bit nervous, but here's a funny story I thought I'd start things off with. I had just delved into using Firebase, so I was very ignorant on a lot of things.

Recently I've been working on a blog app. In the past I had made a very minimal blog app following a Udemy course, but this time I wanted to start from scratch and use some new tools I had acquired since then. I had just finished implementing a feature that allows each user to upload their own profile picture that is displayed throughout the site.

Since I was already using Firebase's Realtime Database to store some data, I decided to just store the images in there as well temporarily. This was my first mistake. My second mistake was sharing a link to the app with my family, thinking it would help me see if the site was user-friendly. Be careful what you wish for 🤷🏻

A little background on how my app is set up.

• A homepage that displays all posts.
  • Each post consists of a title and body, and shows the author's avatar.
• A page that allows you to create a new post.
• A profile page that displays a user's posts along with a bigger image of their avatar. When logged in, your profile has a link that lets you edit your username and avatar.
• A navigation bar that displays the current users avatar.

So I received a warning email from Firebase on Sept. 10th telling me that my Realtime Database was at 8.8gb, and that I only had 1.2gb worth of downloads left for the rest of the month.

Here's the two main factors that caused this:

• I was storing the user avatars in base64, which lead to bigger file size. This meant downloading it from the database took longer and ate up more of my allotted downloads.
• Each time an avatar was being displayed, I was making a new fetch to the database, and not storing it anywhere. So each time a user visits the home page, new call is made to the database to download all the avatars that are displayed on each post.

Lets say each avatar is originally 170kb, which is uploaded to the database as base64, so the image size is now 225kb. That doesn't seem too bad. But if there's 30 posts on the homepage, 30 avatars are being downloaded from the database, so 30 * 225 = 6750kb. Still doesn't seem too bad, but lemme give you an example to find out how I tortured the Realtime Database.

Let's follow a typical user as they sign up and navigate my app.

A user visits my site for the first time. They are sent to the homepage, where there are 30 posts and 30 avatars are downloaded to be displayed on each post. That's 6750kb worth of downloads. This user decides to create an account, which redirects them back to the home page afterwards. 6750 + 6750 = 13,500kb. They go to their profile page to change their profile pic, then decide to check out the homepage feed again, and all 30 avatars are downloaded once again. 13,500 + 6750 = 20,250kb. They decide to write a post, and are redirected to the homepage afterwards. Again, +6750. Our total is now 27,000kb, which is 0.027gb. This still doesn't seem too bad, but don't forget that this is just one user.

If there are 20 people signed up on my app, and they all are navigating my app like in my example, that's 20 * 27,000kb = 540,000kb. Not to mention that's only one session! So if all 20 people visit my app 4 times per day (that's wishful thinking 🥺), that's 2,160,000kb, or 2.16gb in one day!! Don't forget that this whole example is assuming the images are all 225kb each. 😱 Thankfully my dear family decided to make multiple accounts and upload huge HQ images haha.

But most importantly, since I'm working on this app all day everyday, I'M the one using it the most. 😭 At the moment I don't have separate databases for development/production. In addition I'm using webpack-dev-server, so the app was refreshing each time I saved a file. All the while making real calls to the database to make downloads.

Anyway, here's how I solved this problem 😊

• Most importantly I'm now using Firebase's Storage for images, and the Realtime Database is solely for storing usernames and post info.
• I'm no longer storing the image in base64, instead using the original files format.
• Once an avatar is uploaded, I'm storing it's Firebase Storage URL in Redux, so the image is downloaded from the Redux store the next time it needs to be displayed.

This was already enough to decrease the downloads, but there's a few more features I'm working on that will greatly help improve things.

• Allow users to crop their image, and then a step further would be somehow compressing the image. Only then will it upload to Storage.
• Persisting the Redux state to local storage. If you visit the homepage, all the avatar URLs are downloaded and saved to the Redux store. This only works during one session. If you refresh the page or visit my app again another time, the store is reset, and all the URLs need to be downloaded from the database again. Saving the state in the browser's local storage fixes this. The next time you visit my blog app, the browser remembers what data has already been downloaded.

So anyway, that concludes my first big production bug. Rushing to do damage control so my database wouldn't be shut down for the rest of the month. 😅 Thankfully, ever since I switched to Firebase Storage and storing the image URLs in Redux/local storage, my Realtime Database still shows that I have 1.2gb left. Hopefully this means that my app will still be up and running for the rest of September!