Hosting Images with Storj and Cloudflare

For a while now I have been looking for a way to put images on my website. At first I just embedded them in the website github repository, but this just doesn't feel right. Putting one or two image assets in a codebase is one thing, putting an ever growing list of images in there feels icky to me. For this reason I put the last few cover images of my blog posts on the imgur platform. This is slightly cleaner from a git standpoint but now i have to trust imgur to keep serving these images. Additionally, as I recently discovered, this seems to be against imgurs TOS:

[...] Also, don't use Imgur to host image libraries you link to from elsewhere, content for your website, advertising, avatars, or anything else that turns us into your content delivery network.

Finally when I started indie-webifying my website, and was implementing the micropub protocol (which I will blog about at a later time), I decided that it was at the time to host the images on a platform that was meant to do that. I looked at a few storage providers such as cloudinary and S3 based object storage and landed on Storj.io, mostly because of the generous free tier, which should suffice for this little blog for quite a while.

One thing that bothered me slightly was that all storage providers I looked at charge for traffic. It's not the fact that it's an additional expense (if your not in the free tier anymore) that bothers me, but the fact that I don't have any control over how much this will cost me. In all likelihood this will never cost me anything since this blog has not much traffic, but if a post were to go viral (one can dream...), this could result in a surprise bill at the end of the month.

To help with the traffic costs I decided to try to use the free CDN functionality of Cloudflare to reduce the traffic to Storj. In this blog post I will describe how I did that.

Is this the right solution for you?

If you are in a similar situation as me, and just want to have somewhere to host your images for a personal website or to share images or screenshots as links while still having control over all your data, this could be a good solution.

If you want to build a robust image pipeline with resizing and image optimization, or you are building an enterprise website this is probably not the right way. You should take a look at cloudinary or one of the big cloud providers.

Prerequisites

To use Cloudflare as a CDN, you need to have Cloudflare setup as your DNS host for the domain you want to serve the images from. Even if you just want to use a subdomain like media.example.com, the whole example.com domain needs to be on cloudflare. For me this was not much of an issue, I followed the instructions from cloudflare and pointed the nameserver of my domain to cloudflare. Although I did have an issue during the migration, which resulted in my website being down for two hours. But I'm pretty sure this was caused by my previous nameserver provider.

Setting up Storj & Cloudflare

I assume you already have an account at storj.io. The next step is creating a bucket for your images. A bucket is just a place for your files and folders to live in storj, just like in any other S3 compatible storage provider. (Actually there are no folders in storj and other S3 services, the folders are just prefixes of the filenames). When creating a bucket, make sure you save the passphrase securely, such as in your password manager. Whenever storj asks you for the passphrase, make sure you don't let storj generate a new one! Every new passphrase will create access to a new bucket.

The next step is installing the uplink cli. Follow the quick start tutorial to get an access grant. Remember to use the same passphrase from above. Now follow the next quickstart tutorial to add the bucket to the uplink cli. The file accessgrant.txt in the tutorial only contains the access-grant string that you got from the last step.

Finally we want to share the bucket so the images can be accessed from the web. For this you can run the following command:

uplink share --dns <domain> sj://<bucket>/<prefix> --not-after=none

Replace <domain> with the domain you want to serve the images from. In my case I use media.tiim.ch. Then replace <bucket> with the name of your bucket and <prefix> with the prefix.

As mentioned above, you can think of a prefix as a folder. If you use for example media-site1 as a prefix, then every file in the "folder" media-site1 will be shared. This means you can use multiple prefixes to serve files for multiple websites in the same bucket.

You will get the following output:

[...]
=========== DNS INFO =====================================================================
Remember to update the $ORIGIN with your domain name. You may also change the $TTL.
$ORIGIN example.com.
$TTL 3600
media.example.com IN CNAME link.storjshare.io.
txt-media.example.com IN TXT storj-root:mybucket/myprefix
txt-media.example.com IN TXT storj-access:totallyrandomstringofnonsens

Create the DNS entries in Cloudflare with the values printed in the last three lines. Make sure you enable the proxy setting when entering the CNAME entry to enable Cloudflares CDN service.

And that's it. All files you put in the bucket with the correct prefix are now available under your domain! :)

If this blog post helped you, or you have some issues or thoughts on this, leave a comment via the comment box below or via the comment box on my website.