A software engineer’s favourite command? — live checking the incoming HTTP requests

disclaimer: if you are not new to bash the ending of this article might surprise you :)

I had an EC2 instance with the Apache HTTP server installed for a static site hosted in a far-faraway region.

Latency was not an issue for the site, but it sometimes took up to 600ms to load, and this still is not that much, but the client was planning on uploading more photos and videos to attract customers from across the globe.

I wanted to leverage the CloudFront service to cache the content closer to the customer for faster asset delivery but also relieving the database from unnecessary requests, thus shrinking the costs.

Before the CloudFront was in place, the network showed that the content was delivered in about 600ms, after setting up the CloudFront we got down to 150ms. It was a huge difference, and the client was happy. We made a good baseline for their site that when the client and customers would upload the photos and videos. The content would now be accessible without the need to worry about latency. Making this architectural change would also mean that we are not keeping the database busy with “read”-requests. Talk about all the free space for the DB writer to do its work now :)

I believe the CloudFront should be a default service to add in such situations. It is not difficult to set up and it adds a lot of benefits by means of caching for faster access, assets availability, and also for cutting down on trips between the customer and database.

Have you tried setting up CloudFront?
Do you have shell access and want to view the HTTP requests as they come to your site?

SSH to your Amazon Linux 2 and locate the apache logs by running:

$ cd /var/log/httpd/

and then:

$ tail -f access-log

(Tail to output the last part of files and -f to “follow” along as requests “log lines” are written to the access-log file)

This should show every request that you get (live) on your HTTP server :)

Use tail before and after the CloudFront implementation and see how decreased the requests would be to the server :)

How was this article for you? Is there anything I can help out in this regard? Maybe there is something you’d like me to improve och deep dive into for the next article? Please, do let me know :)

also, @maher-rj is tail still one of your favourite commands? What other commands do you use that you can’t afford not-to-have? :)

— Saif;