How <canvas> Saved the Day - Handling Large Images in the Browser

A client uploaded 28 images, each around 5800 x 9500 pixels and 28 MB in size. When attempting to display these images in the browser, the entire tab froze - even refusing to close - for a solid 10 minutes. And this was on a high-end machine with a Ryzen 9 CPU and an RTX 4080 GPU, not a tiny laptop.

Here’s The Situation

In our CMS (Grace Web), when an admin uploads a file - be it a picture or a video - we display it immediately after the upload finishes. It's great for UX - it confirms the upload was successful, they uploaded what they intended to, and it shows responsiveness of the UI for better feel.

But with these massive images, the browser could barely display them and froze up.

It's important to note that the issue wasn't with the upload itself but with displaying these large images. Rendering 28 huge images simultaneously and one by one was just too much. Even on a blank page, it's just... nope.

Why Did This Happen (Deep Dive)?

Anytime you display a picture in an <img> tag, the browser goes through an entire process that employs the CPU, RAM, GPU, and sometimes the drive - sometimes even cycling through this process multiple times before the picture is displayed during image loading and layout recalculations. It goes through calculating pixels, transferring data among hardware, interpolating algorithms, etc. For a browser, it’s a significant process.

The Brainstormed Solutions

We considered several options:

1. Resize Immediately After Upload and Display Thumbnail
   • Issue: Resizing could take too long and isn't guaranteed to be instantaneous, especially if the resizing queue isn't empty - results in not-a-responsive system.
2. Check File Sizes and Display Only Their Names for Large Files
   • Issue: This would detract from the user experience, especially for those who require greater control and checks of any media material (incidentally, those are often the same ones with large files).

Neither of these solutions felt usable. We didn't want to limit our admins' ability to upload whatever they needed, nor did we want to compromise on the user experience.

Lightbulb! Using <canvas>

I recalled that the <canvas> element utilizes the GPU much more effectively to render content. I recently saw someone building quiet complex game using JavaScript and <canvas>, so it should theoretically handle large images more efficiently than <img>.

Testing the Theory

We decided to replace the <img> elements with <canvas> for displaying the uploaded images. Here's the code snippet we used:

const canvas = document.createElement('canvas');
const ctx = canvas.getContext('2d');

const img = new Image();
// img.src = URL.createObjectURL(file); // Use this if you're working with uploaded files
img.src = filesroot + data.file_path; // Our existing file path
img.onload = function() {
  // Resize canvas to desired size (for preview)
  canvas.width = 80 * (img.width / img.height); // Adjust scaling as needed
  canvas.height = 80;
  // Draw the image onto the canvas, resizing it in the process
  ctx.drawImage(img, 0, 0, canvas.width, canvas.height);

  file_div.innerHTML = ''; // Prepared <div> in UI to display the thumbnail
  file_div.appendChild(canvas);
};

The Results

The improvement was immediate and dramatic! Not only did the browser handle the 28 large images without any hiccups, but we also pushed it further by loading 120 MB images measuring 28,000 x 17,000 pixels, and it still worked as if it was a tiny icon.

Where Is the Difference?

Using the <canvas> element, the drawing is entirely in the hands of the GPU, and those chips are literally made for this task.

• <img> still attempts to render 28,000 * 17,000 = 476,000,000 pixels.
• This <canvas> draws only 80 * 80 = 6,400 pixels.

By resizing the image within the <canvas> to a small preview size, we're drastically reducing the amount of data the browser needs to process - send hundreds of millions of pixels to GPU and get back thousands, a great deal, right?

Next Steps

Given this success, we're now considering replacing <img> with <canvas> in other parts of our application, particularly in admin screens that display around 250 images at once. While these images are already properly resized, we're curious to see if using <canvas> can offer even better performance.

We'll be conducting testing and benchmarking to measure any performance gains. I'll be sure to share our findings in a future article.

Important Caveats

While using <canvas> worked wonders for our specific use case, I must note that you should not use <canvas> for displaying images on public-facing parts of websites. Here's why:

• Accessibility: <img> elements are readable by screen readers, aiding visually impaired users.
• SEO: Search engines index images from <img> tags, which can improve your site's visibility. (In the same way, AI-training scrapers "index" the images, too. Well, what can you do...)
• <img> with <picture> can be used to automatically detect pixel density and show bigger or smaller pictures for a perfect fit—<canvas> can’t. I wrote an article on how to build your .

For public websites, it's best to properly resize images on the server side before serving them to users. If you're interested in how to automate image resizing and optimization, I wrote an article on that topic you might find useful.

Conclusion

I love optimization! Even though we are moving towards local high-performance hardware each day, by optimizing - even when "it's good, but can be better" - we could prevent issues like this from the beginning.

This approach is not standard. And I don’t care for standards very much. They are often treated as hard rules, but they are just guidelines. This solution fits. We do not have clients with ancient devices without solid <canvas> support. So, even though now we have more code to manage, multiple ways of displaying pictures, it’s a perfect fit, and that’s what matters!

Do you have some similar experiences? How did you handle it? Feel free to share your experiences or ask questions in the comments below!