Should I finish loading lazy images while the browser is idle?

I've heard of JS packages that prefetch linked pages while the browser is idle.

Would there be any downside to doing something similar with lazily-loaded images?

At work, I'm on a fast connection. The Crutchfield home page loads very quickly, partly because below-the-fold images are lazy loaded.

But when I scroll down, even though I've been idle a few seconds, lazy-loaded images still "snap" into place. Not an ideal experience.

Shouldn't idle time be used to complete the page load?

Questions that come to mind:

1. How do I determine "idle"? requestIdleCallback, setTimeout(loadImages, 5000), etc.?
2. How do I performantly order the images to be eagerly loaded? Slowly expand the margin of an IntersectionObserver? (Is that even possible?)
3. How will this interact with the new loading attribute? Would I simply adopt the same strategy, just setting loading="eager" instead of changing the src attribute?

I'd love to know your thoughts, especially if you've implemented this - or explicitly chosen not to implement it for some reason.