I don't use an IDE for two main reasons:
My experience with IDE's is that they allow inexperienced or incompetent developers to get very far with autogenerated code, refactoring, etc without understanding what's going on. This was especially true in previous .NET jobs I've had. Devs became very dependent on Visual Studio's refactoring and code generation tools that they struggled to operate when these tools were unavailable in other projects, languages, etc. This was from senior engineers who'd been maintaining several monstrous legacy C# .NET applications for years.
Bloat is definitely an issue in IDE's I've used. Most of my experience is in Visual Studio + .NET, and working in that editor was very frequently a pain just due to sluggishness. There's also just a lot more features and functionality in front of you that you might never touch. It was also common for project config/IDE files, etc to make their way into Git diffs which was another point of unnecessary information and bloat.
I've been very productive in VS Code for the last two years or so. Mostly working on JS and Ruby projects. I like having control over the amount of features in my environment through extensions. I can enable/disable extensions for specific workspaces, etc. I also have coworkers who are equally productive in Sublime and Atom.
I'd just like to say that, though you've framed this post as a discussion, you do come across as very evangelical and a bit militant in most of your comments. I've seen engineers with vast knowledge of their IDE's be very productive, and I've seen engineers ship impeccable and maintainable codebases with nothing more than VIM. There are certainly productivity boosts to be had with IDE's, but they can certainly come with trade offs.
We're a place where coders share, stay up-to-date and grow their careers.
We strive for transparency and don't collect excess data.