Finally I found a discussion I can contribute to! I love the books example below. If I can add to it a little:
now imagine that the library had an event for an author like JK Rowling. Everyone was wild to attend, including you, but you're locked outside. When the event starts and they open the doors, everyone rushes in. there's no priority, just "run for the door and shove your way through", but of course all the HP books are checked out right away. This is how I think of connection "storms" that result after a database node is down - there's a larger demand for connections than there is supply, and when the library is 'unavailable', those connections queue up outside
Next parallel... it turns out that just like freedom aint free, neither is your library book. There is some invisible quantity threshold. No one mentions this to you when you check out one book, or two, or ten, but when you breach the invisible limit, the books grow tiny hands with knives and kill you in your sleep. AFAIK databases require some amount of resources for connection management. If you're running an architecture with many thousands of connections (for example, microservices), then you could consume all the memory available to your instance and cause an OOM kill, especially if you dont close connections (return your books).
Finally I found a discussion I can contribute to! I love the books example below. If I can add to it a little:
now imagine that the library had an event for an author like JK Rowling. Everyone was wild to attend, including you, but you're locked outside. When the event starts and they open the doors, everyone rushes in. there's no priority, just "run for the door and shove your way through", but of course all the HP books are checked out right away. This is how I think of connection "storms" that result after a database node is down - there's a larger demand for connections than there is supply, and when the library is 'unavailable', those connections queue up outside
Next parallel... it turns out that just like freedom aint free, neither is your library book. There is some invisible quantity threshold. No one mentions this to you when you check out one book, or two, or ten, but when you breach the invisible limit, the books grow tiny hands with knives and kill you in your sleep. AFAIK databases require some amount of resources for connection management. If you're running an architecture with many thousands of connections (for example, microservices), then you could consume all the memory available to your instance and cause an OOM kill, especially if you dont close connections (return your books).
Expanded my view greatly.