DEV Community

Discussion on: Lend your voice to an episode of the DevDiscuss podcast by telling us your WORST on-call horror story 😱

Collapse
 
oraclesean profile image
Oracle Sean ♠️

Friday afternoon on Labor Day weekend, 2003-ish, getting ready to enjoy a long weekend getaway with my family, a tech in an adjacent cage at the CoLo spilled a Big Gulp into our SAN, wiping out 8 databases. It also shorted out the tape carousel. We borrowed a tape drive—a manual unit—and spent the weekend running restores, serially. I had it easy, all I had to do was sit at home and call the sysadmin to let him know when to switch the tapes. The sysadmin spent the weekend at the CoLo, wrapped in a blanket.

We finished around 4PM on Monday and I had just sat down on the couch and turned on the TV when my wife and kids got home... "Is this what you've been doing all weekend? Watching TV??"

Collapse
 
graciegregory profile image
Gracie Gregory (she/her)

Oh my goodness!!