DEV Community

Cover image for Recent 'Panic Moment'? staff for The DEV Team

Posted on

Recent 'Panic Moment'?

Welcome to Code Chatter, your go-to series for conversational coding insights. What makes this series of questions different from all the others? Well, truth be told, not much, but they're still thought-provoking and fun. Join us as we explore the coding world, one witty question at a time.

Could you share a recent 'panic moment' during development, and how did you handle it?

Follow the DEVteam for more discussions and online camaraderie!

Top comments (6)

wraith profile image
Jake Lundberg • Edited

A few years ago, I built an A/B test for an enterprise level client. The test was making modifications to their SPA on one page, however, we were not aware that their checkout shared some of the same structure and classnames and we inadvertently removed their "Checkout" button, so user's couldn't complete their orders.

We quickly turned off the test, and I was able to fix it, but that was definitely one of the bigger panic moments because I had made a mistake and probably cost the company a good chunk of money...

Afterward, we had a long conversation about what had happened. There were many things that lead up to this, like:

  • no time was baked into the onboarding for us to familiarize ourselves with the client's site.
  • the PM had agreed to a deadline before talking with the team, and so we would meet that deadline, they had us skip QA where something like this would have most likely been caught.
  • I had been so focused on the timeline, that I didn't take the time to test core functionality of the site.

Moral of the story, there is a process for a reason, and if you rush that process, or skip steps, eventually something will go wrong.

ksolomon profile image
Keith Solomon

Most recent turned out to not be something I needed to sweat. Two of our most major clients called in reporting server not found errors on their site. Pull up both, sure enough, nothing. Start going back through all the changes we’d done recently, nothing would have caused this, much less in two unrelated sites.

Check the DNS records, and there’s NOTHING for either site…no A records, no MX, no cnames, nada…both sites completely wiped off the net. Panic mode sets in HARD through the entire department. Phone calls were made, no one has any clue what’s going on.

Turns out, the two “unrelated” sites were in face related. Both sites’ DNS were managed by the IT department of the provincial government…which was suffering a “network incident”, later determined to be a DDoS against their network, that took out the DNS servers.

We all went directly to the bar after work…

taikedz profile image
Tai Kedzierski • Edited

Most recent? Just now.

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: "Could not find ''"', src/lenker/
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Enter fullscreen mode Exit fullscreen mode
srbhr profile image
Saurabh Rai

Facing one right now. Deployed project to QA. Without getting approval from the team members.
Now only now half of the features are moved. Dev team will be getting some amazing and interesting defects 😂😂

sidswirl profile image
Sid Probstein

I pushed the EPO button in the server room, thinking it was the light switch...

mitchiemt11 profile image
Mitchell Mutandah

Pushed a silly typo to prod.