This sounds horrific. Why are you deploying at 3am if it's possible to fail - what's the mission critical item in the deployment?
Guards - well, this is called an 'on call' duty from where I've been. There was a small monetary compensation from going on the rota. The trick is to make sure that your code can't break.
This code that's keeping you awake - did you write it? Did you test it? Is the deployment automated? Does the automated deployment run automated tests? It's stuff like that that let's me sleep at night.
we are deploying at 3am because we have less traffic at that time.
my teams apps doesn't crash regularly, but, external services or other teams APIs does, so, i usually have to contact the other team and ask what is happening and get some info about the issue, the same to external service.
I work in a +1500 devs company, we follow standards, ci/cd, but, shit happens 💩
Blue/Green, rolling, canary - these are deployment strategies you want to look into.
If you have a CI/CD pipeline, and it is being used correctly, there is only very few reasons to not have an constant deployment process.
We're a place where coders share, stay up-to-date and grow their careers.
We strive for transparency and don't collect excess data.