Two stories:
When I've served in the army a not-so-funny story happened to one of my friends:
He played with his M16 because he was bored (It was the year 2000. no smartphones...). And in some weird way he ended up having his gun loaded without him knowing it.
So how do I know it?
Because an officer checked his gun... (lucky for us all).When working with SQL SERVER a friend of mine (it's always friends. never me...) worked on his local copy of the database.
Or that's what he thought he was doing.
Actually he had two tabs opened inside SSMS:
One representing his local enviroment.
One representing our production.
Can you guess what happened?
You are probably right:
The production database was deleted.
Well let's just FF 3 hours later when all the managers came back from home because nothing worked. And although it was on SQL AZURE which has great backup, he somehow managed to do all the steps which avoids the normal way of restoring to point in time.
We were so helpless that we tried to restore the data from the cache into the DB!
Finally the guys from Azure came to our help and gave us what we needed.
What did I learn from this?
Seems like less then I thought.
Because yesterday it almost happened to me again!
It's so easy to get into these places.
The only way is DISCIPLINE.No fancy tool will save you if you don't use it.
You set rules, and you understand deeply why they exist (In some proffessions they say: "Regulations are written in blood").
Try not to hold a loaded rifle, and if you must hold it- at least be aware of it.
Top comments (11)
Umm, that's why you set up permissions on production. And although I'm a developer... never give developers production rights.
In the world of devops, developers typically do have production rights. Still, it shouldn't be that easy to access them in the way OP mentions.
there's a difference between a developer having access to a production environment account that has production rights vs. giving the normal developer's normal account that he/she uses every day production rights.
I don't use sql server but there's definitely something fishy about the above situation. And this statement is straight up false:
Relying on developers to "be careful" is how that database got deleted in the first place. You are supposed to build systems with the expectation that humans will screw up, no matter how disciplined your team is.
Exactly. Security 101.
Lel, would you like to say that you even don't fix bugs right on production servers via FTP?
Well, that depends.
If it's a client side issue - I prefer clicking F12 and changing through the dev console :-)
This is why for my Azure databases and others I've setup unique and separate admin credentials so there is only one user I can connect with for each environment. I also only use the Registered Servers option in Sql Server Management Studio and have my production servers color coded in red so it's easier to tell when I'm in the danger zone. I've done the same with my virtual machines where the desktop background is red for the production server. But I agree it takes discipline to not only set this up but to use it and not take shortcuts occasionally.
Some of the most important lessons we learn are also the more painful ones. :)
By no means idiot-proof (since they'll always build a better idiot), but, to help me avoid being the source of tragedy:
While it doesn't cover all use cases one trick I've learned over the years is to setup different color schemes and prompts between your environments.
For example, all my local terminal windows are a dark theme. All of my production terminals are a light theme. This visually helps remind me I'm somewhere different.
The terminal prompts also change to include the environment itself. They are prefixed with EDGE, STAGING, and PRODUCTION. Colors change too. EDGE is in green, STAGING is in yellow, PRODUCTION is in red. You can do the same thing for irb/pry/rails-console prompts.
Even the website itself. For EDGE and STAGING, throw a banner/blammy across the top in a way that it doesn't affect the layout (float it on top something). Some bar with enough color and labeling to help remind you where you are. If you don't see that, you're in PRODUCTION.
None of this would stop me from doing something stupid, but it helps keep me aware of just where I am.
i like the metaphor
Some comments may only be visible to logged-in visitors. Sign in to view all comments.