Discussion on: An Engineer’s Rite of Passage

I don't have an interesting story to share, but here are some of my general tips for not breaking production. Hope some of these are helpful. I'm sure there are plenty more, feel free to share yours.

• It starts with writing good code. This can mean many things depending on the person and language, but my general rules are:
  • Have consistent styling or follow your team's style guide. I find this makes it easier to see when something is out of place during development.
  • Keep things simple and clear in your code. When problems arise, you may not be thinking straight. If your code is too confusing and unclear, this may only compound the problem. Other developers may have a difficult time helping get things online if they cannot decipher the code.
• Test locally, on a dev server, then on production.
  • Do not just run automated tests or test locally, but test on a development server if available.
  • Once your change is deployed, test it on production.
  • Test small changes too.
• Have someone review your code before going live with it.
  • Have them test it.
  • Make sure they actually review it and don't just give the go ahead.
  • Create some guidelines around this with your team if none exist.
• Don't write/run queries directly on production.
  • Write and test them locally or on a dev server. After running them in a testing environment, make sure the updated data looks correct in the final product.
  • If it is an update or delete statement, write a select version of the same query first. This will help ensure you are pulling in the correct data. This will also help in the next step.
  • BACK-UP THE DATA. If you are unsure how, this can be a simple select statement, copied to a spreadsheet, and uploaded somewhere (as opposed to leaving _temp tables cluttering the DB).
  • Again, have queries reviewed by someone before running them.
  • If you are new, you should not have production database access on your first day. If you are in this situation seek out senior members of the team to verify and help run queries with you.
• Do not push changes towards the end of the day or before the weekend.
  • Save yourself the trouble of having to scramble to fix something during your personal time or letting the problem continue while you are out of office.
  • Push things live in the morning while everyone is in the office.
• Don't beat yourself up over it.
  • Development is hard, every project has a lot of different things to worry about, it happens.
  • Learn from your mistakes and help future developers avoid them as well.