Had a similar one. I had just rewritten a bunch of cron jobs that send time sensitive emails.
There was one cron job that was supposed to run every minute and check the status of a bunch of records that where created 20 minutes before.
When I was testing this little script I changed the time span where the record needed to be created from 1 minute to something like three days and I forgot to change it back before deploying.
When we deployed the change everything went fine because the script wasn't supposed to run till the following morning at around 9am.
Well the morning after I got to the office and everybody was going around crazy because some customers had already received ~30 mails each. I ran to my computer and started killing the container that was sending the emails before going through the source code and finding where the problem was.
I sent around 10k emails in 10 minutes.
Never have I ever wrote a cron job without a unit test after that.
Had a similar one. I had just rewritten a bunch of cron jobs that send time sensitive emails.
There was one cron job that was supposed to run every minute and check the status of a bunch of records that where created 20 minutes before.
When I was testing this little script I changed the time span where the record needed to be created from 1 minute to something like three days and I forgot to change it back before deploying.
When we deployed the change everything went fine because the script wasn't supposed to run till the following morning at around 9am.
Well the morning after I got to the office and everybody was going around crazy because some customers had already received ~30 mails each. I ran to my computer and started killing the container that was sending the emails before going through the source code and finding where the problem was.
I sent around 10k emails in 10 minutes.
Never have I ever wrote a cron job without a unit test after that.
Wait...cron jobs have unit tests?
Yours don't?