I increased my write performance to a mysql database from 48s down to 3s writing 100,000 records at once.
I am no DBA, but I got curious when I noticed sluggish write performance on a mysql database, and I remembered somewhere that you should always use batch writes over sequential writes (when you have a lot of writes to be dine at once)
So I decided to test it out, using a python script and a mysql server.
For the sequential writes, I generated random data and in the for loop, I made a write to the mysql database every time I generated data for the record.
For the batch write, I defined a list/array and for every record that I generated, I appended the record to the list and once I exited the loop, I write to the database using executemany
and including the list.
The scripts I used, the results and more info can be viewed on my blog:
Top comments (0)