re: Coding Sidekiq Workers the Right Way VIEW POST

FULL DISCUSSION
 

I need to process large(size and number) CSVs (couple of MBs having more than 0.1 million records each). Each record in the CSV has to be processed sequentially.

There are lot many relations comes into picture and so I have use memoization a lot to reduce db calls.

So in such cases, I cannot use the technique mentioned in point no 2.

 

Load raw-ish data into a work table and then process in the database?

code of conduct - report abuse