Discussion on: How to import a big delimited datatable into AWS DynamoDB without opening your piggy bank?

View post

Replies for: Okay so 5 to 6 Million, isn't that much. Solution 1: Split your CSV into 6 smaller once so that they are 1 Mil each. Create an Athena table. I...

Do you have a working example of this ? I am currently trying the same thing.

Rehan van der Merwe • Nov 1 '19 • Edited

I will DM you, just follow me so that we are connected. I don't have an example ready, but I quickly found this stackoverflow.com/a/33755463 with a quick search. Just change the async.series to async.parallelLimit, and make it say 25 as well so then it will do 25*25 = 625 concurrent writes.

Rehan van der Merwe • Nov 11 '19

I wrote a post on how to do this for you/others -> rehanvdm.com/serverless/dynamodb-i...