DEV Community 👩‍💻👨‍💻

Discussion on: How do you merge millions of small files in a S3 bucket to larger single files to a separate bucket daily?

Collapse
kishanbsh profile image
Kishan B

whitfin.io/quickly-concatenating-f...

This tool does the job of concatenating within S3 without requiring a download.. It uses the multipart api of S3 to achieve the same.