If you want to download multiple files as zipped from AWS S3, and you have your server to take care of the file metadata handlings, then this article may help you understand how the process works. AWS doesn't provide the default multi-files download, so in order to achieve you may want to add lambda function or to use your own implemented service. This article uses the custom service to support multi-files download function.
To know the basic setup of the code (AWS keys, the meaning of Key in the code, bucket, etc.), please refer to another article that I worte AWS SDK with Javascript: Download File from S3.
import * as AWS from 'aws-sdk';
import archiver from 'archiver';
import path from 'path'
import { PassThrough } from 'stream';
const s3bucket = new AWS.S3({
accessKeyId: process.env.AWS_ACCESS_KEY,
secretAccessKey: process.env.AWS_SECRET_KEY,
signatureVersion: 'v4',
region: process.env.AWS_REGION, // ex) us-west-2
});
Here is the main function that allows to receive file infos as an input parameter and to create stream object that can be used to write or pass the data as zipped format.
const multiFilesStream = (infos) => {
// using archiver package to create archive object with zip setting -> level from 0(fast, low compression) to 10(slow, high compression)
const archive = archiver('zip', { zlib: { level: 5 } });
for (let i = 0; i < infos.length; i += 1) {
// using pass through stream object to wrap the stream from aws s3
const passthrough = new PassThrough();
s3bucket
.getObject({
Bucket: process.env.AWS_BUCKET_NAME,
Key: path.join(infos[i].path, infos[i].filename);
})
.createReadStream()
.pipe(passthrough);
// name parameter is the name of the file that the file needs to be when unzipped.
archive.append(passthrough, { name: infos[i].filename });
}
return archive;
};
const files = [
{path: '/drive', filename: 'file1.jpg'},
{path: '/drive', filename: 'file2.jpg'},
{path: '/drive', filename: 'file3.jpg'},
{path: '/drive', filename: 'file4.jpg'},
];
const mfStream = multiFilesStream(files);
// res is the response object in the http request. You may want to create your own write stream object to write files in your local machine
mfStream.pipe(res);
// use finalize function to start the process
mfStream.finalize();
Top comments (6)
Hey, thanks a lot for the example. I was able to run the code on my own app but the only issue I am having is that I can't find where the zip file is being saved on my local.
I tried to add
.createReadStream(__dirname + '/example.zip')
but I couldn't find it.Please look my code below and see if you can help me, it is the last part I need to finalize my project. Thanks a lot for the help
I call the "listTst" on my node app from my react app.
I can see the 2 files being downloaded on the network tab of my browser but no zip is saved.
Thank you for this great article. It is really helpful for me.
Hey thanks for posting this amazing solution. I was getting an error (0, archiver_1.default) is not a function. Would you care to help me out resolve it???
Hey thanks, I figured it out long ago. I was developing an app and needed this file archiving and download feature. I used this archiver utility for my app dockefy. Take a look and let me know if you like it at dockefy.com
Hi, I had such a feeling...)
I left that comment in case someone might run into the same issue)
dockefy.com looks awesome, great job!