DEV Community

Sergey Telpuk
Sergey Telpuk

Posted on • Updated on

Nestjs(fastify, multer). Uploading & cropping image.

Hello friends!

At this article, I wanna show you how we can download and crop image with the help of Multer and in the same time to use different adapters, for instance, I'm gonna show two adapters FTP, AWS.

In the beginning, adapt Multer to Nest, main.js can look bellow:

import {NestFactory} from '@nestjs/core';
import {AppModule} from './app.module';
import {INestApplication} from '@nestjs/common';
import fs from 'fs';
import {FastifyAdapter, NestFastifyApplication} from '@nestjs/platform-fastify';
import fmp from 'fastify-multipart';

(async function bootstrap() {
    const fastifyAdapter = new FastifyAdapter({
        http2: true,
        logger: true,
        https: {
            allowHTTP1: true, // fallback support for HTTP1
            key: fs.readFileSync(APP.HTTPS_SERVER_KEY),
            cert: fs.readFileSync(APP.HTTPS_SERVER_CRT),
        },
    });

    fastifyAdapter.register(fmp, {
        limits: {
            fieldNameSize: 100, // Max field name size in bytes
            fieldSize: 1000000, // Max field value size in bytes
            fields: 10,         // Max number of non-file fields
            fileSize: 100,      // For multipart forms, the max file size
            files: 1,           // Max number of file fields
            headerPairs: 2000,   // Max number of header key=>value pairs
        },
    });
    const app: INestApplication = await NestFactory.create<NestFastifyApplication>(
        AppModule,
        fastifyAdapter,
    );
    app.enableCors();

    await app.listen(APP.PORT, APP.HOST);
})();

Now, Nest can parse multipart/form-data.

After add some abstraction for our storage.
Create UploadImageFactory:

export const UploadImageFactory: FactoryProvider = {
    provide: 'IUploadImage',
    useFactory: () => {
        return StorageFactory.createStorageFromType(TYPE_STORAGE);
    },
};

IUploadImage looks like:

export interface IUploadImage {
    /**
     *
     * @param value
     */
    setFilename(value: string);

    /**
     *
     * @param value
     */
    setCroppedPrefix(value: string): IUploadImage;

    /**
     *
     * @param value
     */
    setCroppedPayload(value: CropQueryDto): IUploadImage;

    getMulter(): any;
}

StorageFactory looks like:

export class StorageFactory {
    static createStorageFromType(type: string): IUploadImage {
        switch (type) {
            case TYPE_STORAGE.FTP:
                return new FtpStorageAdapter({
                        fileFilter(req, file, cb) {
                            //TODO validate files on mime-type

                            cb(null, true);
                        },
                    },
                );
            case TYPE_STORAGE.AWS: {
                return new AwsStorageAdapter({
                    fileFilter(req, file, cb) {
                          //TODO validate files on mime-type

                        cb(null, true);
                    },
                });
            }
            default:
                return null;
        }
    }
}

Create the first adapter for FTP:
FtpStorageAdapter

import FtpStorage from 'multer-ftp';
import fs from 'fs';
import path from 'path';
import {Options, StorageEngine} from 'fastify-multer/lib/interfaces';
import multer from 'fastify-multer';

export class FtpStorageAdapter extends StorageAbstract implements StorageEngine {

    private readonly storage;
    private readonly storageForCropping;

    constructor(options: Options | undefined) {
        super();

        this.setMulter(multer(
            {
                ...options,
                storage: this,
            },
        ).single('file'));

        this.storage = new FtpStorage({...FTP_STORAGE});
        this.storageForCropping = new FtpStorage({...FTP_STORAGE});
    }

    async _handleFile(req, file, cb) {
        const filePath = await this.saveAsTemp(file);

        await this.resize(filePath).then((resizedFile) => {
            this.storageForCropping.opts.destination = (inReq, inFile, inOpts, inCb) => {
                inCb(null, this.croppedPrefix + this.filename + path.extname(inFile.originalname));
            };
            this.storageForCropping._handleFile(req, {
                ...file,
                stream: fs.createReadStream(resizedFile as string),
            }, (err, destination) => {
                if (err) {
                    Promise.reject(err);
                }
                Promise.resolve(true);
            });
        });

        const storage: any = await new Promise((resolve, reject) => {
            this.storage.opts.destination = (inReq, inFile, inOpts, inCb) => {
                inCb(null, this.filename + path.extname(inFile.originalname));
            };
            this.storage._handleFile(req,
                {
                    ...file,
                    stream: fs.createReadStream(filePath as string),
                },
                (err, destination) => {
                    resolve(() => cb(err, destination));
                });
        });

        this.reset();

        storage();
    }

    async _removeFile(req, file, cb) {
        this.reset();
    }
}

Create the second adapter for AWS:
AwsStorageAdapter

import AwsStorage from 'multer-s3';
import fs from 'fs';
import AwsS3 from 'aws-sdk/clients/s3';
import path from 'path';
import {Options, StorageEngine} from 'fastify-multer/lib/interfaces';
import {StorageAbstract} from '../storage.abstract';
import multer from 'fastify-multer';

export class AwsStorageAdapter extends StorageAbstract implements StorageEngine {

    private readonly storage;
    private readonly storageForCropping;

    private readonly AWS_CONFIG = {
        s3: new AwsS3({
            credentials: {
                accessKeyId: AWS_STORAGE.AWS_ACCESS_KEY_ID,
                secretAccessKey: AWS_STORAGE.AWS_SECRET_ACCESS_KEY,
            },
            s3ForcePathStyle: AWS_STORAGE.AWS_S3_FORCE_PATH_STYLE,
            s3BucketEndpoint: AWS_STORAGE.AWS_S3_BUCKET_ENDPOINT,
            endpoint: AWS_STORAGE.AWS_ENDPOINT,
        }),
        acl: AWS_STORAGE.AWS_ACL,
        bucket: AWS_STORAGE.AWS_BUCKET,
    };

    constructor(options: Options | undefined) {
        super();

        this.setMulter(multer(
            {
                ...options,
                storage: this,
            },
        ).single('file'));

        this.storage = new AwsStorage({
            ...this.AWS_CONFIG,
        });

        this.storageForCropping = AwsStorage({
            ...this.AWS_CONFIG,
        });
    }

    async _handleFile(req, file, cb) {

        const filePath = await this.saveAsTemp(file);

        await this.resize(filePath).then((resizedFile) => {
            this.storageForCropping.getKey = (inReq, inFile, inCb) => {
                inCb(null, this.croppedPrefix + this.filename + path.extname(inFile.originalname));
            };

            this.storageForCropping.getContentType = (inReq, inFile, inCb) => {
                inCb(null, inFile.mimetype);
            };

            this.storageForCropping.getMetadata = (inReq, inFile, inCb) => {
                inCb(null, {fieldName: inFile.fieldname});
            };

            this.storageForCropping._handleFile(req, {
                ...file,
                stream: fs.createReadStream(resizedFile as string),
            }, (err, destination) => {
                if (err) {
                    Promise.reject(err);
                }
                Promise.resolve(true);
            });
        });

        const storage: any = await new Promise((resolve, reject) => {
            this.storage.getKey = (inReq, inFile, inCb) => {
                inCb(null, this.filename + path.extname(inFile.originalname));
            };

            this.storage.getContentType = (inReq, inFile, inCb) => {
                inCb(null, inFile.mimetype);
            };

            this.storage.getMetadata = (inReq, inFile, inCb) => {
                inCb(null, {fieldName: inFile.fieldname});
            };

            this.storage._handleFile(req,
                {
                    ...file,
                    stream: fs.createReadStream(filePath as string),
                },
                (err, destination) => {
                    resolve(() => cb(err, destination));
                });
        });

        this.reset();

        storage();
    }

    async _removeFile(req, file, cb) {
        this.reset();
    }
}

StorageAbstract looks like:

import fs from 'fs';
import sharp from 'sharp';

export abstract class StorageAbstract implements IUploadImage {

    protected filename: string;
    protected croppedPayload: CropQueryDto;
    protected croppedPrefix: string;

    private multer: any;

    protected constructor() {
    }

    protected setMulter(multer: any) {
        this.multer = multer;
    }

    setFilename(value): IUploadImage {
        this.filename = value;
        return this;
    }

    setCroppedPrefix(value: string): IUploadImage {
        this.croppedPrefix = value;
        return this;
    }

    setCroppedPayload(value: CropQueryDto): IUploadImage {
        this.croppedPayload = value;
        return this;
    }

    getMulter(): any {
        return this.multer;
    }

    protected async saveAsTemp(file): Promise<string> {
        return new Promise((resolve, reject) => {
            const tmpFile = '/tmp/' + uuid4();
            const writeStream = fs.createWriteStream(tmpFile);
            file.stream
                .pipe(writeStream)
                .on('error', error => reject(error))
                .on('finish', () => resolve(tmpFile));
        });
    }

    protected async resize(file: string): Promise<string> {
        return new Promise((resolve, reject) => {
            const tmpFile = '/tmp/' + uuid4();
            let readStream = fs.createReadStream(file as string);
            const writeStream = fs.createWriteStream(tmpFile);

            if (Object.keys(this.croppedPayload).length !== 0) {
                const {cw, ch, cl, ct} = this.croppedPayload;
                readStream = readStream.pipe(sharp().extract({
                    left: cl,
                    top: ct,
                    width: cw,
                    height: ch,
                }));
            }

            readStream
                .pipe(writeStream)
                .on('error', error => reject(error))
                .on('finish', () => resolve(tmpFile));

        });
    }

    protected reset() {
        this.setFilename(null);
        this.setCroppedPrefix(null);
        this.setCroppedPayload({
            ch: 0,
            cl: 0,
            cw: 0,
            ct: 0,
        });
    }
}

Add IUploadImage into a contrived controller:

...
import {BadRequestException, Post,Req,Res} from '@nestjs/common';
...
    constructor(
        @Inject('IUploadImage')
        private readonly uploadImage: IUploadImage,
    ) {

    }
    @Post('/')
    async upload(
        @Query() avatarCropDto: CropQueryDto,
        @Req() req,
        @Res() res,
....
    ): Promise<void> {
        try {
...
             return new Promise((resolve, reject) => {
               this.uploadImage
                .setFilename(uuid)
                .setCroppedPrefix(croppedPrefix)
                .setCroppedPayload(cropPayload)
                .getMulter()(req, res, (err) => {
                    if (err) {
                        reject(err);
                    }
                    resolve(req.file);
                });
        });
        } catch (e) {
            throw new BadRequestException(e.message);
        }
    }

After those manipulations, We can not only download but also crop images.

Done repository.

Top comments (11)

Collapse
 
ruslangonzalez profile image
Ruslan Gonzalez

I was looking something about uploading to S3 AWS... thanks... I'll try following a little about your setup and hope will works. If you have recommendations would be much appreciated. My challenge is to just host files to AWS with some Multer Validations (I think it already do very good)...

Collapse
 
sergey_telpuk profile image
Sergey Telpuk • Edited

Hello, I can put forward using localstack/localstack:latest for trying it out.

Collapse
 
ruslangonzalez profile image
Ruslan Gonzalez

Actually I used part of your code and got AWS I integrated with my stack. I am appreciated.

Collapse
 
devzepto profile image
Nutchapong

Thank you so much for your guide. I have tested it out and it works great so far. The response from the controller is replied with the original image detail uploaded to S3 AWS. Is it possible to get the detail of both the cropped image and the original image as a response after finish uploading?

Collapse
 
aayani profile image
Wahaj Aayani

Couldn't get it running. How to export the UploadImageFactory to a module?

Collapse
 
sergey_telpuk profile image
Sergey Telpuk

Hi, You can do it like below:

import { Module } from '@nestjs/common';

export const UploadImageFactory: FactoryProvider = {
    provide: 'IUploadImage',
    useFactory: () => {
        return StorageFactory.createStorageFromType(TYPE_STORAGE);
    },
};

@Module({
  controllers: [],
  providers: [UploadImageFactory],
})
export class CatsModule {}
Collapse
 
aayani profile image
Wahaj Aayani • Edited

Thanks for your response. Figured that out earlier. I was somewhat trying to make it a dynamic module to be exported and used elsewhere in the code. I can see that you have hard-coded AWS config in it's adapter itself which is ofc not the best of practices, and now if you want to take it out of there and pass in from the module itself. How would you recommend doing that?!

Thread Thread
 
sergey_telpuk profile image
Sergey Telpuk

To be honest, I made this solution in a hurry. You know, I start thinking about making a library MulterCropper.
You are right about hard-coded config, This solution sucks, As for me the best way is to take it out and pass via a constructor.

Thread Thread
 
aayani profile image
Wahaj Aayani

It really helped me to say the least. Thank you. That's a nice idea for a library though.

Collapse
 
rainercedric23 profile image
Cedric Mariano

great implementation indeed, couldn't think any better on how to implement this, as I created one with the traditional, controller service approach with providers. one question though, have you tried handling multiple images upload on single request for this one? if yes, how many files do you recommend to be uploaded once or how big is the size for example on the s3 storage?

Collapse
 
sergey_telpuk profile image
Sergey Telpuk • Edited

Hello, by default this solution doesn't support multiple files, about size and how big it depends on your aims