As soon as your website or app reaches a point when you realize you cannot always display static or dummy images among all the dynamic content fetched from your backend, you start looking for ways to integrate image uploads in your project. After looking around on the internet for a while, you begin to hear some names and if you aren't willing to let AWS have your card details, you will fall back to Cloudinary. Now, Cloudinary is quite convenient for small project and specially when you are still studying. But last time I tried getting it set up in my project, it wasn't the most "copy-from-docs and go" set up. Along with Cloudinary's own configuration, an important aspect is how you actually bring your files to Cloudinary's API for the upload.
In this one, we will be implementing image uploads to cloudinary wherein we will demonstrate it using a form to set profile pictures for the user. Our focus here would be on how we will handle the file so that it can be uploaded. I will use a node.js backend with authentication already implemented. In the client, I will use a form that handle user's profile information but we mostly care about the profile picture.
Preface
I will be going in depth in this one. If you need a to-the-point solution to how you can get image upload set-up with cloudinary you can skip to In short
Let's get started
So, I am trying to add a profile picture to the users in my database by uploading them to cloudinary. I would have a form in the front end, I would use a file input to select a file and then it would send this file to the backend where it would be uploaded to cloudinary. Now, these things come with hidden complexities at each steps. The file input gives us a Javascript File
Object which the database or cloudinary doesn't really like or understand. We need raw image data or maybe a url. Also, JSON wouldn't send images as it is basically just fancy text. Even if we do manage to send these File Objects to our endpoint, this file object would not be enough to upload the image to cloudinary. Let's test each of these using an express server and a mongodb database.
Situation
I have a user collection in my local mongodb and I have added one user to it manually which looks like the following:
blogdb> db.users.find()
[
{
_id: ObjectId("629f08b134f91b645470ca97"),
username: 'suyious',
name: 'Suyash Kumar',
email: 'solo@gmail.com',
password: '$2b$10$eT7nsqWx3jVjlH9xbHOU2.EcowoHlXf8VWMldb2klFKL2CzxSeC5a',
role: 'user',
createdAt: ISODate("2022-06-07T08:13:37.436Z"),
__v: 0,
avatar: {
public_id: 'xetra/products/vunlvjcijttmyu6aah1o',
url: 'https://res.cloudinary.com/djv5txrzp/image/upload/v1667916852/xetra/products/vunlvjcijttmyu6aah1o.jpg'
}
}
]
This should give an idea of how the user collection looks like. We have an avatar
field which is an object with a key url
. Now I would like to change this avatar field and by changing it, I imply sending an image from my frontend to the server which would then get uploaded to cloudinary and the avatar
field would have it's new url
. Now we can always manually upload our images to cloudinary and just send it's url through a JSON, but we need that to happen on it's own and we should just need to use the html file input.
Sending the image from the frontend
I guess even now, you would have started getting a hint that JSON would probably not send our file object to our server. But, let's test that anyway. All we need is an <input type="file"/>
and an axios.put('/upload', body)
. When we are sending the image from the frontend, the request body looks like the following in Javascript:
Looks fine for now but as soon as this request reaches the backend, thing begin to look different.
Clearly, JSON doesn't seem to know what a file exactly is. Guess, it's time to run to stack overflow.
Turns out, apart from application/json
, there are other content-type for your request body and one of them is mutipart/form-data
. In short, this is what we use when we have files along with text to send to server. Looks like that should solve our problem and the File
would show up in our backend.
multipart/form-data
In order to send multipart/form-data
through an HTMl form, you need <form enctype='multipart/form-data'>
tag and when using axios, you need axios.put('/put', body, { "Content-Type": "multipart/form-data" })
. You can also create a FormData
Object in Javascript and set key/value pairs to construct your request body.
So, now we should just try sending form-data from our frontend similar to before, only this time our request body would be constructed using:
const body = new FormData(formElement);
The formElement
is a reference to am html form which can be the name of a form ( or ref.current when using React.useRef)
In the frontend, the request body looks like this:
But now, we turn to our server only to see an even more obscure output.
We should also try to send the same form-data using postman just in case we are making some mistake in our frontend code.
Clearly, even postman gives us a similar output. It seems like the backend isn't able to understand what we are throwing at it. Hmm. That doesn't happen often. Or does it? One might begin to remember that we couldn't earlier even read the JSON output if we just threw it at the backend. If that didn't ring a bell, I am talking about the express.json()
or earlier bodyparser()
middlewares that we needed to parse our JSON requests. It is fairly obvious that we would also need to parse form-data before our backend can understand it.
Now, there are multiple similar libraries for accomplishing this and the most famous one that I can find is multer
. Naturally, we would start by installing it and including it in our server. We could add the middleware for every route similar to express.json()
with express.use()
, but we would want to only add it for the routes that actually need form data.
multer
So, if we go forward, install multer and add a middleware multer().single('avatar')
(see docs) to our upload route, we find this in our console:
Looks like our form-data did pass through. But where is the file. Turns out the files are not a part of req.body
. Instead they are in req.file
.
We can finally see our file reach the backend. This looks good. Now all we have to do is throw this file at Cloudinary and it should just work out of the box. Right? Well, we all know our history with throwing things around here.
Cloudinary
Now, let's install cloudinary
and set it up with the configurations. You need following environment variables for the config:
cloudinary.config({
cloud_name: process.env.CLOUDINARY_NAME,
api_key: process.env.CLOUDINARY_API_KEY,
api_secret: process.env.CLOUDINARY_API_SECRET
})
Above is also the code that you need to add to your server for initial configuration of cloudinary. Once this is done, you should be able to talk to your cloudinary account. ( If it wasn't obvious, you need to create an account. It won't charge you or ask for card details. You will get your api_key
and api_secret
from the dashboard
Now, as from the docs, we need the following to upload our image:
import { v2 as cloudinary } from 'cloudinary'
cloudinary.uploader
.upload(file, options)
.then(callback);
Once we are set up, we should try to use the file we received from the frontend on the upload
function provided by the api. I have the following code added to my upload route:
cloudinary.uploader.upload(req.file)
.then((result => {
console.log(result);
}))
This time, the server gives us a new error. Depending on how your server is set up, the error might show up at different places. As for me, the error comes as a 500 Error Response.
The error message tells us that cloudinary's API does not accept a Javascript File
object and instead is asking for some string. If we look into the docs again, we find the following:
Given above, the upload
function takes only the allowed types as the file parameter. Out of these, we do not have a local file path in the server as the file comes from the frontend. Neither do we have a file URL or S3 bucket URL. We are left with using a byte-array buffer or a Data URI (Base64 encoded).
As we can see above, we do seem to be getting a buffer key in the file object received. Let's inspect it's value and see if it can be of our use. If we just plug the buffer value to the upload
function, the cloudinary api still cannot process it and gives a similar error.
We could try to process this byte array and make it work with the API but if we look at multer
's documentation, we notice using the buffer option requires storing the images into memory which could be bad for the server in terms of performance.
Now, we are left with either trying to convert the file into a byte-array in the frontend or using the data URI option. Also, we should try to do the computations in the frontend itself as much as possible, now that we are reminded.
If we dig around a little to find some information about byte-array or Data URI, we come across the FileReader
object provided by Javacript's File
API. This object provides functions for dealing with both byte-arrays and Data URIs.
FileReader
This FileReader
object delivers the data using events, as reading from disk may take time. The Syntax for using it is generally of the form:
const reader = new FileReader();
reader.onload = () => {
<your-processing-function>(reader.result);
}
reader.<instance function>(blob);
We should try the readAsDataURL(file)
function to send the image file as a Data URI. Note that the function is readAsDataURL
and NOT readAsDataURI. The result of this function call returned through reader.result
can also be used with <img src=""/>
to preview the image. The following code demonstrate how we would process the file before sending it.
const reader = new FileReader();
reader.onload = () => {
console.log(reader.result);
}
reader.readAsDataURL(input.files[0])
Here, input
may be e.target
if you are listening to events on a file input or inputref.current
if inputref is a ref to an input element when using React.useRef
. The resulting Data URI is returned as a string through reader.result
. Now, that this is a string, it should work as an argument to cloudinary API's upload
function.
The result looks like the following:
It is basically a bunch of gibberish to us but will represent our image file to the server. When we send this new Data URI string to the server, it gets sent as is, that is as a string. But now that it is no longer a file. It is part of req.body
itself and not of the req.file
Now, we could plug this string into cloudinary API's upload function and see if it works:
cloudinary.uploader.upload(req.body.avatar)
.then((result => {
console.log(result);
}))
This, after all this work, actually works and we get the following result in our server console:
We can go to the provided secure url and it is actually uploaded to cloudinary. We can also see this image in our cloudinary media library. Here, I have used the folder
option that can be provided in the second argument of the upload function as follows:
const result = await cloudinary.uploader
.upload(req.body.avatar, {
folder: "/fraise/avatars"
})
The API returns an object with a url for the uploaded image as secure_url
, which can now be stored in the database to update our user profile.
const user = await User.findByIdAndUpdate(req.user.id, req.body, {
new: true,
runValidators: true,
useFindAndModify: false
})
Here, my server is setting req.user
while validating authentication, though it is only the providing the id
of the entry to update in our database. This updates the current user's profile picture.
In short
In short, all I did all this while was:
- send files as form-data and not JSON.
here you might need:
<form enctype='multipart/form-data'>
axios.put('/put', body, { "Content-Type": "multipart/form-data" })
- receive form-data using multer.
you might need to install multer
and add the middleware:
multer().single('name')
for other usages, here's the docs
- setup cloudinary and upload
cloudinary.config({
cloud_name: process.env.CLOUDINARY_NAME,
api_key: process.env.CLOUDINARY_API_KEY,
api_secret: process.env.CLOUDINARY_API_SECRET
})
import { v2 as cloudinary } from 'cloudinary'
cloudinary.uploader
.upload(path, options)
.then(callback);
- send files as dataURI for the
path
inupload()
above:
const reader = new FileReader();
reader.onload = () => {
console.log(reader.result);
}
reader.readAsDataURL(input.files[0])
But this makes me think
Even though we used multer
to be able to read multipart/form-data in the server, we don't quite seem to have used it for actual file transfer between the frontend and the backend. It makes me wonder if using form-data was for nothing. I did try to use JSON for the same but it didn't quite seem to work. Now, I didn't want to pull apart all that code again. So, I urge you to try it.
Additionally, the data URI seems to be a very long string which might be too long to send back and forth. Tell me if there are other simpler ways to upload the images. I do know of another method, creating a temporary file on the server and using it's file path for the upload. This would require creating and removing files on the server.
I would be really happy if I am pointed to a better direction here, but if my solution was helpful to you, that would make me really happy too.
Top comments (1)
This was helpful, thanks Suyash