If you've never been messing around with file upload before and you were given a task to do so, perhaps you will feel scared out of it (well, a lil bit personal experience here 😛).
In fact, if you're a web developer, you will definitely face this task sooner or later because it's widely used in every web application.
In this article, I'm gonna show you how to do it in my way using Javascript.
I've posted the second article related to this one about how to add cancellation & retry feature. Go on check it if you are interested, I'll put the link here How to Upload Multiple File With Feature Cancellation & Retry Using ReactJS
Now before we continue, here is the example of final result that we want to achieve:
If you want to look at the source code, you can take a look in here. But I will explain it step by step how to build it from scratch.
Getting Started
First thing first, let's talk about what kind of technologies that we're gonna use for backend and frontend.
- ReactJS - our main frontend application framework [FE]
- Redux - state management that being used for ReactJS [FE]
- Redux-thunk - to be able to do asynchronous logic on redux [FE]
- Axios - promised based http request for client & server [FE]
- Lodash - a bundle of utility javascript function [FE]
- ExpressJS - a NodeJS server to mock our API server [BE]
-
Multer - a Node.js middleware for handling
multipart/form-data
[BE]
Now let's start creating the project folder:
$ mkdir file-upload-example
$ cd file-upload-example
$ mkdir server
// Our folder structure will be like this
./file-upload-example
../server
Setting up Server & API
First we need to install all of dependencies for the backend side
$ cd server
$ touch server.js // creating new file
$ npm init -y // creating default package.json file
$ npm i express multer cors
I'll just show you the server.js
code directly, since we'll be more focus on the frontend side, here is the code:
Let's try running it on terminal by typing node server.js
.
If you saw message Server running on port 5000
, that means your server running successfully. Great! We've finished configure our backend side, let's move to the frontend side. By the way, if you're curious about the multer library, you can check it here.
NOTE: you can let the server running while we're developing our frontend side
Setting up Frontend Side
Now open a new terminal (because we want to run 2 localhost, #1 server and #2 client) and go to the root of our folder. We will set up our frontend with create-react-app and also installing our dependencies, so let's get started:
$ npx create-react-app client
$ cd client
$ npm i redux react-redux redux-thunk axios lodash
$ npm start
// Now our folder structure will be like this
./file-upload-example
../server
../client
Now your react app will be opened in new browser tab on localhost:3000. Great, let's start adding stuff! First we will modify our App.js
By doing so, we've added an input button that when we upload a file, it will console.log
the file that being uploaded.
Now let's set up our redux.
The idea is, every time we attach files, the files will be stored into redux store with a certain data structure.
First, we create a new folder redux
along with its file (still empty) like this:
//uploadFile.types.js
const uploadFileTypes = {
SET_UPLOAD_FILE: 'SET_UPLOAD_FILE',
}
export default uploadFileTypes
//uploadFile.actions.js
import uploadFileTypes from './uploadFile.types'
export const setUploadFile = data => ({
type: uploadFileTypes.SET_UPLOAD_FILE,
payload: data,
})
// uploadFile.reducer.js
import uploadFileTypes from './uploadFile.types'
import { modifyFiles } from './uploadFile.utils'
const INITIAL_STATE = {
fileProgress: {
// format will be like below
// 1: { --> this interpreted as uploaded file #1
// id: 1,
// file,
// progress: 0,
// },
},
}
const fileProgressReducer = (state = INITIAL_STATE, action) => {
switch (action.type) {
case uploadFileTypes.SET_UPLOAD_FILE:
return {
...state,
fileProgress: {
...state.fileProgress,
...modifyFiles(state.fileProgress, action.payload),
},
}
default:
return state
}
}
export default fileProgressReducer
We will define the modifyFiles
utils later, but now I want to explain about the data structure of the fileProgress
. We're gonna save those files in Object format instead of array format, but WHY? Well, it's because every time the upload progress is incrementing, we need to update the progress field of each file in the redux store.
In order to do that, if the fileProgress
type is array:
- We should loop the array first (to find the index) then finally we can update the desired item. And we always need to do the looping every time we want to update any progress of each files. This is not good.
But if we use Object type instead for fileProgress
:
- We don't have to do the looping, we only need to give the exact object key of each files then it can update the progress directly.
Probably some of you get confused of this, let's just move on and understand it by looking at the real code later.
Now let's define the modifyFiles utils on uploadFile.utils.js
.
import { size } from 'lodash'
export const modifyFiles = (existingFiles, files) => {
let fileToUpload = {}
for (let i = 0; i < files.length; i++) {
const id = size(existingFiles) + i + 1
fileToUpload = {
...fileToUpload,
[id]: {
id,
file: files[i],
progress: 0,
},
}
}
return fileToUpload
}
This utils function will modify the incoming files, into an Object and finally will populate each file object to be the same as the data structure on the INITIAL_STATE
comment (as we mentioned before).
Now in order to test it, we should apply this redux into our App, let's do it.
// root-reducer.js
import { combineReducers } from 'redux'
import UploadFile from './uploadFile/uploadFile.reducer'
const rootReducer = combineReducers({
UploadFile,
})
export default rootReducer
Now don't forget to utilize setUploadFile
into the upload button App.js
Now it's time to check our localhost, the behavior should be similar like this
As you can see above, we could trace the file that we upload on the redux store. Some of you might wondering for 2 questions, first: why the files that we console.log
show nothing? Second: why the value of file
on fileProgress
on redux store have empty object instead of the file data?
Let's discuss it one by one
- The
console.log
shows nothing because after we save it to the redux store, we directly set the value of the input element into''
(e.target.value = '')
. We want to clear theinput
value so that we can upload another file afterwards. - Now we can track the files inside the redux-store but the value is an empty object
{}
, this is because Files type of data is not a literal object and redux-dev-tools cannot read that type, hence redux-dev-tools display it as an empty object (but the files actually there)
Uploading Item
Now we've successfully save our files into redux, the last step is upload it to the backend side.
Step1
First let's make the UploadProgress
component to display our file upload progress. This is how we want to structure our folder.
./src/components
../UploadProgress/
.../UploadProgress.js
.../UploadProgress.module.css
../UploadItem/
.../UploadItem.js
.../UploadItem.module.css
Then in App.js
call UploadProgress
component:
...
...
import UploadProgress from './components/UploadProgress/UploadProgress'
...
...
return (
<div className="App">
<header className="App-header">
<img src={logo} className="App-logo" alt="logo" />
<input type="file" multiple onChange={handleAttachFIle} />
</header>
<UploadProgress /> // --> call the component here
</div>
)
...
Now run the current behavior on the localhost and we will see the upload progress component works properly.
Step 2
Now we should create a function to upload the files to the backend also incrementing the progress of the upload so that the progress bar will increment.
// uploadFile.types.js
...
SET_UPLOAD_PROGRESS: 'SET_UPLOAD_PROGRESS',
SUCCESS_UPLOAD_FILE: 'SUCCESS_UPLOAD_FILE',
FAILURE_UPLOAD_FILE: 'FAILURE_UPLOAD_FILE',
...
// uploadFile.reducer.js
...
...
case uploadFileTypes.SET_UPLOAD_PROGRESS:
return {
...state,
fileProgress: {
...state.fileProgress,
[action.payload.id]: {
...state.fileProgress[action.payload.id],
progress: action.payload.progress,
},
},
}
case uploadFileTypes.SUCCESS_UPLOAD_FILE:
return {
...state,
fileProgress: {
...state.fileProgress,
[action.payload]: {
...state.fileProgress[action.payload],
status: 1,
},
},
}
case uploadFileTypes.FAILURE_UPLOAD_FILE:
return {
...state,
fileProgress: {
...state.fileProgress,
[action.payload]: {
...state.fileProgress[action.payload],
status: 0,
progress: 0,
},
},
}
...
...
// uploadFile.actions.js
...
...
export const setUploadProgress = (id, progress) => ({
type: uploadFileTypes.SET_UPLOAD_PROGRESS,
payload: {
id,
progress,
},
})
export const successUploadFile = id => ({
type: uploadFileTypes.SUCCESS_UPLOAD_FILE,
payload: id,
})
export const failureUploadFile = id => ({
type: uploadFileTypes.FAILURE_UPLOAD_FILE,
payload: id,
})
export const uploadFile = files => dispatch => {
if (files.length) {
files.forEach(async file => {
const formPayload = new FormData()
formPayload.append('file', file.file)
try {
await axios({
baseURL: 'http://localhost:5000',
url: '/file',
method: 'post',
data: formPayload,
onUploadProgress: progress => {
const { loaded, total } = progress
const percentageProgress = Math.floor((loaded/total) * 100)
dispatch(setUploadProgress(file.id, percentageProgress))
},
})
dispatch(successUploadFile(file.id))
} catch (error) {
dispatch(failureUploadFile(file.id))
}
})
}
}
Little explanation here:
-
uploadFile
function will receive array of files to be uploaded to backend. Inside the function, we will do looping as many as the files length. Each loop, will add the file intoFormData
(this is how we send data type of file via http to the server), then we send it to the backend usingaxios
POST method to our localhost server. - Axios receives parameter
onUploadProgress
that will subscribe each upload progress, this is where we want to utilize oursetUploadProgress
function to upload our progress bar (you can read the documentation here) - Then if it success, we will dispatch
successUploadFile
and if it failed we will dispatchfailureUploadFile
And the last one, we call the uploadFile in our component UploadProgress.js
like this.
import React, { useEffect } from 'react'
...
...
const { fileProgress, uploadFile } = props
const uploadedFileAmount = size(fileProgress)
useEffect(() => {
const fileToUpload = toArray(fileProgress).filter(file => file.progress === 0)
uploadFile(fileToUpload)
}, [uploadedFileAmount])
...
...
const mapDispatchToProps = dispatch => ({
uploadFile: files => dispatch(uploadFile(files)),
})
export default connect(mapStateToProps, mapDispatchToProps)(UploadProgress)
UploadProgress
component will watch every changes ofuploadFileAmount
usinguseEffect
. So every time new file uploaded (and the file progress = 0), it will calluploadFile
function and upload it to the backend.
Now let's see our localhost (don't forget to run your localhost server too).
Look, it's working! Now the progress bar no longer 0% and we manage to upload multiple files and multiple type (pdf, png, mp4) on it.
But this is not the end of our journey, have you realize? When you upload files, the progress bar seems like not incrementing, it's like glitching from 0% to 100% instead. What happen? 🤔
Now the reason is explained precisely in here, but I will try to summarize it into a little one.
What happens there is that we developed our frontend and backend application on the same machine (localhost on our laptop) which there is no real time issue with sending data to the backend side. But if it's on production env which usually we will save the files into cloud storage (ex: AWS S3), there will be amount of time needed to transmit the files from our server into AWS server and that's when our progress bar will be functioning perfectly.
But no worries, we actually can simulate that amount of time on our browser, take a look on below GIF to implement how to do it.
Voila! That's it! We've reached at the end of this tutorial. You can take a look at the full source code if you want in here.
Thank you for those who manage to read from top to bottom of this article. Since this is my first blog article, I'm sorry if there is something unusual or not understandable. I will try to write more article and make it better and better.
I've posted the second article related to this one about how to add cancellation & retry feature. Go on check it if you are interested, I'll put the link here How to Upload Multiple File With Feature Cancellation & Retry Using ReactJS
Happy Coding! 🎉🎉
Top comments (24)
This is great, thank you!
I'm trying to do exactly what you mentioned at the end of the article and ensure that files go into cloud storage, specifically an AWS S3 bucket. Do you have any further article/walkthrough or tips on how to link it up and do just that?
Unfortunately I don't have any expert experience on the backend side.
But all I could tell is if you're using the same backend side of this article (which using multer), whenever client upload file and hit the
POST /file
endpoint, backend side will receive the file which contain a lot of information (you could see it on the multer docs) and perhaps you could use its data to pass to AWS S3.And if I do a quick search, turns out multer have another package that work along with S3 bucket, you can see it here
Goodluck!
Amazing, thanks a lot!! :)
This article is amazing.
Thank you very much!
I got one question though.
if i want to upload many images (around 250+) what would you recommend doing? i'm afraid this action will kill the page
?
My personal opinion it's just like you said, probably it will kill the page, so my suggestion is to add maximum files limitation for on going upload file (for example 10 max), then from the UI you should disable the dropdown area and give some message to the user.
It's a rare case to have people upload 250+ files at the same time, but this is my opinion.
But what if i do need this?
There maybe situations that some one will upload 1000+ pictures.
What would you recommend in these situations? (photos can be also very heavy)
No, my answer is still the same, IMO it will kill the page because of memory leaks, so i really wouldn't recommend doing that.
And can you elaborate more (perhaps with example) how do users able to upload 1000+ pictures at a time? Because I never find that case in any apps (at least for me).
But if you insist, honestly I don't have any valid answer for this one. Probably you should change the data structure or you can upload it by queuing every 5 files or....yeah sorry, I don't have any valid answer for this one.
@noamatish you can checkout the p-limit package, it can serve the purpose of limiting number of uploads at a time while queueing the rest.
github.com/sindresorhus/p-limit
Amazing article. Very well explained. Congrats.
Just one question, how to add the button to close the loading window, without affecting the progress logic? I saw you added the button layout on css but the button component it is not present.
Thanks in advance mate.
Actually I also implemented this upload in my company, in my case I put the close button on the title upload box, but it will only be shown when all of the upload is finish (whether success or fail). In my case I would just remove all of the data from uploadProgressRedux when the close button is clicked and unmount the upload progress box.
But if you want to be able to close it while it's uploading, IMO you can try to add the close button and when it's clicked, unmount the upload progress box but don't have to remove the data from uploadProgressRedux. That way the progress will still be running on the background (haven't try it tho).
But personally thinking this would be bad for user experience, since they didn't know the upload progress status. It would be better to encourage user not to close the upload progress box until it's finish
Thank you very much for your quick answer kind sir!! I really apreciate it!
thanks for explaining well around this,
For the scenario you said we have to call upload_api for each file that selected by user! I think this causes overload server if we upload many files together
In my case, I have a batchFile api that get multiple files together, but question is how can I get progress for each file?!
More article like this please!
Can I apply your code in my project of usb transfer files?
I'm not sure, I have no experience using usb transfer files. But I personally think, if your transferred files structure are the same with my example, then it should work.
This is well explained. Very comprehensive! Bravo!
Thank you, glad to hear that
Great article. Exactly what I was looking for.
Thanks for taking the time to make this.
how to upload all files all at once and show the progress of each?
Umm...actually that's what I explained on this article.
Perhaps you can try it on your local by using the repository.
Or did I misunderstand your question?
Some comments may only be visible to logged-in visitors. Sign in to view all comments.