How to automate all the things with Gulp

Have you ever been tired of hitting the F5 key once too often? Yeah, I understand the pain. Here’s where task automation shines. Writing scripts to continuously handle the time-consuming tasks you actually need in your development workflow. Don’t get cold feet on me here. It’s a lot less scary than it sounds. Trust me. Once you get the hang of it, you’ll never want to write code without it.

So why Gulp? Because you’re writing the tasks with your best buddy, plain old JavaScript. It doesn’t get any better than that.

gulp.js

Blowing up the Death Star.

In comes Gulp. The hero you need, and most definitely deserve. There are several key features regarding the use of Gulp which would make you want to use it. The one I hold as the most important is the way it can simulate the server environment where you will ultimately be hosting your code. This includes moving files around your project directory, and more importantly placing them in a development directory where you will be running a web server. Gulp also enables you to compile , minify and concatenate any files you want. All with the sole goal of getting your code base’s footprint down to the absolute minimum. In this process making it ready for shipping to production. It’s perfectly fine if you don’t know any of the terms above, we’ll go through them in more detail a bit further down.

TLDR;

You can severely hurt my feelings by only reading this TLDR; or be awesome instead, and read the whole article. Have fun!

1. Three main Gulp commands: gulp.task, gulp.src, gulp.dest.
2. pipeing is freaking awesome! The pipe() method is built into Node.js and is used to copy files from one folder to another.
3. Have a logical folder structure, with three main folders. The src folder for pre-processed files, tmp for the local development server and dist for processed and minified files.
4. Create separate tasks for pipeing HTML, CSS and JavaScript files from src to  tmp.
5. Combine the HTML, CSS and JavaScript tasks into one main ‘copy’ task. This will let you copy all the files with one command.
6. After all files have been copied you want to reference them automatically in your main index.html. This is done with an ‘inject’ task and is called injecting dependencies.
7. When the files have been copied and injected it’s time to run a development server on the tmp  folder.
8. While the server is running you ‘watch’ for changes, and enable live reloading on the local development server.
9. Satisfied with your code? Go ahead and ‘build’ the production files and place them in the dist directory.
10. Delete tmp and dist before pushing to GitHub. (Or just add them to your .gitignore)
11. The code you will be writing during this tutorial is on GitHub and will be linked at the bottom of the article. Sorry, but you have to scroll to the bottom. :)

Install the tools

First of all, you will need to have Node installed on your machine. To check if you do, open up a command prompt and type node -v. The version number of Node you have installed will be shown as output on the command line. If not, you do not have Node installed on your machine. Not to worry, head over to Node’s official site to get started.

Download Node.js

All done? Now you’re set to install the necessary tool regarding the actual task automation. Go back to your command prompt and run:

npm install -g gulp

Note: Linux and Mac users will most likely need to run this command with sudo , to enable the required permissions.

You’ve now installed Gulp globally on your machine. This step is important as it is required for Gulp to be visible in your command line regardless of which directory you are in. The more technical explanation would be that Gulp has now been added to the  PATH.

You’re now ready to get started with writing tasks.

Introduce yourself

There are three main commands Gulp provides:

• gulp.task – defines a new task giving it a name, an array of dependencies and a callback function, which will be called when the task is run.
• gulp.src – sets the source folder where files are located.
• gulp.dest – sets the destination folder where files will be placed.

The backbone of using Gulp at all lies in the interaction between gulp.src and gulp.dest with the .pipe method as a bridge.

The .pipe method is a default method in Node.js, not to delve deeper into this particular subject, view it as a means to copy files from one directory to another.

What does this mean?

Gulp by itself only provides the basis needed for task automation. The bulk of the work you would need in a development workflow lies in the vast plug-ins created for Gulp. Without them, you only have a shell. Of course, you could write your own, but with more than 3000 available for download on npm, I would advise against this. Do not reinvent the wheel. Use the resources which are already at your disposal. Feel free to take a look here, or just Google “gulp whatever and witness the magic.

Baby steps

Let’s code up a very basic backbone, just to wrap our heads around the concept. We’ll incrementally progress further down this article.

First of all create a new directory giving it a super inspiring name, something like super-awesome-gulp-tutorial should be fine. Once you’ve done this open up a command prompt within this directory. As Gulp is a package on npm you will need to initialize npm to have a link to the package manager.

Not familiar with npm – the Node Package Manager? Take a look here.

npm

After you’ve done this, you will also need to install Gulp locally. Why? Gulp will always use the local version of itself in project folders.

npm init
npm install gulp --save-dev

Installing it with the --save-dev flag will include it in the package.json beneath the development dependencies. Great job, ready for some code? Let’s jump in. Create a new file, name it gulpfile.js . This file is the entry point for Gulp, here’s where you will be writing all the code to automate tasks. Go ahead and write this in your gulpfile.js :

var gulp = require('gulp');

gulp.task('default', function () {
  console.log('Hello World!');
});

Congratulations. You’ve just written your first Gulp task. I bet this seems familiar to you, an event listener followed by a callback function. Let’s try it out. Jump back to your command prompt, and just type:

gulp

Hit enter and you will see something like this get logged back to the command line:

[19:41:16] Using gulpfile ~/super-awesome-gulp-tutorial/gulpfile.js
[19:41:16] Starting 'default'...
Hello World!
[19:41:16] Finished 'default' after 162 μs

By using the keyword gulp you told Gulp to interpret a particular task, and as nothing was written after the keyword, the ‘default’ task was run. Let’s say you have a task named ‘ build ’ and you wish to run it. Writing gulp build will trigger that particular Gulp task. You can even run multiple tasks, which is also perfectly fine. It looks like this.

gulp sometask anothertask

Have you been following along? Great, you’re now ready to code some serious stuff. Let’s get a development environment up and running.

Full hacker mode, ON

I’ll start by outlining a basic folder structure of a sample project. One of many best practices is to have three main folders, src for all of your source files, dist for the bundled and minified files, and finally a tmp directory which will be used as the sandbox for our local web server.

• src  – source files, pre-processed, un-minified.
• tmp  – development files, pre-processed, un-minified. The directory where you will be running the web server.
• dist  – production files, processed, minified.

Create the src folder, but do not create the dist nor the tmp folder yet. You will see a bit further down how you can create it dynamically and incorporate it into an automated task. Let’s add some files to the src folder, to finally have something to play with. An index.html , a script.js and a style.css should be more than enough. These will serve as your source files, the only files you will be editing. Gulp will handle everything else.

Let’s get started with the Gulping of all the things!

Step 1 – Set up the folder structure

First of all you need a paths variable to store all the file and directory paths of your project. Place this right under where you required gulp.

// gulpfile.js
var gulp = require('gulp');
var paths = {
  src: 'src/**/*',
  srcHTML: 'src/**/*.html',
  srcCSS: 'src/**/*.css',
  srcJS: 'src/**/*.js',

tmp: 'tmp',
  tmpIndex: 'tmp/index.html',
  tmpCSS: 'tmp/**/*.css',
  tmpJS: 'tmp/**/*.js',

dist: 'dist',
  distIndex: 'dist/index.html',
  distCSS: 'dist/**/*.css',
  distJS: 'dist/**/*.js'
};

gulp.task('default', function () {
  console.log('Hello World!');
});

Specifying the /**/* part is equivalent to including all files within the folder and any possible subfolders.

Step 2 – Set up the HTML task

Now you need to create a task to copy all HTML files from the src directory to the tmp directory where you’ll be running the web server.

gulp.task('html', function () {
  return gulp.src(paths.srcHTML).pipe(gulp.dest(paths.tmp));
});

Step 3 – Set up the CSS task

Same thing for the CSS files.

gulp.task('css', function () {
  return gulp.src(paths.srcCSS).pipe(gulp.dest(paths.tmp));
});

Step 4 – Set up the JavaScript task

Yup, same thing for the JavaScript files.

gulp.task('js', function () {
  return gulp.src(paths.srcJS).pipe(gulp.dest(paths.tmp));
});

Step 5 – Combine all tasks into one task

This part is fun. Gulp allows you to combine tasks, and add tasks to other tasks as dependencies. This feature is incredibly useful because you can tell Gulp to run and complete certain tasks before even starting other tasks.

gulp.task('copy', ['html', 'css', 'js']);

Try it out! Here’s what you’ll see after running gulp copy.

[19:39:08] Using gulpfile ~/super-awesome-gulp-tutorial/gulpfile.js
[19:39:08] Starting 'html'...
[19:39:08] Starting 'css'...
[19:39:08] Starting 'js'...
[19:39:08] Finished 'css' after 19 ms
[19:39:08] Finished 'html' after 30 ms
[19:39:08] Finished 'js' after 18 ms
[19:39:08] Starting 'copy'...
[19:39:08] Finished 'copy' after 4.67 μs

Jump back to your project folder and take a look. Now you have a tmp directory. It was created dynamically. Magic! (Just kidding, not really.)

The tmp directory contains the same files you have in the src directory. The .pipe() command has copied files from the source to the given destination.

Step 6 – Inject files into the index.html

What’s this? Well, you’ve copied the files over to the tmp folder. Now you need to tell the index.html which CSS and JavaScript files you want to reference. This is done quite easily with a Gulp plug-in called gulp-inject. Go ahead jump back to the command prompt and run:

npm install gulp-inject --save-dev

Now, you need to add a reference in the index.html where you wish to inject the files.

<!DOCTYPE html>
<html>
  <head>
    <!-- src/index.html -->

    <!-- inject:css -->
    <!-- endinject -->
  </head>
  <body>

<!-- inject:js -->
    <!-- endinject -->
  </body>
</html>

After gulp-inject has been run, there will be files between these comments. Keep in mind they need to look exactly like what is written above.

Jump back to the gulpfile.js and add this:

var inject = require('gulp-inject');

Keep in mind, you’ve already added the reference to the files within the tmp folder. It should look like this:

var paths = {
  src: 'src/**/*',
  srcHTML: 'src/**/*.html',
  srcCSS: 'src/**/*.css',
  srcJS: 'src/**/*.js',

tmp: 'tmp', // tmp folder
  tmpIndex: 'tmp/index.html', // index.html in tmp folder
  tmpCSS: 'tmp/**/*.css', // css files in tmp folder
  tmpJS: 'tmp/**/*.js', // js files in tmp folder

dist: 'dist',
  distIndex: 'dist/index.html',
  distCSS: 'dist/**/*.css',
  distJS: 'dist/**/*.js'
};

Now you’re ready to add another task to inject the files.

gulp.task('inject', ['copy'], function () {
  var css = gulp.src(paths.tmpCSS);
  var js = gulp.src(paths.tmpJS);
  return gulp.src(paths.tmpIndex)
    .pipe(inject( css, { relative:true } ))
    .pipe(inject( js, { relative:true } ))
    .pipe(gulp.dest(paths.tmp));
});

Look at this now. You’ve added the ‘copy’ task as a dependency for the ‘inject’ task. Gulp will first copy all the files to the tmp directory, only then will it do the injecting. Let’s break this down. You’re placing the gulp.src of the files copied to the tmp folder into two respective variables. One for the CSS, the other for the JavaScript files. In the return statement you grab the index.html which has already been piped to tmp with the ‘copy’ task and .pipe() the inject() function with the variables where you placed the respective files from the tmp directory. The second parameter you pass to the inject() function is an options object. Here, relative:true means that the file paths to be referenced in the index.html will be relative. You want this. Trust me. It’ll save you so much headache.

Toggle back to the command prompt and run gulp inject. The index.html within your tmp directory should now look like this.

<!DOCTYPE html>
<html>
  <head>
    <!-- tmp/index.html -->

    <!-- inject:css -->
    <link rel="stylesheet" href="style.css">
    <!-- endinject -->
  </head>
  <body>

<!-- inject:js -->
    <script src="script.js"></script>
    <!-- endinject -->
  </body>
</html>

Step 7 – Serve the development web server

I bet you’d like to see the fruits of your labor. I would too. Let’s get a server up and running to quench that thirst.

Go ahead and switch back to the command line and run:

npm install gulp-webserver --save-dev

This is a Gulp plug-in which allows you to run a web server on your local machine. Exactly what you need. After you’ve installed it, go ahead and require it at the top of your gulpfile.js.

var webserver = require('gulp-webserver');

Great! Let’s get the ‘serve’ task coded up.

gulp.task('serve', ['inject'], function () {
  return gulp.src(paths.tmp)
    .pipe(webserver({
      port: 3000,
      livereload: true
    }));
});

Once again you need to include a dependency. Here you want the ‘inject’ task to finish before running the web server. Simply reference the tmp directory and .pipe() it to the web server. The gulp-webserver plug-in takes an options object as a parameter. You will need to specify the port on which it will run, and tell the server to reload if it senses any changes. Once you get used to this. There’s no going back.

Let’s test this out. Add some lines of code to the files in the src directory. Here’s a simple example:

index.html

<!DOCTYPE html>
<html>
  <head>
    <!-- inject:css -->
    <!-- endinject -->
  </head>
  <body>
    <div class="awesome">This is awesome!</div>

<!-- inject:js -->
    <!-- endinject -->
  </body>
</html>

style.css

.awesome {
  color: red;
}

script.js

console.log('Awesome!');

Jump back to the command line and run gulp serve. You should see this get logged back to you.

[23:50:44] Using gulpfile ~/super-awesome-gulp-tutorial/gulpfile.js
[23:50:44] Starting 'html'...
[23:50:44] Starting 'css'...
[23:50:44] Starting 'js'...
[23:50:44] Finished 'html' after 30 ms
[23:50:44] Finished 'js' after 19 ms
[23:50:44] Finished 'css' after 22 ms
[23:50:44] Starting 'copy'...
[23:50:44] Finished 'copy' after 4.77 μs
[23:50:44] Starting 'inject'...
[23:50:44] gulp-inject 1 files into index.html.
[23:50:44] gulp-inject 1 files into index.html.
[23:50:44] Finished 'inject' after 16 ms
[23:50:44] Starting 'serve'...
[23:50:44] Webserver started at http://localhost:3000
[23:50:44] Finished 'serve' after 18 ms

By running the serve task all of the tasks specified as dependencies have been run first. Exactly what you want. Head over to your browser of choice and open up http://localhost:3000. Hopefully, you’ll see something like this.

Step 8 – Watch for changes

Watching for changes means Gulp will constantly be checking for changes among your files. You only have to specify which files it has to watch.

gulp.task('watch', ['serve'], function () {
  gulp.watch(paths.src, ['inject']);
});

The ‘watch’ task will first wait for the ‘serve’ task to finish, only then will it start the watching. You tell Gulp to watch the files in the src directory. If it senses any changes it will fire the ‘inject’ Gulp task. Now whenever you save the changes in any of the specified files, Gulp will fire the ‘inject’ task. Awesome stuff! You can even go ahead and link the ‘watch’ task to the default task.

gulp.task('default', ['watch']);

Now you can just run gulp which will in turn run all of the tasks you have already built. I guess you just did Gulp all the things.

Step 9 – Building the dist

With the development environment up and running you’ve come to the point where you want to pack up your files to make them ready for production. Here’s where Gulp really flexes its muscles. Go ahead and install the following Gulp plug-ins.

npm install gulp-htmlclean --save-dev
npm install gulp-clean-css --save-dev
npm install gulp-concat --save-dev
npm install gulp-uglify --save-dev

And require them at the top of the gulpfile.js.

var htmlclean = require('gulp-htmlclean');
var cleanCSS = require('gulp-clean-css');
var concat = require('gulp-concat');
var uglify = require('gulp-uglify');

You can now re-use the majority of the already written tasks to create the build tasks.

gulp.task('html:dist', function () {
  return gulp.src(paths.srcHTML)
    .pipe(htmlclean())
    .pipe(gulp.dest(paths.dist));
});

gulp.task('css:dist', function () {
  return gulp.src(paths.srcCSS)
    .pipe(concat('style.min.css'))
    .pipe(cleanCSS())
    .pipe(gulp.dest(paths.dist));
});

gulp.task('js:dist', function () {
  return gulp.src(paths.srcJS)
    .pipe(concat('script.min.js'))
    .pipe(uglify())
    .pipe(gulp.dest(paths.dist));
});

gulp.task('copy:dist', ['html:dist', 'css:dist', 'js:dist']);

gulp.task('inject:dist', ['copy:dist'], function () {
  var css = gulp.src(paths.distCSS);
  var js = gulp.src(paths.distJS);
  return gulp.src(paths.distIndex)
    .pipe(inject( css, { relative:true } ))
    .pipe(inject( js, { relative:true } ))
    .pipe(gulp.dest(paths.dist));
});

gulp.task('build', ['inject:dist']);

The added plug-ins are piped between the gulp.src and gulp.dest commands. Everything else remains the same.

Only one more thing to add to you index.html  :

<!DOCTYPE html>
<html>
  <head>
    <!--[htmlclean-protect]-->
    <!-- inject:css -->
    <!-- endinject -->
    <!--[/htmlclean-protect]-->
  </head>
  <body>
    <div class="awesome">This is awesome!</div>

<!--[htmlclean-protect]-->
    <!-- inject:js -->
    <!-- endinject -->
    <!--[/htmlclean-protect]-->
</body>
</html>

The htmlclean plug-in cleans all comments out of your templates by default. You need to disable this behavior only for the inject comments.

Go ahead and run the ‘build’  task.

gulp build

Take a look at your project folder. You can now see a dist folder. The files within have been concatenated and minified, ready to be sent off to a production server.

Step 10 – Cleaning up

It’s not considered good practice to send off the tmp and dist folders to GitHub or whatever version control you may be using. You will need a way to delete them without much hassle.

npm install del --save-dev

Install the package above. This will make your life a lot easier. Require it at the top of gulpfile.js like so:

var del = require('del');

And add this snippet of code:

gulp.task('clean', function () {
  del([paths.tmp, paths.dist]);
});

Jump back to your command line and run gulp clean. Watch the magic. The tmp and dist folders have been deleted!

Another good practice would be to add the tmp and dist folders to your  .gitignore , making sure you definitely never push them to your GitHub repository.

If you missed any of the steps above feel free to jump over to the GitHub repository and catch your breath. The whole code base is over there, no need to worry.

You’ve faced the dark side and emerged a hero. Great job! This has been a “throw you in the deep end of the pool” crash course on task automation. I’m confident to say you can swim if you’ve come this far. Don’t stop here though. There’s so much more to learn about Gulp, this was just the beginning. But, regardless of that, it’s more than enough for you to get started with building serious applications and start shipping code. The best way of learning something is through concrete examples. Hack away and ship as much code as possible.

Hope you guys and girls had as much fun reading this article as I had writing it.

This post was originally published on medium.com