Email automation with Node and Dropbox

I choose a lazy person to do a hard job. Because a lazy person will find an easy way to do it. - Bill Gates

This quote, I feel, describes why I love being a programmer. We are generally lazy yet quite good at solving problems, including our own! Sometimes, this may not involve even writing code, but most of the time you can assume we did 🤓

So, my most recent problem: My Mum is required to email an invoice to her client fortnightly, and is fairly computer illiterate. This is handwritten and needs to be scanned as a PDF, and because I’m not around the house much anymore, my sister scans it for her, but she is too lazy to organise the email, so she adds it to my Dropbox so that, finally, I can email it to the client.

I hate this entire process... and receiving "Have you sent that invoice?" text messages.

The steps involved for me are:

1. Downloading the file from Dropbox
2. Logging into Mum's email account
3. Typing a very generic email to the client
4. Attaching downloaded file
5. Send the email

Solution: AUTOMATE ALL THE THINGS!

Javacript/Node to the rescue!

Javascript and Node seemed the most appropriate for my solution as I knew I would need to run a server-side application to check my Dropbox regularly to find the file. I am also in the process of trying to become more of a fullstack developer, so I knew this would be a great learning exercise.

Recently I had completed Wes Bos' Learn Node course which greatly assisted in the design choices for my final solution. This included but was not limited to: Node, ES6, Promises, Nodemailer, Node Cron, and shell scripting for continuous deployment (but I will go into this further in my next post - follow me on Twitter!).

I won't go into too much more detail about specifics of the application as you can just take a look at it here on GitHub. However, I would like to go on further to explain the issues I faced, how I could improve the application, and what utilities made writing this application a joy to create!

Promises and the Dropbox API

Previously, I had worked with the Dropbox API using PHP to create an application that would randomly pick a set of photos and display them onto a webpage. This was fairly basic and just didn't feel right because we were just calling the API with a curl function and I am trying to use less PHP where I can these days.

When it came round to building the invoice application, I found out that Dropbox had created a Javscript SDK to interact with the API. This was exciting, and even more exciting when I read the documentation to find out that it was promise based! Promises mean that you can easily chain a few API calls to get the data that you require or perform the actions you need to with little to no effort.

Here is an example of a promise chain to download a file. It assumes you are passing the path of the file, which you can get easily using another API call/promise.

const Dropbox = require('dropbox');
const dbx = new Dropbox({ accessToken: process.env.ACCESS_TOKEN });

exports.getFile = function (path) {

  const file = dbx.filesDownload({ path: path })
    .then(function (response) {
      return response;
    })
    .catch(function (error) {
      console.log('Error downloading the file ❎');
      return Promise.reject(error);
    });

  return file;
};

I can't believe it's not butter! So simple, much file. 🐕

Just to show you I'm not bluffing, I created another function that I called once the email was sent. This moves the file in Dropbox to another folder to indicate that this invoice has been sent.

exports.archiveFile = function (path, subFolderName) {
  const archivedFile = dbx.filesMove({
    from_path: path,
    to_path: '/sent/' + subFolderName + path,
    allow_shared_folder: true,
    autorename: true,
    allow_ownership_transfer: true
  })
    .then(function (fileMove) {
      console.log('File ' + fileMove.name + ' archived successfully! 🗳️');
      return fileMove;
    })
    .catch(function (error) {
      console.log('Error archiving the file 💥');
      return Promise.reject(error);
    });

  return archivedFile;
};

Here I pass the client name as the subFolderName which means that you get a well organised file path like /sent/client-name/INV0001.PDF

But what about the Email?

Oh right, so before we go archiving the file, we obviously send the email. The creation of this email involves a few small parts but the sending of it is very straightforward.

As my Mum has multiple clients, the solution needed to incorporate some form of reusability and scalability. I managed this by creating each client as a JSON file that would look something like this:

{
  "name": "Recipient",
  "email": "test@email.com",
  "subject": "An interesting Email Subject",
  "text": "Hi John Doe,\n\nInvoice attached.\n\nKind Regards,\nJane Doe",
  "file-prefix": "INV"
}

This ensured each file to be sent from Dropbox would be mailed out based on its filename prefix, allowing each client to have a different name, email, subject, or text within the email. This also means that if she ever get more clients, it is just a matter of creating new JSON files to also be a part of the automation train. 🚂

Using the data above and the calls to the Dropbox API we are able to build our email and send it using Nodemailer.

The code for sending an email through Nodemailer is a function call with a few option parameters (as seen below). These are passed then used in conjunction with a transport function, with most of its config set using environment variables (because you don't want people spamming you or knowing your SMTP credentials).

In this application, I added the file using a binary file stream/buffer which sounds far more complicated than it is or needs to be. In reality, it just means we get the binary version of the file from Dropbox, save it as a variable, pass it to the buffer, and then it becomes a file attachment.

 ...

 const sendInvoice = attachedFile.searchFilePath(filePrefix)
      .then(function (filePath){
        foundFilePath = filePath;
        const file = attachedFile.getFile(filePath);
        return file;
      })
      .then(function (file) {
        const mailPromise = mail.send({
          email: recipient.email,
          subject: recipient.subject,
          text: recipient.text,
          attachments: {   // binary buffer as an attachment
            filename: file.name,
            content: new Buffer(file.fileBinary, 'binary'),
            encoding: 'binary'
          }
        });
        return mailPromise;
      })

  ...

Voila! There is the majority of the application in just a few function calls. If mailPromise resolves, then our email will send.

To test email sending while in development, using a service such as Mailtrap is a lifesaver as it is free and doesn't fill up anyone’s inboxes 😆

Once I got to production, I changed it over to Mailgun as you can send up to 10,000 emails every month for free!

Automation 🤖

So it seems the application covers all your previously mentioned steps... but what makes it automatic?

Not much really, just run the function once every hour (or as much as you'd like) using a cron. A cron is "a command to an operating system or server for a job that is to be executed at a specified time". In this case, the application checks if there are any files to be sent. If there are, execute the rest of the application; if not, don't do anything. As previously mentioned, promise chains make this process a breeze.

Like everything, there is always room to improve. The cron could be removed by only running the function when a file has been uploaded, and obviously you can't just do this with the API but you smart cookies out there would've realised you can do this with the use of webhooks (but that’s for another time).

Conclusion

This project was super enjoyable! I learnt a multitude of things from Node to Shell scripting, from Cron jobs to Promises. Little side projects like these really push you forward as a developer. They allow for you to be the perfectionist you want to be and create something to improve your life (and sometimes others’ lives too) in more ways than one.