Node.js Cron: Handling Overlapping Tasks Like a Noob

One common cron problem is when a cron job takes longer to complete than the interval between two consecutive runs. In this case, the second instance of the job will start before the first one has finished, leading to overlaps and potentially unexpected behavior such as duplicates.

The script consists of a job that is run every 5 seconds using the CronJob library, and a delay function that simulates a long running task.

The script uses a simple flag isJobRunning to prevent overlapping runs of the job, and a jobSkippedCount variable to keep track of how many times the job has been skipped due to overlapping.

We're going to use nanoid to know job ids and cron to schedule jobs.

npm install nanoid
npm install cron

Let's dive into the code:

import { CronJob } from 'cron'
import { nanoid } from 'nanoid'

let isJobRunning = false
let jobSkippedCount = 0

const delay = (ms) => {
  return new Promise((resolve) => {
    setTimeout(resolve, ms)
  })
}

const job = new CronJob('*/5 * * * * *', async () => {
  if (isJobRunning) {
    jobSkippedCount++
    console.log("__ JOB SKIPPED COUNT __ ", jobSkippedCount)
    if (jobSkippedCount > 2) {

      /* If a job takes more than 15 seconds 
         to complete we can stop this job, 
         throw a error and start a new job */

      console.log("__ TIMEOUT __")
      jobSkippedCount = 0
      isJobRunning = false

    }
    return
  }
  isJobRunning = true
  try {
    const id = nanoid(10)
    console.log("\n\n__ JOB RUNNING __ ", id, new Date())

      /* Your  functionality goes here 
    I have used a delay function to 
    simulate a long running job */

    await delay(16000)
    isJobRunning = false
    jobSkippedCount = 0
    console.log("__ JOB COMPLETED __ ", id, new Date())

  }
  catch (err) {
    console.error(err);
  }
})

job.start();

There are 3 scenarios that happen with this our cycle is 5 seconds once and max skip count is 2

1. Everything is smooth

delay(2000) - The cron job executes and finishes its task before the next cycle, producing the following output:

__ JOB RUNNING   __  EC6MqpB4FG 2023-04-06T17:53:35.004Z
__ JOB COMPLETED __  EC6MqpB4FG 2023-04-06T17:53:37.017Z
__ JOB RUNNING   __  6nutPdea6_ 2023-04-06T17:53:40.004Z
__ JOB COMPLETED __  6nutPdea6_ 2023-04-06T17:53:42.005Z

2. Skipped a job but no TIMEOUT

delay(7000) - The cron job executes and finishes its task by skipping one cycle, producing the following output:

__ JOB RUNNING   __  soR8V3YrQp 2023-04-06T17:54:15.006Z
__ JOB SKIPPED COUNT __  1
__ JOB COMPLETED __  soR8V3YrQp 2023-04-06T17:54:22.019Z
__ JOB RUNNING   __  8-LBtb85Jq 2023-04-06T17:54:25.002Z
__ JOB SKIPPED COUNT __  1
__ JOB COMPLETED __  8-LBtb85Jq 2023-04-06T17:54:32.004Z

3. Reaches Timeout

delay(17000) - The cron job fails to complete within timeout so a new job starts: Regardless whether the previous job completed or not. We will start the job after timeout.

You may need to kill your logic using event emitters in order to stop the function execution.

__ JOB RUNNING   __  lrMOLDgtzm 2023-04-06T18:00:05.004Z
__ JOB SKIPPED COUNT __  1
__ JOB SKIPPED COUNT __  2
__ JOB SKIPPED COUNT __  3
__ TIMEOUT __
__ JOB COMPLETED __  lrMOLDgtzm 2023-04-06T18:00:22.015Z
__ JOB RUNNING   __  qPK1lQ8QuV 2023-04-06T18:00:25.003Z
__ JOB SKIPPED COUNT __  1
__ JOB SKIPPED COUNT __  2
__ JOB SKIPPED COUNT __  3
__ TIMEOUT __
__ JOB COMPLETED __  qPK1lQ8QuV 2023-04-06T18:00:42.003Z

isJobRunning - is a flag that indicates whether the job is currently running.

jobSkippedCount is a counter that keeps track of how many times the job has been skipped due to overlaps. This also helps **TIMEOUT** , We can set a maximum number of skips allowed like 5. Then if the job is skipped for 5th time then it will reset and start a new job.

delay() - A function that returns a promise that resolves after a given number of milliseconds, Typically your logic.

This is more like a temporary hack to avoid overlaps , these timeout scenarios are so dangerous.

A more scalable approach is to use a queueing system to manage the jobs. This can allow multiple instances of the job to be queued up and executed in sequence, without overlaps or duplicates. Popular queueing systems for Node.js include Bull, Bee-Queue, and Agenda.

Thanks for reading
🕊 Peace

Try Our new product for free!

DocsAI - Create AI support agents with your documents in the most affordable price, starts at 0$. Don't need a bot , but need ai help on your docs just upload and start chating !

Using for a company ? Check out our pricing Just contact me for personalized pricing !