DEV Community

Cover image for How To Build Scheduled Task on Github Libraries Releases via Slack Apps and NodeJs
Yazan Tarifi
Yazan Tarifi

Posted on

How To Build Scheduled Task on Github Libraries Releases via Slack Apps and NodeJs

In this Article, We Will build a Slack Application To Send Messages to Slack Channel when Any Library you follow on Github Publish New Release

Description

As a Developer, you worked on a lot of projects and inside these projects, you should use Libraries to implement a Feature whatever if the library is a 3rd part library or native library from the Framework itself and this is totally fine, The Problem I faced when I use The Libraries that I should check or Follow Someone on Twitter, Reddit or medium to get notifications on the Libraries that I'm using inside my Project, but if I didn't open any application from social media apps I will never know if any library pushed new Version on their Repository or maybe I know about this updates after 2 weeks and for this reason I need to get Notifications in the same day of the Release because some libraries are still pushing major release changes and it's really a big problem if we discover this Updates after 2 Weeks from the Release date

The Simplest Solution to Build Scheduler for This Process

We should Create a Scheduled Task to check on all libraries that we are using inside our projects to get notifications on the same day inside this release and we gonna build it From Scratch with some tools that will help us build this Task

The Components Used inside This Project

  1. Node Js Project
  2. Slack Application
  3. The Source Links of The Libraries

The Full Example will be Available at the End of the Article

First Thing is to Build The Backend Project

We Will Use NodeJs to Build Backend Side of this Project and Especially NestJs Framework and Typescript and We Need to Use one of the Backend Frameworks To Use Cron Jobs and CronJob is a Scheduled Event That Will Trigger Some Actions in Specific Time that You Specify it when Create the instance of the Task Service

You can Use any Backend Framework because most of them has Cron Job implemented inside each one of them but for me, I Prefer to Build these Things in NestJs

Second Thing is to Create a Slack Application

Slack Application is a Ready Api from Slack To Create Application with Id, Name, Logo That will Send Messages To Members, Channels inside your Workspace and for this project, we will configure this application to send messages with the New Versions of the Libraries on specific Channel

The final Part is Configuring The Source of Libraries

This is Really Important is to Know each Library which source is the Best to Fetch it, For Example, when I build Android Applications I have multiple Sources to Fetch Libraries not all of them from one Source like (MavenCentral, GoogleMavenRepository, GithubRepository, GradlePortal) and We Need to Find a way to Fetch The Libraries from Multiple Sources inside the Same Project
But In this Part, I saw Something Common Between all of them is 90% of the Libraries Source code inside Github Repositories and all of them has Releases and Tags Version so We Can Track all of them from a Common Source which is (Github API)

Now Let's Start with The Implementation of the Project and We Will Start with Creating Slack and Github Configuration

The First Step is To Configure Slack and Github to Get Tokens, Keys that we Need to Use inside our NodeJs Project

First Step Create Slack Application inside Your Workspace and Specify the Logo and Name of the Application Then Add The Following Configuration inside App Manifest

_metadata:
  major_version: 1
  minor_version: 1
display_information:
  name: Zilon
features:
  app_home:
    home_tab_enabled: true
    messages_tab_enabled: true
    messages_tab_read_only_enabled: false
  bot_user:
    display_name: Zilon
    always_online: true
oauth_config:
  redirect_urls:
    - https://example.com/slack/auth
  scopes:
    bot:
      - commands
      - chat:write
      - chat:write.public
settings:
  org_deploy_enabled: false
  socket_mode_enabled: false
  token_rotation_enabled: true
Enter fullscreen mode Exit fullscreen mode

Now You Need to Create a Github Application from OAuth Settings Inside Your Github Account Settings and Take the Client Id and Secret Client Id then Save Them on Text File with Slack Keys (Token, Signing Key) and All of these Keys can be Found inside the Application Settings In General Tab Then Save All Keys and Tokens in One Text File because We Will Need them Later

Now Create Channel inside Your Slack Workplace and Invite the Application you created inside this channel to get access to the Channel

Now Create NestJs Project

Generate New project with NestJs By Executing the Following Commands inside Your Terminal

npm install -g @nestjs/cli
npx nest new project-name

cd project-name
npm install --save @nestjs/schedule
npm install --save-dev @types/cron
npm install axios
npm install @slack/bolt
Enter fullscreen mode Exit fullscreen mode

Now We Want to Add Cron Job to start Scheduled Task

This Task will be started at a specific time like the following example

import { Injectable } from "@nestjs/common";
import { Cron, CronExpression } from "@nestjs/schedule";

@Injectable()
export class TasksService {

  @Cron(CronExpression.EVERY_DAY_AT_1AM, {
    name: "dependencies"
  })
  handleCron() {
   // Handle Libraries Checks
  }

}

// Now Declare this TaskService inside your App Module
import { Module } from '@nestjs/common';
import { AppController } from './app.controller';
import { AppService } from './app.service';
import { ScheduleModule } from '@nestjs/schedule';
import { TasksService } from "./task/TasksService";

@Module({
  imports: [ScheduleModule.forRoot()],
  controllers: [AppController],
  providers: [AppService, TasksService],
})
export class AppModule {}
Enter fullscreen mode Exit fullscreen mode

Now We Will Use Axios To Send API Requests on GitHub to check all Libraries and get Releases Using GitHub API v3

import axios, { Axios } from "axios";

export class NetworkInstance {

  public static SUCCESS_RESPONSE_CODE = 200;

  // General Url's For Requests
  public static GROUP_ARTIFACTS = "/group-index.xml";
  public static GITHUB_REPOS_KEY = "/repos/";
  public static GITHUB_RELEASES_KEY = "/git/refs/tags";

  public static getGithubRepositoriesInstance(): Axios {
    let instance = axios.create({
      timeout: 5000,
      baseURL: "https://api.github.com",
      responseType: "json",
      headers: { Accept: "application/json" }
    });

    instance.interceptors.request.use(request => {
      console.log("Github Starting Request", request.url);
      return request;
    });

    return instance;
  }

}
Enter fullscreen mode Exit fullscreen mode

Now the Functionality will be like the Following, We want to Store all libraries that we need to check every day then we will store the latest released tag and on each day the scheduler will send a request to the GitHub repo to check the latest tag if not similar to stored tag then we will send a slack message with this library

In this stage, you have the option to store all of them in the way you like if you want you can use the database to store all of them but I prefer to write all of them inside JSON file in this type of project

This is a Simple Example of how to check all of them in this stage you will need to get Github app clientId, SecreteId from the GitHub app that you created in your GitHub profile settings

export class GithubDependenciesManager {

  private static GITHUB_LIBRARIES_FILE = "github-libraries.json";
  private static CONSOLE_LOGGING_KEY = "[Github Dependencies Manager]";
  private static GITHUB_CACHE_FILE = "github-libraries-cache.json";
  private static CONFIG_FILE = "config.json";

  /**
   * Main Method to Start inside This Manager
   * 1. Create and Validate the Local Json Files
   * 2. Start Validating The Old Files if Exists, if Not Will Create Default Files
   * 3. Will loop on all of them to see if the current version on github is similar to cached version
   * if not will send message on slack channel via config.json token, channelId
   */
  public async validateGithubLibrariesFile() {
    const fs = require("fs");
    this.createGithubLibrariesFile();

    let configFile = new ApplicationConfigFile("", "", "", true, "", "");
    if (fs.existsSync(GithubDependenciesManager.CONFIG_FILE)) {
      const dataFile = fs.readFileSync(GithubDependenciesManager.CONFIG_FILE);
      configFile = JSON.parse(dataFile.toString());
    }

    let librariesInformation = new Array<GithubRepositoriesInformation>();
    let librariesFile = new GithubContainerFileContent(new Array<GithubLibrary>());
    if (fs.existsSync(GithubDependenciesManager.GITHUB_LIBRARIES_FILE)) {
      const data = fs.readFileSync(GithubDependenciesManager.GITHUB_LIBRARIES_FILE, "utf8");
      librariesFile = JSON.parse(data);
      for (let i = 0; i < librariesFile.libraries.length; i++) {
        const library = librariesFile.libraries[i];
        await timer(5000);
        await NetworkInstance.getGithubRepositoriesInstance().get<Array<GithubRepositoryRelease>>(this.getGithubRequestUrl(configFile, NetworkInstance.GITHUB_REPOS_KEY + library.url + NetworkInstance.GITHUB_RELEASES_KEY), {
          method: "get"
        }).then((response) => {
          if (response.status == NetworkInstance.SUCCESS_RESPONSE_CODE) {
            librariesInformation.push({
              name: library.name,
              url: library.url,
              releases: response.data
            });
          } else {
            console.error(GithubDependenciesManager.CONSOLE_LOGGING_KEY + " Exception : " + response.data + " Response : " + response.statusText);
          }
        }).catch((exception) => {
          console.error(GithubDependenciesManager.CONSOLE_LOGGING_KEY + " Exception : " + exception);
        });
      }

      this.validateGithubRepositoriesReleasesVersions(librariesInformation);
    }
  }

  private getGithubRequestUrl(config: ApplicationConfigFile, url: string): string {
    return url + "?client_id=" + config.githubClientId + "&client_secret=" + config.githubClientSecrete;
  }

  /**
   * After Get all Releases From Github Api to Get All  Releases Information
   * We Will Validate the First Release With The Cached Versions if Not Equals
   * Will Send Slack Message with The New Version Triggered ...
   * @param libraries
   * @private
   */
  private validateGithubRepositoriesReleasesVersions(libraries: Array<GithubRepositoriesInformation>) {
    const fs = require("fs");
    let librariesFile = new GithubLibrariesCacheContainer(new Array<GithubCacheLibrary>());
    const requireUpdateLibraries = new Array<LibraryUpdateModel>();
    fs.readFile(GithubDependenciesManager.GITHUB_CACHE_FILE, "utf8", function readFileCallback(err, data) {
      if (err) {
        console.log(err);
      } else {
        librariesFile = JSON.parse(data);
        for (let i = 0; i < librariesFile.libraries.length; i++) {
          const cachedLibrary = librariesFile.libraries[i];
          for (let j = 0; j < libraries.length; j++) {
            const triggeredLibrary = libraries[j];
            if (cachedLibrary.name.includes(triggeredLibrary.name) && triggeredLibrary.releases != null) {
              if (!cachedLibrary.release.includes(triggeredLibrary.releases[triggeredLibrary.releases.length - 1].ref.replace("refs/tags/", ""))) {
                console.log(GithubDependenciesManager.CONSOLE_LOGGING_KEY + " Library Need Update : " + triggeredLibrary.name + " Version : " + cachedLibrary.release + " Updated Version : " + triggeredLibrary.releases[triggeredLibrary.releases.length - 1].ref.replace("refs/tags/", ""));
                requireUpdateLibraries.push({
                  isGithubSource: true,
                  releaseUrl: "https://github.com/" + triggeredLibrary.url + "/releases",
                  version: triggeredLibrary.releases[triggeredLibrary.releases.length - 1].ref.replace("refs/tags/", ""),
                  url: "https://github.com/" + triggeredLibrary.url,
                  artifact: "",
                  groupId: "",
                  name: triggeredLibrary.url.split("/")[1]
                });
              }
            }
          }
        }

        new MessagingManager().sendMessageUpdateDependencies(requireUpdateLibraries);
        GithubDependenciesManager.saveNewGithubRepositoriesCacheFile(libraries);
      }
    });
  }

  /**
   * After Updating the Required Dependencies and Send All of them inside Messages in Slack
   * Now we Want to Refresh the Json File with New Cached Data
   * To Save The Notified Releases
   * @param libraries
   * @private
   */
  private static saveNewGithubRepositoriesCacheFile(libraries: Array<GithubRepositoriesInformation>) {
    const fs = require("fs");
    if (fs.existsSync(GithubDependenciesManager.GITHUB_CACHE_FILE)) {
      const librariesFile = new GithubLibrariesCacheContainer(new Array<GithubCacheLibrary>());
      for (let i = 0; i < libraries.length; i++) {
        try {
          const library = libraries[i];
          librariesFile.libraries.push({
            name: library.name,
            release: library.releases[library.releases.length - 1].ref.replace("refs/tags/", "")
          });
        } catch (error) {
          console.error(error);
        }
      }

      const json = JSON.stringify(librariesFile, null, "\t");
      fs.writeFile(GithubDependenciesManager.GITHUB_CACHE_FILE, json, "utf8", (exception) => {
        if (exception != null) {
          console.error(GithubDependenciesManager.CONSOLE_LOGGING_KEY + " Exception : " + exception);
        }
      });
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

Now We Have the Updated Libraries inside Array and we want to loop on them and send messages via slack API using the Signing Key, Secret Key

private static sendSlackMessage(configFile: ApplicationConfigFile, message: string) {
    try {
      MessagingManager.getSlackApplicationInstance(configFile.signingSecret, configFile.token).client.chat.postMessage({
        channel: configFile.channelId,
        mrkdwn: true,
        text: message,
        as_user: true,
        parse: "full",
        username: "Zilon"
      }).then((response) => {
        console.log("Slack Message Response : " + response.message.text);
      }).catch((exception) => {
        console.error(exception);
      });
    } catch (error) {
      console.error(error);
    }
  }
Enter fullscreen mode Exit fullscreen mode

Use this Method inside your Loop and Create your Own Message on each Library, in my case I have added all libraries and their documentation links, official websites that I need to my JSON file, and on each message, I check all of them and send them with the message

In Slack Application Create a Channel and invite the app to this channel by typing /invite then pick the application and inside the code when you want to send a message on the channel you should write it to be like this (#general)

The Scheduled Task Result

Full Example is Available on Github

Github Repository

Top comments (0)