DEV Community

KiminLee
KiminLee

Posted on

Add -i/--ignore option to insane-cli(Finding broken URL CLI program)

getting start

insane-cli is a great tool for checking if there is any broken URLs in your files. However, there is still a room for enhancement. This week I have decided to add a new feature so that the URLs can be excluded by passing ignore.txt.

New feature

The --ignore, -i, \i flag is optional, and if present should take a filepath and use it to load and parse an ignored URLs text file. All URLs should be checked against this list to see if they need to be ignored.

For example,

# Ignore All Seneca College URLs
https://senecacollege.ca
# Ignore all Learn@Seneca URLs
https://learn.senecacollege.ca/
Enter fullscreen mode Exit fullscreen mode

Any text following # must not be read!!

Coding 1 -ignore.txt files-

first to test the feature I have created ignore.txt file like this

# 2. File with a comment and single URL that matches any URL
# that starts with https://www.google.com, for example:
#
# https://www.google.com/index.html would match
# https://www.google.com/ would match
# https://www.google.ca would NOT match
https://www.google.com
https://stackoverflow.com/
https://wiki.cdot.senecacollege.ca
Enter fullscreen mode Exit fullscreen mode

Coding 2 -add -i/--ignore flag-

insane-cli program check every single argument that would be passed. So if certain flag matches the condition, each task will be exectued.

let argv3 = process.argv[2]
let argv4 = process.argv[3]
...
if (argv3 != null) {
    if (argv4 != null) { // for URL process
        if (argv3 === '-url' || argv3 === '/url') {
        ...
        } else if (argv3 === '-w' || argv3 === '/w') {
        ...
        } else {
          console.log('Command not found!');
    ...
}
Enter fullscreen mode Exit fullscreen mode

To add a new flag -i, I need to add one more condition

...
else if (argv4 === '-i' || argv4 === '--ignore'){
...
} else{
   console.log('Command not found!');
}
Enter fullscreen mode Exit fullscreen mode

So, if the process.argv3 is -i or --ignore, it would run the else if (argv4 === '-i' || argv4 === '--ignore') block.

Coding 3 -read ignore.txt-

fs.readFile is basically read whole txt at once. So it doesn't read the txt line by line. All we want to do in this feature is checking line by line, so that if the URL is following '#', it would be ignore.

To do that, I have added new library readline. This makes the program read the file line by line. First we need to read stream const fileStream = fs.createReadStream(process.argv[4]);
then, create the readline interface

    const fileStream = fs.createReadStream(process.argv[4]);
    const rl = readline.createInterface({
        input: fileStream,
    });
Enter fullscreen mode Exit fullscreen mode

Now, we read the text file line by line,

If there is '#' ignore the line

    let ignoreURL = []
    ...
    for await (const line of rl) {
        if (!line.startsWith('#')) {
            ignoreURL.push(line)
        }
    }
    ...
Enter fullscreen mode Exit fullscreen mode

So, the readIgnoreFile() function looks like:

const readIgnoreFile = async () => {
    let ignoreURL = []
    const fileStream = fs.createReadStream(process.argv[4]);
    const rl = readline.createInterface({
        input: fileStream,
    });

    for await (const line of rl) {
        if (!line.startsWith('#')) {
            ignoreURL.push(line)
        }
    }
    return ignoreURL;
}
Enter fullscreen mode Exit fullscreen mode

Put altogether

} else if (argv4 === '-i' || argv4 === '--ignore') {
            readIgnoreFile().then(lists => {
                let filename = argv3;

                try {
                    if (fs.existsSync(filename)) { // check if file exist
                        (async () => {
                            await fs.promises.readFile(filename)
                                .then(function (data) {
                                    let finalURLs = getURLs(data)
                                    finalURLs = finalURLs.filter(url => {
                                        let flag = true;
                                        for (let i = 0; i < lists.length; i++) {
                                            if (url.includes(lists[i])){
                                                flag = false;
                                            } 
                                        }
                                        return flag
                                    })
                                    printURLStatus(finalURLs)

                                })
                                .catch(function (error) {
                                    console.log(error);
                                })
                        })()
                    }
Enter fullscreen mode Exit fullscreen mode

PR

I let the owner of insane-cli know about this enhancement and make a PR on my fork repo. After he check the PR, he will remotely merge my branch commit to his master branch

Top comments (0)