FIND WRONG URLs OF YOUR FILES INSIDE YOUR PROJECTS.

URL-fi is a really awesome and easy to use when you need to test if a wrong URL is in your file or files. However It only supported a command option for files to be tested. Most of the time when I tried to find some bad URL in my HTML files, there were tons of different HTML files in my project. So I thought "why don't we test recursively in a project with one single argument option?"

Start

The entire URL-FI is written by hyunjiLeeTech. So before I contributed to this project, I first posted an issue suggesting this idea and I would love to work on that in Slack. Happily, she replied "GO AHEAD!"

Find a files in a directory!

To find a files in a directory, it was necessary to find out if the file is folder or not folder.
So I used fs.readdirSync(path, { encoding: "utf-8",withFileTypes:true }). This returned DIRENT class. Inside the class, there are name of the files and type of them (1 is non-folder and 2 is folder). Best way to filter the files in JS is... of course using filter!

        files = files.filter(file=>{
            return !file.isDirectory()
        });

And then, let's find our wrong URL!

files.map(file=>{
            let content = 0;
            try {
                content = fs.readFileSync(`${path}/${file.name}`, {encoding: "utf-8"})
            } catch (error) {
                console.log(colors.yellow(`${error}`));
            }
            let links = content.match(regex);
            for (let i = 0; links && i < links.length; i++) {
                let link = links[i];
                if (link.startsWith("https://")) {
                    checkUrl(link);
                } else {
                    checkUrl(link);
                    if (sFlag) {
                        checkUrl(link.replace(/^http/, "https"));
                    }
                }
            }
        })

Add a -r option and f

As the following code, this program only support three options. s, h, v for https testing, help message and version each.

        if (arg.includes("s")) {
            ...
        }
        if (arg.includes("h")) {
            ...
        }

        if (arg.includes("v")) {
            ...
        }

Just below the code, I added r option which stands for 'recursively'.

        if (arg.includes("r")) {
            findFilesInDir(process.argv[3])
        }

Fix some issues

After I implemented all codes, I commit the change, but there were following issues.

1.In my previous code, I allow users to enter multiple arguments at the same time (e.g., url-fi -hvs test.html). But when I use your code, it doesn't allow to do this (e.g., url-fi -hs test.html). Can you please check this?
2.When I use -r argument with file (not directory), it gives an error (ENOTDIR: not a directory, scandir 'test.html). Can you make it print message rather than error?

so I have changed

        if (arg.includes("r")) {
            findFilesInDir(process.argv[3])
        }

        if (arg.includes("r")) {
            rFlag = true;
            findFilesInDir(process.argv[3])
        }

And add some try and catch block
So the entire findFilesInDir looks

try {
        let files = fs.readdirSync(path, { encoding: "utf-8",withFileTypes:true })

        files = files.filter(file=>{
            return !file.isDirectory() && !file.name.endsWith('.js')
        });   

        files.map(file=>{
            let content = 0;
            try {
                content = fs.readFileSync(`${path}/${file.name}`, {encoding: "utf-8"})
            } catch (error) {
                console.log(colors.yellow(`${error}`));
            }
            let links = content.match(regex);
            for (let i = 0; links && i < links.length; i++) {
                let link = links[i];
                if (link.startsWith("https://")) {
                    checkUrl(link);
                } else {
                    checkUrl(link);
                    if (sFlag) {
                        checkUrl(link.replace(/^http/, "https"));
                    }
                }
            }
        })
    } catch (error) {
        console.log(colors.yellow(`${error}`));
    }