How I created a tool to efficiently sort a folder

It is a lockdown period and we all have a lot of spare time. My Downloads folder is messed up with a huge number of files of different types. Every time I decide to clean that folder I go one by one and delete unwanted files. That is not a big task when the numbers are low. But this time there are 2000 files.

So one thing was clear in my mind, We need automation.

I am a fan of shell commands and always fascinated by how big tasks can be done with a few lines. Now speaking to the point we need to create a shell script that will sort the files in the order in which it will be easy for us to keep what is important and delete what is not. I was analyzing a few files noted their types.

Files that are very important

1. Receipts of online transactions -> PDF
2. Media from the phone -> JPEG and MOV
3. Documents -> DOCX, DOC, PPT etc.

Files that are not important and can be downloaded later

1. Movies -> MKV, MP4, VOB, AVI
2. Torrents ->TORRENT
3. Songs -> MP3
4. Installer or executables -> DMG

Files that can be important

1. Txt files
2. Zip files

So we will now sort files with their types.
The first thing is to get all the types of files that exist in the current directory. A simple ls command in Linux can do the job. but it will list directories too.
Note: We are not sorting the directory in a recursive manner, only considering files that are present in the current directory.

Having a look at the man page of ls solves our problem. -p option can be used to put a ‘/’ after each filename if that file is a directory. Yes you heard it right “Everything in a Linux system is a file, even directories”.
Now the code is

$ ls -p

To filter only files we will pipe the output into grep with -v option to ignore the result containing “/”

$ ls -p | grep -v /

Now we have a list of all the files, from now we only need extensions. Generally, we have the file name as “name.extension”, so we will use “.” as the separator and will get the second column. So the code will be

$ ls -p | grep -v / | cut -d “.” f2

But I see a problem here,

What if there are files that contain “.”(dot) in the filename for example- “some.thing.extenion”. The above code will take “thing” as an extension. We have a solution for this. Try
understanding the below code.

$ ls -p | grep -v / | rev | cut -d “.” f1 | rev

Reversing the string and using cut with “.” as the delimiter to get the first column will give us the last column in the reversed string, again reversing will give the actual extension.

This will give us all the extension in a list (duplicates for multiple files with the same types).
We will store this in a file. To make a temporary file we can use mktemp. By the below code we will get a list of
extensions in a temp file.

$ temp = `mktemp`
$ ls -p | grep -v / |  rev | cut -d "." -f1 | rev > $temp

Sure we will remove duplicates.

$ utemp = `mktemp`
$ sort $temp | uniq > $utemp

We have $utemp file that has a list of all unique extensions.
Now only thing left is to pick a line from the $utemp file and create a directory with that name and copy all files with that extension.

while read ext;
do
   mkdir $ext
   mv *.$ext $ext/
done < $utemp

That’s it we are now done sorting. Remove the temp files now.

rm $temp
rm $utemp

So, bringing all these commands together in a file and giving permission to execute

chmod +x filter.sh

We can place this script in the Downloads directory and run it.

I have added a few echo commands to know the progress.
Taking about the results we get, After running this script I simply moved the important files in a backup drive. Quickly deleted folders of files that were not important, looked for other file types and kept what was important. These other files were small in number, so the overall process took only a few minutes.

You can see this project on GitHub here.
This is my first article, hope you like it.

Twitter - 0xjoshi