DEV Community

keonakhon
keonakhon

Posted on

Monitor all the files changed on the Linux server with Python watchdog

Watchdog is a Python library used to monitor files and directories. On the Linux server, it depends on inotify (inode notify), which is a kernel subsystem of Linux used to monitor changes to the filesystem and report those changes.

I had tried to use Watchdog to monitor files changed since the root level "/" on macOS, and it worked fine. However, on the Linux server, I encountered an error.

[Errno 2] No such file or directory
Enter fullscreen mode Exit fullscreen mode

The reason for this error to occur is that it couldn't catch up with the Process ID in the /proc directory, as the files or directories were created and disappeared instantly; it's too fast.

Here are the details of the directories and files inside /proc: there's a directory named 666 in the first ls command run and 667 in the second ls command run.
/proc directory details

Those are the numbers of Process IDs we saw in the ps command.
ps command

Make sure you have already installed watchdog library.

pip3 install watchdog
Enter fullscreen mode Exit fullscreen mode

The provided code below is going to monitor all files with .txt, .js, .py, and .php extensions that were created or updated. It will monitor in every directory inside "/". I have to ignore /proc, as well as all symlink directories or files in order to prevent the error and improve performance, respectively.

import time
from datetime import datetime
from os import listdir
from os.path import isdir, join, islink
from watchdog.observers import Observer
from watchdog.events import PatternMatchingEventHandler


# Scan for newly created files
class MyHandler(PatternMatchingEventHandler):
    def __init__(self):
        super().__init__()
        patterns = ['*.txt', '*.js', '*.py', '*.php', '*.html', '*.css']
        PatternMatchingEventHandler.__init__(self,
                                             patterns=patterns,
                                             ignore_directories=True)

    def on_created(self, event):
        current_time = datetime.now()
        if not event.is_directory and event.event_type == 'created':
            print(f"[created] {current_time} - {event.src_path}")

    def on_modified(self, event):
        current_time = datetime.now()
        if not event.is_directory and event.event_type == 'modified':
            print(f"[modified] {current_time} - {event.src_path}")


def main():
    try:
        root_path = "/"
        directories = []
        for entry in listdir(root_path):
            entry_path = join(root_path, entry)
            if isdir(entry_path) and not islink(entry_path) and entry_path != '/proc':
                directories.append(entry_path)

        observers = []

        observer = Observer()
        for directory_to_monitor in directories:
            event_handler = MyHandler()
            observer.schedule(event_handler,
                                path=directory_to_monitor,
                                recursive=True)
            observers.append(observer)

        observer.start()

        try:
            while True:
                time.sleep(1)
        except KeyboardInterrupt:
            for obs in observers:
                obs.stop()
        for obs in observers:
            obs.join()
    except Exception as error:
        print(error)


if __name__ == '__main__':
    main()
Enter fullscreen mode Exit fullscreen mode

If you encounter an error related to inotify limits, please follow this Stack Overflow anwser.

Lastly, you might see duplicate messages of the same file on both events (created and modified) because it depends on the command you used. It's not actually an issue with watchdog or my code. For more details, please refer to this Stack Overflow anwser and watchdog's repo.

P.S. It might not be an error in your case, but I hope this is helpful.



Looking for qualify software with affordable price? contact: funwarn.dev@gmail.com

Top comments (0)