Basically I am designing a system, which can have thousand of cron jobs. I am creating a UI where one can create cron job, and specify the interval in it. What system design you will suggest making it scalable?
More Details: My Question is in terms of the backend infrastructure. So I am going to use the package called npmjs.com/package/html-differ to compare two HTML. There will be a user interface where the user will define the webpage URL and interval time where webpage changes will be checked. Let's assume we have 10k URL to check with different time intervals. I wanted to have a recommendation about the database or queuing system like rabbitmq i should use to make it scalable and fault tolerance.