Today I'm introducing a piece of code that I worked on in my enjoyment time, it is called scrapyx or X.
Simply it is a scrapy module (subcommand) that creates a fastapi based server to enable you to control scrapy as you want.
It is a very simple library that uses redis as a persistent queue.
It is fully configurable via the settings.py file of your scrapy project.
you can find it here.
If you found it useful, just start it on github ♥️
Top comments (0)