As my previous blog, I use the python web Crawler library to help crawl the static website. For the Scrapy, there can be customize download middle ware, which can deal with static content in the website like JavaScript. However, the Scrapy already helps us with much of the…
Write a highly efficient python Web Crawler
For further actions, you may consider blocking this person and/or reporting abuse
Top comments (0)