DEV Community

Discussion on: How to crawl website using #bash script?

Collapse
 
msamgan profile image
Mohammed Samgan Khan

that's what I was wondering, wget will only download the page. Crawling means going through the content of the page.

Collapse
 
powerexploit profile image
powerexploit

I admire your response I know its not a pure crawling but if I only want to crawl one page then I will use wget
Or to download or crawl whole I will surely some python stuffs