DEV Community

Discussion on: I wrote a crawler for the first time.

Collapse
 
crimsonmed profile image
Médéric Burlet

I used cheerio a while but realized when processing a lot of pages it's very slow.
I'd recommend using regex I cut down processing time by 80%.

Good job though it's a great start!

Collapse
 
kayla profile image
Kayla Sween

Good to know! I’m just collecting data from one page for now, but I’ll definitely keep that in mind. Thank you!

Some comments have been hidden by the post's author - find out more