Today, drawing on our team's years of experience, we're launching Crawlee, the the web scraping and browser automation library for Node.js that's built for fastest development and maximum reliability in production.
🎛 Automated parallelization and scaling of crawlers for top performance
🐾 Avoids blocking using smart sessions, proxies, and browser fingerprints
🚎 Simple management and persistence of queues of URLs to crawl
🗜 Written completely in TypeScript for type safety and code autocompletion
📚 Comprehensive documentation, code examples, and tutorials
💪🏼 Actively maintained and developed by Apify—we use it ourselves!
Visit crawlee.dev or run the following command:
npx crawlee create my-crawler