I scraped Javascript rendered data from my college's website. The website just has the data and it is not even responsive, still everyone is forced to use it on desktop view on mobile devices. I scraped the attendance, LMS and everything using Casper.js and Phantom.js and created RESTful APIs and developed a mobile app using React Native for Android and iOS. 🙄👍 By the way in my liesure, I find new ways to scrape contents. 🖖
I build developer tools and services at Microsoft (currently Codespaces, Live Share, IntelliCode) and maintain some OSS projects (CodeTour, GistPad, CodeSwing, WikiLens)
Very cool! Have you had trouble maintaining the scraping scripts (e.g. HTML layout changes)? Or does the school website change infrequently enough that scraping it is pretty reliable?
I scraped Javascript rendered data from my college's website. The website just has the data and it is not even responsive, still everyone is forced to use it on desktop view on mobile devices. I scraped the attendance, LMS and everything using Casper.js and Phantom.js and created RESTful APIs and developed a mobile app using React Native for Android and iOS. 🙄👍 By the way in my liesure, I find new ways to scrape contents. 🖖
Very cool! Have you had trouble maintaining the scraping scripts (e.g. HTML layout changes)? Or does the school website change infrequently enough that scraping it is pretty reliable?
They don't bring any updates. So they don't change any HTML content. Hence till now I didn't face anything of that sort...