One of my side projects involved a vanilla JavaScript implementation of a Reddit Comment Analyzer. The program was designed to take any Reddit user and analyze all the information they have ever posted to produce some interesting statistics. While the final product was really cool, I was pretty impressed with how the program was designed to scrape all the information so quickly. The trick to pull information that is located on many different pages was key to the design of the app. To do the dirty work and simulate multiple different fetches, it relied on an "after" parameter that was available via the network.
So what is an after parameter? An after parameter is an alfa-numeric value code that is present in the network request of the pages reddit displays. When that code is posted to the network, it tells the server to present the next batch of user comments. By turning the page into JSON and then calling the after parameter, the fetches occur much faster.
The next step is to use ES7 keywords to fetch and await the completion of the multiple fetch requests. This is implemented like so and allows the entire user history to be pulled before analytics are performed.
The result is a really fast and interest end product. Demo it for yourself here:
https://andyreadpnw.github.io/redditanalyze/
Top comments (0)