Since scrapers seem to be trending in DEV, a friend challenge me to make a web scraper using Common Lisp (yeah, I'm a Lisper, you people should
(with-it :deal)). Knowing that he was under the impression that "Lisp is outdated and makes everything harder and more verbose" I decided to accept the challenge and wrote this. It took me 1 minute. My network is slow and Emacs takes forever to open in Windows 10.
;;; Just eval this in your favorite Lisp IDE. ;;; Get the libs (ql:quickload '(:dexador :plump :lquery)) ;;; Get the page (defvar *front-page* (dex:get "https://dev.to")) ;;; Parse the page (defvar *data* (plump:parse *front-page*)) ;; Get the posts! (lquery:$ *data* "div.single-article h3" (text))
Update: My thanks to @RainerJoswig for catching some typos in the code!!!