Since scrapers seem to be trending in DEV, a friend challenge me to make a web scraper using Common Lisp (yeah, I'm a Lisper, you people should (with-it :deal)). Knowing that he was under the impression that "Lisp is outdated and makes everything harder and more verbose" I decided to accept the challenge and wrote this. It took me 1 minute. My network is slow and Emacs takes forever to open in Windows 10.
;;; Just eval this in your favorite Lisp IDE.
;;; Get the libs
(ql:quickload '(:dexador :plump :lquery))
;;; Get the page
(defvar *front-page* (dex:get "https://dev.to"))
;;; Parse the page
(defvar *data* (plump:parse *front-page*))
;; Get the posts!
(lquery:$ *data* "div.single-article h3" (text))
Update: My thanks to @RainerJoswig for catching some typos in the code!!!
Top comments (1)
s/scrapping/scraping/Nice demonstration of libraries I'd never heard of -
dexador, plump & lquery!