DEV Community

loading...

Discussion on: Scraping websites with Xray

Collapse
functional_js profile image
Functional Javascript

Nice one Dennis.
I tested it out and it works.

I converted it from an explicit Promise idiom to an async-await idiom....

const Xray = require('x-ray');

//util
const lpromise = p => p.then(o => console.log(o.items));

/*
@func
retrieve posts using xray

@typedef {{items: string[]}} itemsObj
@return {Promise<itemsObj>}
*/
const getPosts = async () => {
  const x = Xray();
  const url = "https://blog.dennisokeeffe.com";
  try {
    return await x(url, "main:last-child", {
      items: x("div", [
        {
          title: "h3 > a",
          description: "p",
          link: "h3 > a@href",
          date: "small",
        },
      ]),
    });
  } catch (err) {
    console.error(err);
  }
};

//@tests
lpromise(getPosts());

Forem Open with the Forem app