🍪 Screenshot or scrape webpages behind a login with JavaScript (in Node.js)

Let's imagine, that you want to programmatically screenshot your dev.to dashboard. We learned in part 1 how to do screenshots. But so far we only did screenshots of public pages. The dev.to dashboard is only available if you're logged in. No problem! That's actually an easy task because you can set cookies with puppeteer. The only thing you have to do is open the Dev Tools and copy the remember_user_token cookie. With the correct domain settings (I prepared these in the example) you can now access the webpage as if you were logged in. Actually your puppeteer session is logged in. This is possible for most of the pages because usually, a login relies on a cookie, which you can just copy. Try it yourself!

// npm i puppeteer
const puppeteer = require('puppeteer');

const cookie = {
  name: 'remember_user_token',
  value: 'YOUR_COOKIE_VALUE', // replace this!
  domain: 'dev.to',
  url: 'https://www.dev.to/',
  path: '/',
  httpOnly: true,
  secure: true,
};

// we're using async/await - so we need an async function, that we can run
const run = async () => {
  // open the browser and prepare a page
  const browser = await puppeteer.launch();
  const page = await browser.newPage();

  // set the cookie with your user information to appear logged in
  await page.setCookie(cookie);

  await page.goto('https://dev.to/dashboard');
  await page.screenshot({
    path: 'my_dev_to_dashboard.png',
    fullPage: false, // set to true to get the whole page
  });

  // close the browser
  await browser.close();
};

// run the async function
run();

Please consider following me, if you're interested in what else you can do with puppeteer and if you don't want to miss any of my upcoming articles in this series.

Thanks for reading!