DEV Community

Željko Šević
Željko Šević

Posted on • Originally published at sevic.dev on

Browser automation with Puppeteer

#puppeteer #node

Puppeteer is a headless browser for automating browser tasks. Here's the list of some of the features:

  • Turn off headless mode 
  const browser = await puppeteer.launch({
    headless: false
    // ...
  });
Enter fullscreen mode Exit fullscreen mode
  • Resize the viewport to the window size 
  const browser = await puppeteer.launch({
    // ...
    defaultViewport: null
  });
Enter fullscreen mode Exit fullscreen mode
  • Emulate screen how it's shown to the user via the emulateMediaType method 
  await page.emulateMediaType('screen');
Enter fullscreen mode Exit fullscreen mode
  • Save the page as a PDF file with a specified path, format, scale factor, and page range 
  await page.pdf({
    path: 'path.pdf',
    format: 'A3',
    scale: 1,
    pageRanges: '1-2',
    printBackground: true
  });
Enter fullscreen mode Exit fullscreen mode
  • Use preexisting user's credentials to skip logging in to some websites. The user data directory is a parent of the Profile Path value from the chrome://version page. 
  const browser = await puppeteer.launch({
    // ...
    args: [
      '--user-data-dir=C:\\Users\\<USERNAME>\\AppData\\Local\\Google\\Chrome\\User Data'
    ]
  });
Enter fullscreen mode Exit fullscreen mode
  • Use Chrome instance instead of Chromium by setting up the executable path from the chrome://version URL. Close Chrome browser before running the script 
  const browser = await puppeteer.launch({
    executablePath: 'C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe'
    // ...
  });
Enter fullscreen mode Exit fullscreen mode
  • Wait for a specific selector to be loaded. You can also provide a timeout in milliseconds 
  await page.waitForSelector('.success', { timeout: 5000 });
Enter fullscreen mode Exit fullscreen mode
  • Manipulate with a specific element and click on some of the elements 
  await page.$eval('#header', async (headerElement) => {
    // ...
    headerElement
      .querySelectorAll('svg')
      .item(13)
      .parentNode.click();
  });
Enter fullscreen mode Exit fullscreen mode
  • Extend execution of the $eval method 
  const browser = await puppeteer.launch({
    // ...
    protocolTimeout: 0,
  });
Enter fullscreen mode Exit fullscreen mode
  • Manipulate with multiple elements 
  await page.$$eval('.some-class', async (elements) => {
    // ...
  });
Enter fullscreen mode Exit fullscreen mode
  • Wait for navigation (e.g., form submitting) to be done 
  await page.waitForNavigation({ waitUntil: 'networkidle0', timeout: 0 });
Enter fullscreen mode Exit fullscreen mode
  • Trigger hover event on some of the elements 
  await page.$eval('#header', async (headerElement) => {
    const hoverEvent = new MouseEvent('mouseover', {
      view: window,
      bubbles: true,
      cancelable: true
    });

    headerElement.dispatchEvent(hoverEvent);
  });
Enter fullscreen mode Exit fullscreen mode
  • Expose a function in the browser and use it in $eval and $$eval callbacks (e.g., simulate typing using the window.type function) 
  await page.exposeFunction('type', async (selector, text, options) => {
    await page.type(selector, text, options);
  });

  await page.$$eval('.some-class', async (elements) => {
    // ...
    window.type(selector, text, { delay: 0 });
  });
Enter fullscreen mode Exit fullscreen mode
  • Expose a variable in the browser by passing it as the third argument for $eval and $$eval methods and use it in $eval and $$eval callbacks 
  await page.$eval(
    '#element',
    async (element, customVariable) => {
      // ...
    },
    customVariable
  );
Enter fullscreen mode Exit fullscreen mode
  • Intercept page redirections (via interceptor) and open them in new tabs rather than following them in the same tab 
  await page.setRequestInterception(true);
  page.on('request', async function (request) {
    const url = request.url();
    if (url !== REDIRECTION_URL) {
      return request.continue();
    }

    await request.respond({
      contentType: 'text/html',
      status: 304,
      body: '<body></body>',
    });
    const newPage = await browser.newPage();
    await newPage.goto(url, { waitUntil: 'domcontentloaded', timeout: 0 });
    // custom logic
    await newPage.close();
  });
Enter fullscreen mode Exit fullscreen mode

Top comments (0)

Read next

thejaredwilcurt profile image

Bun hype. How we learned nothing from Yarn

The Jared Wilcurt -

electromorphous profile image

NPM Error Code 4058 (ENOENT)

Aseer -

deepak22448 profile image

"TypeScript and process.env: A Match Made in Node.js Heaven"

Deepak Sharma -

drsimplegraffiti profile image

From Code to the Cloud: A Step-by-Step Guide to Deploying Your Node.js App on AWS EC2

Abayomi Ogunnusi -