loading...
Cover image for ๐Ÿงญ Programmatically taking Screenshots while navigating &  searching on webpages using JavaScript (puppeteer)

๐Ÿงญ Programmatically taking Screenshots while navigating & searching on webpages using JavaScript (puppeteer)

benjaminmock profile image Benjamin Mock ใƒป2 min read

Super useful puppeteer snippets ๐Ÿ’ญ๐Ÿ’ญ๐Ÿ’ญ (8 Part Series)

1) ๐Ÿ“ท How to take a screenshot of a webpage with JavaScript in Node.js (using puppeteer) 2) ๐Ÿ“ท More on Screenshots with JavaScript in Node.js (emulating devices) 3 ... 6 3) ๐Ÿ“ท More on Screenshots with JavaScript in Node.js (creating PDFs) 4) ๐Ÿ—„๏ธ The easiest way to scrape a website with Javascript (in Node.js) 5) ๐Ÿช Screenshot or scrape webpages behind a login with JavaScript (in Node.js) 6) ๐Ÿงญ Programmatically taking Screenshots while navigating & searching on webpages using JavaScript (puppeteer) 7) ๐Ÿ‘Ÿ Executing JavaScript in the page context of puppeteer 8) ๐Ÿš€ Performance measuring of a webpage with JavaScript in Node.js using puppeteer

So far in this series, we only visited one page and took a screenshot or extracted its data. But you can actually navigate a page by clicking on elements, typing in fields and hitting keys.

So let's make the following plan:
We want to

  • visit the dev.to homepage
  • search for "puppeteer navigating" to find some articles about our current topic
  • and click on that very article

On our way, we'll take a screenshot of every step.

  • home.png for the homepage
  • search-results.png for the search results page
  • and article.png for this article

This is the homepage
Home

These are the search results
Alt Text

And this is our article
Alt Text

And this is how it's done:

// npm i puppeteer
const puppeteer = require('puppeteer');

// we're using async/await - so we need an async function, that we can run
const run = async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();

  // let's navigate to the dev.to homepage
  await page.goto('https://dev.to');

  // and let's take a screenshot
  await page.screenshot({
    path: 'home.png',
  });

  // and enter "Puppeteer navigating" in the search input field
  await page.type('#nav-search', 'Puppeteer navigating');

  // there's no search button, so we have to trigger the search by hitting the "Enter" key
  await page.type('#nav-search', String.fromCharCode(13));

  // let's wait until the search results are loaded
  await page.waitForSelector(
    '.stories-search .single-article .index-article-link'
  );

  // and take a screenshot of the search results
  await page.screenshot({
    path: 'search_results.png',
  });

  // select an article and click on the headline
  // if there are multiple elements satisfying the selector, the first one will be clicked
  await page.click('.index-article-link h3');

  // this time let's wait 2 seconds, which we assume is enough time for the page to load
  await page.waitFor(2000);

  // and take another screenshot
  await page.screenshot({
    path: 'article.png',
  });

  // we're done; close the browser
  await browser.close();
};

// run the async function
run();

Super useful puppeteer snippets ๐Ÿ’ญ๐Ÿ’ญ๐Ÿ’ญ (8 Part Series)

1) ๐Ÿ“ท How to take a screenshot of a webpage with JavaScript in Node.js (using puppeteer) 2) ๐Ÿ“ท More on Screenshots with JavaScript in Node.js (emulating devices) 3 ... 6 3) ๐Ÿ“ท More on Screenshots with JavaScript in Node.js (creating PDFs) 4) ๐Ÿ—„๏ธ The easiest way to scrape a website with Javascript (in Node.js) 5) ๐Ÿช Screenshot or scrape webpages behind a login with JavaScript (in Node.js) 6) ๐Ÿงญ Programmatically taking Screenshots while navigating & searching on webpages using JavaScript (puppeteer) 7) ๐Ÿ‘Ÿ Executing JavaScript in the page context of puppeteer 8) ๐Ÿš€ Performance measuring of a webpage with JavaScript in Node.js using puppeteer

Posted on Jan 11 by:

benjaminmock profile

Benjamin Mock

@benjaminmock

I'm Ben, a Frontend Developer from Germany working at ebay

Discussion

markdown guide