DEV Community

Muhammad Areeb
Muhammad Areeb

Posted on

Linked In profile Scraping

Hello everyOne
I am trying to scrape a user's LinkedIn profile, but when the page opens in Chrome, it redirects to the login page. However, I want to scrape the user's information without logging in. Can someone help me with how to scrape the user's profile using node.js and puppeteer?
const browser = await puppeteer.launch({
// timeout: 30000,
args: ['--no-sandbox'],
headless: true,
// devtools: false
});
const page = await browser.newPage();
await page.goto('https://www.linkedin.com/in/ishaqbhojani', {
waitUntil: ['domcontentloaded', 'load', 'networkidle0', 'networkidle2']
})
// await page.goto()
await delay(800)
let profile = await page.evaluate(() => {
const name = document.querySelector('.pv-top-card--list > li:first-child')?.innerText;
const title = document.querySelector('.pv-top-card--list > li:nth-child(2)')?.innerText;
const location = document.querySelector('.pv-top-card--list > li:nth-child(3)')?.innerText;
const summary = document.querySelector('.pv-about-section .pv-about__summary-text')?.innerText;
const experience = Array.from(document.querySelectorAll('.pv-profile-section__card-item-v2')).map(item => ({
title: item.querySelector('.pv-entity__summary-info > h3')?.innerText,
company: item.querySelector('.pv-entity__secondary-title')?.innerText,
date: item.querySelector('.pv-entity__date-range > span:last-child')?.innerText,
description: item.querySelector('.pv-entity__description')?.innerText,
}));
return { name, title, location, summary, experience };
})
console.log(profile)
await browser.close()

Top comments (0)