Introduction
In this tutorial, we will learn to scrape LinkedIn profiles from Google Search Results.
Requirements:
Web Parsing with CSS selectors
Searching the tags from the HTML files is not only a difficult thing to do but also a time-consuming process. It is better to use the CSS Selectors Gadget for selecting the perfect tags to make your web scraping journey easier.
This gadget can help you to come up with the perfect CSS selector for your need. Here is the link to the tutorial, which will teach you to use this gadget for selecting the best CSS selectors according to your needs.
User Agents
User-Agent is used to identify the application, operating system, vendor, and version of the requesting user agent, which can save help in making a fake visit to Google by acting as a real user.
You can also rotate User Agents, read more about this in this article: How to fake and rotate User Agents using Python 3.
If you want to further safeguard your IP from being blocked by Google, you can try these 10 Tips to avoid getting Blocked while Scraping Google.
Install Libraries
Before we begin, install these libraries so we can move forward and prepare our scraper.
Or you can type the below commands in your project terminal to install the libraries:
npm i unirest
npm i cheerio
Target:
We will target employees working at Amazon in the USA.
Process:
In this section, we will make our scraper to scrape the LinkedIn profiles of the employees working at Amazon.
Now, let us start making our scraper. First, we will make a GET request on the target URL to extract the HTML data.
const url = "https://www.google.com/search?q=site:linkedin.com/in+'amazon.com'&gl=us";
const response = await unirest
.get(url)
.header({
"User-Agent":
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.77 Safari/537.36",
})
const $ = cheerio.load(response.body);
Step-by-step explanation:
- In the fourth line, we made a GET request to our target URL.
- In the next line, we passed the headers required with our target URL.
- Then we stored the returned response in the Cheerio instance.
You can also make an array of user agents to rotate them on every request so that Google doesn’t block your request.
const selectRandom = () => {
const userAgents = ["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36",
"Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36",
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.157 Safari/537.36",
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36",
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.45 Safari/537.36",
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.71 Safari/537.36",
]
var randomNumber = Math.floor(Math.random() * userAgents.length);
return userAgents[randomNumber];
}
let user_agent = selectRandom();
let header = {
"User-Agent": `${user_agent}`
}
Now, we will prepare our parser and will extract the important data we need in our response.
As you can see the tag for the title is .DKV0Md. It contains the employee name and his post in the company.
let title = [];
$(".g").each((i,el) => {
title[i] = $(el).find(".DKV0Md").text();
if(title[i].includes(" - LinkedIn") || title[i].includes(" | LinkedIn"))
{
title[i] = title[i].replace(" - LinkedIn" , "")
title[i] = title[i].replace(" | LinkedIn", "")
}
})
Now, we will find the location from which the employee is working.
From the above image, we found the tag for the location as .WZ8Tjf, which makes our code looks like this:
let title = [], location = [];
$(".g").each((i,el) => {
title[i] = $(el).find(".DKV0Md").text();
if(title[i].includes(" - LinkedIn") || title[i].includes(" | LinkedIn"))
{
title[i] = title[i].replace(" - LinkedIn" , "")
title[i] = title[i].replace(" | LinkedIn", "")
}
location[i] = $(el).find(".WZ8Tjf").text();
})
Now, we will parse the description.
From the above image, we find the tag for the description as .lEBKkf. Similarly, we can also find the tag for the link.
$(".g").each((i,el) => {
title[i] = $(el).find(".DKV0Md").text();
if(title[i].includes(" - LinkedIn") || title[i].includes(" | LinkedIn"))
{
title[i] = title[i].replace(" - LinkedIn" , "")
title[i] = title[i].replace(" | LinkedIn", "")
}
location[i] = $(el).find(".WZ8Tjf").text();
about[i] = $(el).find(".lEBKkf").text();
link[i] = $(el).find("a").attr("href")
})
Now, we have completed the work to prepare our parser. We will now push this data to the employees_data array and will print it in the terminal.
let employees_data = [];
for (let i = 0; i < title.length; i++) {
employees_data.push({
title: title[i],
location: location[i],
about: about[i],
link: link[i]
})
}
console.log(employees_data)
This is how our results looks like:
[
{
title: 'Werner Vogels - VP & CTO - Amazon.com',
location: 'Seattle, Washington, United States · VP & CTO · Amazon.com',
about: 'I believe in democratising business creation, simplifying operation and driving innovation by providing a low cost, scalable and reliable infrastructure that ...',
link: 'https://www.linkedin.com/in/wernervogels'
},
{
title: 'Jolene Bacca - Executive Assistant - Amazon.com',
location: 'Seattle, Washington, United States · Executive Assistant · Amazon.com',
about: 'I am very capable at efficiently solving difficult and complex problems affecting people within my various departments and other senior level personnel. The key ...',
link: 'https://www.linkedin.com/in/jolene-bacca-9497147'
},
{
title: 'J.R. Harris - Operations Manager - Amazon.com',
location: 'McCordsville, Indiana, United States · Operations Manager · Amazon.com',
about: 'Manufacturing management leading hourly and salary associates within a moderate volume and highly customized environment. Experienced with ISO 9001, ...',
link: 'https://www.linkedin.com/in/jrharris1'
},
……
Here is the full code:
const cheerio = require("cheerio");
const unirest = require("unirest");
const getData = async() => {
try
{
const url = "https://www.google.com/search?q=site:linkedin.com/in+'amazon.com'&gl=us";
const selectRandom = () => {
const userAgents = ["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36",
"Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36",
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.157 Safari/537.36",
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36",
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.45 Safari/537.36",
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.71 Safari/537.36",
]
var randomNumber = Math.floor(Math.random() * userAgents.length);
return userAgents[randomNumber];
}
let user_agent = selectRandom();
let header = {
"User-Agent": `${user_agent}`
}
const response = await unirest
.get(url)
.header(header)
const $ = cheerio.load(response.body)
let title = [],location = [], about = [],link=[];
$(".g").each((i,el) => {
title[i] = $(el).find(".DKV0Md").text();
if(title[i].includes(" - LinkedIn") || title[i].includes(" | LinkedIn"))
{
title[i] = title[i].replace(" - LinkedIn" , "")
title[i] = title[i].replace(" | LinkedIn", "")
}
location[i] = $(el).find(".WZ8Tjf").text();
about[i] = $(el).find(".lEBKkf").text();
link[i] = $(el).find("a").attr("href")
})
let employees_data = [];
for (let i = 0; i < title.length; i++) {
employees_data.push({
title: title[i],
location: location[i],
about: about[i],
link: link[i]
})
}
console.log(employees_data)
}
catch(e)
{
console.log(e)
}
}
getData();
This data can be useful when you are generating leads for your company, you can send messages to your target company's founders and CEO and tell them about your product. Another usage is that when you are opening your own lead generation company like snov.io or findthatlead.com you can scrape these results to gather all the employee lists and update them from time to time.
With Google Search API
If you don't want to code and maintain the scraper in the long run then you can definitely try our Google Search API. Serpdog | Google Search API also supports all advanced featured snippets like knowledge graphs, answer box results, etc.
Scraping Google also requires solving captchas, a large pool of User agents, and proxies, but Serpdog solves all these problems on its behalf for a smooth scraping experience.
Our users also get 100 free requests on the first sign-up.
const axios = require('axios');
axios.get('https://api.serpdog.io/search?api_key=APIKEY&q=coffee&gl=us')
.then(response => {
console.log(response.data);
})
.catch(error => {
console.log(error);
});
Results:
"knowledge_graph": {
"title": "Coffee",
"type": "Drink",
"header_images": [
{
"source": "https://en.wikipedia.org/wiki/Coffee",
"image": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"source": "https://www.tastingtable.com/718678/coffee-brands-ranked-from-worst-to-best/",
"image": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"source": "https://www.healthline.com/nutrition/top-evidence-based-health-benefits-of-coffee",
"image": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"source": "https://www.bbcgoodfood.com/recipes/collection/coffee-recipes",
"image": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
}
],
"description": "Coffee is a brewed drink prepared from roasted coffee beans, the seeds of berries from certain flowering plants in the Coffea genus. From the coffee fruit, the seeds are separated to produce a stable, raw product: unroasted green coffee.",
"source": {
"name": "Wikipedia",
"link": "https://en.wikipedia.org/wiki/Coffee"
},
"Acidity level": "4.85 to 5.10",
"list": {
"Total Fat": [
"0 g",
"0%"
],
"Saturated fat": [
"0 g",
"0%"
],
"Trans fat regulation": [
"0 g",
""
],
"Cholesterol": [
"0 mg",
"0%"
],
"Sodium": [
"5 mg",
"0%"
],
"Potassium": [
"116 mg",
"3%"
],
"Total Carbohydrate": [
"0 g",
"0%"
],
"Dietary fiber": [
"0 g",
"0%"
],
"Sugar": [
"0 g",
""
],
"Protein": [
"0.3 g",
"0%"
],
"Caffeine": [
"95 mg",
""
],
"Vitamin C": [
"",
"0%"
],
"Calcium": [
"",
"0%"
],
"Iron": [
"",
"0%"
],
"Vitamin D": [
"",
"0%"
],
"Vitamin B6": [
"",
"0%"
],
"Cobalamin": [
"",
"0%"
],
"Magnesium": [
"",
"1%"
]
},
"Compounds In Coffee": [
{
"title": "Chlorogenic acid",
"link": "https://google.com/search?gl=us&hl=en&q=Chlorogenic+acid&stick=H4sIAAAAAAAAAONgFuLUz9U3MCorTMtVAjPNizNy4rVEspOt9JPzc3Pz86xS8svzyhOLUopXMQo6Z6TmZiYn5jjn5xbkl-alFC9iFXDOyMkvyk9PzctMVkhMzkzZwcoIAJ5I0VJYAAAA&sa=X&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQxA16BAhlEAU",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"title": "Trigonelline",
"link": "https://google.com/search?gl=us&hl=en&q=Trigonelline&stick=H4sIAAAAAAAAAONgFuLUz9U3MCorTMtVAjNNqwwq47VEspOt9JPzc3Pz86xS8svzyhOLUopXMQo6Z6TmZiYn5jjn5xbkl-alFC9i5QkpykzPz0vNycnMS93ByggAa_A38FQAAAA&sa=X&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQxA16BAhlEAc",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"title": "Quinic acid",
"link": "https://google.com/search?gl=us&hl=en&q=Quinic+acid&stick=H4sIAAAAAAAAAONgFuLUz9U3MCorTMtVAjPNi5NNLbREspOt9JPzc3Pz86xS8svzyhOLUopXMQo6Z6TmZiYn5jjn5xbkl-alFC9i5Q4szczLTFZITM5M2cHKCAAvEyRDUwAAAA&sa=X&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQxA16BAhlEAk",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"title": "Kahweol",
"link": "https://google.com/search?gl=us&hl=en&q=Kahweol&stick=H4sIAAAAAAAAAONgFuLUz9U3MCorTMtV4gIxTdLMDLJNtESyk630k_Nzc_PzrFLyy_PKE4tSilcxCjpnpOZmJifmOOfnFuSX5qUUL2Jl907MKE_Nz9nByggA3TsOH1AAAAA&sa=X&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQxA16BAhlEAs",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
}
],
"Starbucks Coffee": [
{
"title": "Starbucks Roast Dark Ro...",
"link": "https://google.com/search?gl=us&hl=en&q=Starbucks+Roast+Dark+Roast+Ground+Coffee&stick=H4sIAAAAAAAAAC3IMQrCMBQAUEQKOriIByhO4vKtLtJVwV0PUNLkJ8Yk_8ekteBxHD2Bx9PB7fEm4_kUAmy2j7sOyxkYqKqblrun27frhZM1SA6BqVY80CCSyu9R9TvvUXaWCRzx4FEZbKIg9BnylWO0ZBoXG_TW2Nbjq1hdOpHaXrpcnlnkrjyK5P48Je5JlQfWGvFTjL5shC6JlAAAAA&sa=X&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQxA16BAhmEAU",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"title": "Starbucks Frappuccino, Caramel...",
"link": "https://google.com/search?gl=us&hl=en&q=Starbucks+Frappuccino,+Caramel,+13.7+Oz&stick=H4sIAAAAAAAAAA3IMQrCMBQAUIoUdHARDxBcBCmmqYLQVXB18AAlTX9jyP9JTFoLHsfRE3g8feObz1YLTrysno-eNkuuuRC6OpZEttqtraq58kTe1Z2f3CRjlz6Z-B8iqMF4x63zE0KnoQnSASae7j4E43RjQwNotGkR3vn2NsjYjsomdokyhFEp43zBzjJKAiyYOOxP7Pr65tkPqGF6KJMAAAA&sa=X&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQxA16BAhmEAc",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"title": "Starbucks Coffee Drink Sw...",
"link": "https://google.com/search?gl=us&hl=en&q=Starbucks+Coffee+Drink+Sweet+Cream&stick=H4sIAAAAAAAAAA3IMQ7CIBQA0DSmiQ4uxgOQji7fmtSYrvUGPQCh9BcJ8KGAch9HT-Dx9I1vuznswMH58loX1-xBQdtSl1d55bfT0cgepHfOUz_7QkXEOX2q9n_WoszaExjyxeKskAdBaBOkhw9Bk-ImcLRa6cniu27GLOL0lCaxwS8LIrtHTYaNBTGzIaJw37r6AeTgLQeOAAAA&sa=X&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQxA16BAhmEAk",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"title": "Starbucks Roast Coffee...",
"link": "https://google.com/search?gl=us&hl=en&q=Starbucks+Roast+Coffee+Whole+Bean&stick=H4sIAAAAAAAAAA3IMQ6CMBQA0BBDooOL8QDV0eUDgwOj3kAHR1LKpzRt_y98lPs4egKPp29869VuAxGK6jX28bgFC2UpRZLxnOJp700NhmNkqjteaNFTJ5-s_F8IaGbHBJ54CdhZbJImDAIycEqObONTg8FZ1wZ854f7rKf2abyoG2uZ1ZX7HlE9Bg6oLqjpm2c_as3ybY0AAAA&sa=X&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQxA16BAhmEAs",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
}
],
"People also search for": [
{
"title": "Tea",
"link": "https://google.com/search?gl=us&hl=en&q=Tea&si=AC1wQDCwN61-ebmuwbQCO5QCrgOvEq5bkWeIzJ5JczItzAKNdTku-1rX0tcMn1LKdXOIpKJwlLmVSfa90_0znjAMcL2sAzX4J_UwMexVra0SvN9zBEkwGwA5u6HSaoI47SyRCD9nb-8Glzx2c5rfIJe-aizf7IAxGOhJh_kG0qL9FIdiWvdohKRx4gRfu0e1hMv_cus-OR01faxKayI3akdtDTPtZF4-BCaT_0qctWJy1y5xtkAOr0_dsLhobE2rbdSGteM1GLxNZDARFa9umM8-6UkPFTmf4Q%3D%3D&sa=X&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQxA16BAhnEAU",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"title": "Espresso",
"link": "https://google.com/search?gl=us&hl=en&q=Espresso&si=AC1wQDCwN61-ebmuwbQCO5QCrgOvEq5bkWeIzJ5JczItzAKNdeHFN2RiKqNUOGcF0pmVdz3rIrXiuMEhNdO-SfahftpqQIYK3-hB1rA_kOlkAeWqSr6HTBB8G84PquMrIFHofy3tS0AENRg2iT4kQhARW-cqiwd5mwe4HWoM7NfCdXPHKL-9vcPCuQWsMQH0QsiUPQef1wfYHgpF76EoCEd9WgK2RCOVSWy2_UvrDO9ycBWVfnNdmkvoeclcDeyjTxdu3vRmri1-sGC3Vxg1Xf8C1P-tSU7Ac6Db7sVDHZWBhhei19_FP0o%3D&sa=X&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQxA16BAhnEAc",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"title": "Drink",
"link": "https://google.com/search?gl=us&hl=en&q=Beverage&si=AC1wQDC0KliLIoeyh35S2UVSh9d0at62ormA4JRB9xLdTgvleVEPuIJW3qN0rEFDMoRO5TaIi-eUtrO7i-yBcPsfUzSls6PoTU-buveIcXrKc8j6j8RTk6levipGBVfMDb2AmQYFIlFivgjEWtrw2TTOjzXWO7h0VnyOV-MXRLUeB3vIgKfdyVVgegWm85plPQ3qoUo00QwqxhwByVeRhWtTwWxm0nV7Mhs4OnmQz88vCcewdr1PGbonYzW4YfEznNahmd_nLsKERiRzf7PW6RoKR9MqwLl7tKt3KMgvOj5nUbDOmdXibLM%3D&sa=X&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQxA16BAhnEAk",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"title": "Iced coffee",
"link": "https://google.com/search?gl=us&hl=en&q=Iced+coffee&si=AC1wQDDagiMg03ncxeOQZbwVe-CJxRCchC-jr2hCPTxjc9wbgDb1tUqijDJX85Pg7-Fd4h4ECiqrTLXgOQJbaZmoXfWb8MZKyZ7uj6FykoR0HPdYz7APuA16Lh3IDaqobb6GC1g6vA53Q0ZorgA-FFa18joGMuV0h3kKB2XsyLL0UeBqUl5Ia_r_Auk9F-wZzVIhpHKkmIq6gXo1DP_vcZa8jyTzXy4VXG7vcXdoDmzCjZTsJtClzIOpna_wgbYFRtYbRWqfGISfDDvhjFbeGQDR5bN1gsLT_ra5gs0FdkoknXCL4O-AXnM%3D&sa=X&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQxA16BAhnEAs",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
}
],
"see_results_about": [
{
"title": "Coffee",
"snippet": "Plant",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
}
]
},
"inline_videos": [
{
"title": "Miguel - Coffee (Official Video)",
"source": "YouTube",
"date": "Jun 16, 2015",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"title": "Kelly Rowland - COFFEE (Official Music Video)",
"source": "YouTube",
"date": "Apr 17, 2020",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"title": "Beabadoobee - Coffee",
"source": "YouTube",
"date": "Mar 26, 2020",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
}
],
"local_results": [
{
"position": 1,
"title": "Scooter's Coffee",
"place_id": "5225762576898740739",
"rating": "4.4",
"address": "Coffeyville, KS",
"hours": "Closed ⋅ Opens 7AM",
},
{
"position": 2,
"title": "Terebinth Coffee House & Roastery",
"place_id": "18371722425790879519",
"rating": "4.9",
"address": "Coffeyville, KS",
"hours": "Closed ⋅ Opens 6:30AM Mon",
},
{
"position": 3,
"title": "Ane Mae's Coffee and Sandwich Shop",
"place_id": "1687994686344788766",
"rating": "4.7",
"address": "Independence, KS",
"hours": "Closed ⋅ Opens 7AM Mon",
},
{
"more_locations_link": "https://www.google.com/search?gl=us&hl=en&q=coffee&npsic=0&rflfq=1&rldoc=1&rllag=37129806,-95662854,11423&tbm=lcl&sa=X&ved=2ahUKEwiGzIer7qL5AhXCKEQIHdh0D5YQtgN6BAgmEAE"
}
],
"recipes_results": [
{
"title": "Coffee recipes",
"link": "https://www.bbcgoodfood.com/recipes/collection/coffee-recipes",
"source": "BBC Good Food",
"rating": "",
"reviews": "",
"total_time": "",
"ingredients": [
"Instant coffee"
],
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"title": "20 Great Coffee Drinks From Around the World",
"link": "https://insanelygoodrecipes.com/coffee-recipes/",
"source": "Insanely Good Recipes",
"rating": "4.5",
"reviews": "8",
"total_time": "",
"ingredients": [
"Turkish coffee",
" vietnamese coffee",
" white chocolate mocha",
" cold brew coffee",
" frappuccino"
],
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"title": "Bulletproof Coffee Recipe",
"link": "https://www.bulletproof.com/recipes/bulletproof-diet-recipes/bulletproof-coffee-recipe/",
"source": "Bulletproof",
"rating": "",
"reviews": "",
"total_time": "5 min",
"ingredients": [
"Mct oil",
" bulletproof coffee",
" grass fed"
],
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
}
],
"peopleAlsoAskedFor": [
"What are benefits of coffee?",
"Which is the best coffee to drink?",
"Is it healthy to eat coffee?",
"Is coffee made from poop?"
],
"organic_results": [
{
"title": "9 Health Benefits of Coffee, Based on Science - Healthline",
"link": "https://www.healthline.com/nutrition/top-evidence-based-health-benefits-of-coffee",
"displayed_link": "https://www.healthline.com › Wellness Topics › Nutrition",
"snippet": "Coffee is a popular beverage that researchers have studied extensively for its many health benefits, including its ability to increase energy levels, promote ...",
"rank": 1
},
{
"title": "The Coffee Bean & Tea Leaf | CBTL",
"link": "https://www.coffeebean.com/",
"displayed_link": "https://www.coffeebean.com",
"snippet": "Born and brewed in Southern California since 1963, The Coffee Bean & Tea Leaf® is passionate about connecting loyal customers with carefully handcrafted ...",
"rank": 2
},
{
"title": "Peet's Coffee: The Original Craft Coffee",
"link": "https://www.peets.com/",
"displayed_link": "https://www.peets.com",
"snippet": "Since 1966, Peet's Coffee has offered superior coffees and teas by sourcing the best quality coffee beans and tea leaves in the world and adhering to strict ...",
"rank": 3
},
{
"title": "The History of Coffee - National Coffee Association",
"link": "https://www.ncausa.org/about-coffee/history-of-coffee",
"displayed_link": "https://www.ncausa.org › ... › History of Coffee",
"snippet": "Coffee grown worldwide can trace its heritage back centuries to the ancient coffee forests on the Ethiopian plateau. There, legend says the goat herder ...",
"inline_sitelinks": [
{
"title": "An Ethiopian Legend",
"link": "https://www.ncausa.org/about-coffee/history-of-coffee#:~:text=An%20Ethiopian%20Legend"
},
{
"title": "The Arabian Peninsula",
"link": "https://www.ncausa.org/about-coffee/history-of-coffee#:~:text=The%20Arabian%20Peninsula,-Coffee%20cultivation%20and%20trade%20began"
},
{
"title": "Coffee Comes To Europe",
"link": "https://www.ncausa.org/about-coffee/history-of-coffee#:~:text=Coffee%20Comes%20to%20Europe"
}
],
"rank": 4
},
{
"title": "coffee | Origin, Types, Uses, History, & Facts | Britannica",
"link": "https://www.britannica.com/topic/coffee",
"displayed_link": "https://www.britannica.com › ... › Food",
"snippet": "coffee, beverage brewed from the roasted and ground seeds of the tropical evergreen coffee plants of African origin. Coffee is one of the three most popular ...",
"rank": 5
},
{
"title": "Starbucks Coffee Company",
"link": "https://www.starbucks.com/",
"displayed_link": "https://www.starbucks.com",
"snippet": "More than just great coffee. Explore the menu, sign up for Starbucks® Rewards, manage your gift card and more.",
"rank": 6
},
{
"title": "#coffee hashtag on Instagram • Photos and videos",
"link": "https://www.instagram.com/explore/tags/coffee/",
"displayed_link": "https://www.instagram.com › explore › tags › coffee",
"snippet": "156M Posts - See Instagram photos and videos from 'coffee' hashtag.",
"rank": 7
}
],
"relatedSearches": [
{
"query": "coffee near me",
"link": "www.google.com/search?gl=us&q=coffee near me"
},
{
"query": "coffee shop",
"link": "www.google.com/search?gl=us&q=coffee shop"
},
{
"query": "coffee benefits",
"link": "www.google.com/search?gl=us&q=coffee benefits"
},
{
"query": "best coffee",
"link": "www.google.com/search?gl=us&q=best coffee"
},
{
"query": "coffee - wikipedia",
"link": "www.google.com/search?gl=us&q=coffee - wikipedia"
},
{
"query": "coffee origin",
"link": "www.google.com/search?gl=us&q=coffee origin"
}
],
"pagination": {
"current": "1",
"next": "https://www.google.com/search?q=coffee&gl=us&hl=en&ei=R0TmYuSlH6KdkPIPvs6lqA8&start=10&sa=N&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQ8NMDegQIAxBK",
"page_no": {
"2": "https://www.google.com/search?q=coffee&gl=us&hl=en&ei=R0TmYuSlH6KdkPIPvs6lqA8&start=10&sa=N&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQ8tMDegQIAxA4",
"3": "https://www.google.com/search?q=coffee&gl=us&hl=en&ei=R0TmYuSlH6KdkPIPvs6lqA8&start=20&sa=N&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQ8tMDegQIAxA6",
"4": "https://www.google.com/search?q=coffee&gl=us&hl=en&ei=R0TmYuSlH6KdkPIPvs6lqA8&start=30&sa=N&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQ8tMDegQIAxA8",
"5": "https://www.google.com/search?q=coffee&gl=us&hl=en&ei=R0TmYuSlH6KdkPIPvs6lqA8&start=40&sa=N&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQ8tMDegQIAxA-",
"6": "https://www.google.com/search?q=coffee&gl=us&hl=en&ei=R0TmYuSlH6KdkPIPvs6lqA8&start=50&sa=N&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQ8tMDegQIAxBA",
"7": "https://www.google.com/search?q=coffee&gl=us&hl=en&ei=R0TmYuSlH6KdkPIPvs6lqA8&start=60&sa=N&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQ8tMDegQIAxBC",
"8": "https://www.google.com/search?q=coffee&gl=us&hl=en&ei=R0TmYuSlH6KdkPIPvs6lqA8&start=70&sa=N&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQ8tMDegQIAxBE",
"9": "https://www.google.com/search?q=coffee&gl=us&hl=en&ei=R0TmYuSlH6KdkPIPvs6lqA8&start=80&sa=N&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQ8tMDegQIAxBG",
"10": "https://www.google.com/search?q=coffee&gl=us&hl=en&ei=R0TmYuSlH6KdkPIPvs6lqA8&start=90&sa=N&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQ8tMDegQIAxBI"
}
},
"serpdog_pagination": {
"current": "1",
"page_no": {
"2": "https://api.serpdog.io/search?api_key=APIKEY&q=coffee&gl=us&start=10",
"3": "https://api.serpdog.io/search?api_key=APIKEY&q=coffee&gl=us&start=20",
"4": "https://api.serpdog.io/search?api_key=APIKEY&q=coffee&gl=us&start=30",
"5": "https://api.serpdog.io/search?api_key=APIKEY&q=coffee&gl=us&start=40",
"6": "https://api.serpdog.io/search?api_key=APIKEY&q=coffee&gl=us&start=50",
"7": "https://api.serpdog.io/search?api_key=APIKEY&q=coffee&gl=us&start=60",
"8": "https://api.serpdog.io/search?api_key=APIKEY&q=coffee&gl=us&start=70",
"9": "https://api.serpdog.io/search?api_key=APIKEY&q=coffee&gl=us&start=80",
"10": "https://api.serpdog.io/search?api_key=APIKEY&q=coffee&gl=us&start=90"
}
}
Conclusion:
In this tutorial, we learned to scrape LinkedIn profiles with the help of Google using Node JS. Feel free to message me if I missed something. Follow me on Twitter. Thanks for reading!
Additional Resources
- Web Scraping Google With Node JS - A Complete Guide
- Web Scraping Google Without Getting Blocked
- Scrape Google Organic Search Results
- Scrape Google Shopping Results
Author:
My name is Darshan and I am the founder of serpdog.io.
Top comments (0)