DEV Community

SerpApi profile image Artur Chukhrai
Artur Chukhrai for SerpApi

Posted on • Originally published at serpapi.com

Python SEO Keyword Research Tool: Google Autocomplete, People Also Ask and Related Searches

#webscraping #python #seo #demo

Intro

Here's a simple user guide to help you get started with using seo-keyword-research-tool.

SEO Keyword Research Tool is a Python package designed to generate keywords for SEO purposes. The package is available on PyPi and can be installed and used in your Python project or as CLI. It is used to generate keyword ideas using Google Autocomplete, Related Searches and People also ask results and save them to TXT, CSV, JSON file.

This tool uses SerpApi as a tool to parse data from Google search results. Other search engines coming soon.

seo-keyword-research

Demonstration

image

Installation

$ pip install seo-keyword-research-tool
Enter fullscreen mode Exit fullscreen mode

📌Note: If you want to use this tool for personal or commercial purposes, you must register on SerpApi and get your own API key (free plan available). The default API key used is for informational purposes and only works in the CLI.

Make sure you have Python installed on your computer.

Before use, you need to define your API key from SerpApi. There are several ways to do this:

  • Add an environment variable API_KEY with your key. $ export API_KEY=<API_KEY> (for Linux based terminal)
  • Add your key directly in code: SeoKeywordResearch.api_key = "your_api_key" (for in-code usage only)

CLI Usage

You can see available usage options by typing in the terminal:

$ seo -h
Enter fullscreen mode Exit fullscreen mode

Available arguments

SerpApi SEO Keyword Research Tool [-h] -q  [-e  [...]] [-dl] [-st] [-ak] [-gd] [-gl] [-hl]

Extract keywrods from: Google Autocomplete, People Also Ask, and People Also Search and saves data to CSV/JSON/TXT.

optional arguments:
  -h, --help            show this help message and exit
  -q , --query          Search query (required).
  -e  [ ...], --engines  [ ...]
                        Choices of engines to extract: Autocomplete (ac), Related Searches (rs), People Also Ask (rq). You can select multiple engines. All engines are selected by default.
  -dl , --depth-limit   Depth limit for People Also Ask. Default is 0, first 2-4 results.
  -st , --save-to       Saves the results in the current directory in the selected format (CSV, JSON, TXT). Default CSV.
  -ak , --api-key       Your SerpApi API key: https://serpapi.com/manage-api-key. Default is a test API key to test CLI.
  -gd , --domain        Google domain: https://serpapi.com/google-domains. Default google.com.
  -gl , --country       Country of the search: https://serpapi.com/google-countries. Default us.
  -hl , --lang          Language of the search: https://serpapi.com/google-languages. Default en.

Found a bug? Open issue: https://github.com/chukhraiartur/seo-keyword-research-tool/issues
Enter fullscreen mode Exit fullscreen mode

📌Note: You can use any of the available arguments in any combination. You can see the full description of each of them below.

To get results for a specific query, you need to define the --query argument (or short: -q):

$ seo -q "python how to"
Enter fullscreen mode Exit fullscreen mode

📌Note: This is a required argument.

Output

{
  "auto_complete": [
    "python how to add to a list",
    "python how to round to 2 decimals",
    "python how to install pip",
    "python how to comment out a block",
    "python how to sort a list",
    "python how to call a function",
    "python how to reverse a string",
    "python how to write to a file",
    "python how to add to a dictionary",
    "python how to convert string to int",
    "python how to remove item from list",
    "python how to split a string",
    "python how to open a file",
    "python how to reverse a list",
    "python how to concatenate strings"
  ],
  "related_searches": [
    "python for beginners",
    "python course",
    "python code examples",
    "python for beginners pdf",
    "learn python",
    "python w3schools",
    "python download",
    "python programming"
  ],
  "related_questions": [
    "How do I start programming in Python?",
    "How do I teach myself Python?",
    "Is Python easy to learn?",
    "Can I self teach myself Python?"
  ]
}
Saving data in CSV format...
Data successfully saved to python_how_to.csv file
Enter fullscreen mode Exit fullscreen mode

To get results from all supported engines, you don't need to pass an engine argument (as used in the example above). If you want to get results from one or two selected engines you need to define the --engine argument (or short: -e) divided by a space, e.g. ac rs (Autocomplete and Related Searches):

$ seo -q "lofi" -e ac
Enter fullscreen mode Exit fullscreen mode
Value Engine
ac Autocomplete
rs Related Searches (People Also Search)
rq Related Questions (People Also Ask)

Output

{
  "auto_complete": [
    "lofi girl",
    "lofi music",
    "lofi hip hop",
    "lofi boy",
    "lofi beats",
    "lofi jersey city",
    "lofi study",
    "lofi meaning",
    "lofi girl 34",
    "lofi youtube",
    "lofibra",
    "lofi study music",
    "lofi girl youtube",
    "lofi girl lore",
    "lofi girl synthwave boy"
  ]
}
Saving data in CSV format...
Data successfully saved to lofi.csv file
Enter fullscreen mode Exit fullscreen mode

You can change the default google domain google.com to any of supported domains list by defining the --domain (or short: -gd) argument:

$ seo -q "big ben" -e rs rq -gd "google.co.uk"
Enter fullscreen mode Exit fullscreen mode

Output

{
  "related_searches": [
    "big ben london",
    "big ben country",
    "big ben clock",
    "big ben facts",
    "how old is big ben",
    "why is big ben called big ben",
    "why is big ben famous",
    "big ben tickets"
  ],
  "related_questions": [
    "Is Big Ben still under construction 2023?",
    "Why is Elizabeth Tower called Big Ben?",
    "How much does it cost to go inside Big Ben?",
    "Can you go inside Big Ben?"
  ]
}
Saving data in CSV format...
Data successfully saved to big_ben.csv file
Enter fullscreen mode Exit fullscreen mode

Also, you can change the default country us (United States) of the search using the --country (or short: -gl) argument and set it to any of supported country codes list:

$ seo -q "hotel in" -e rs -gl de
Enter fullscreen mode Exit fullscreen mode

Output

{
  "related_searches": [
    "cheap hotel in germany",
    "cheap hotel in cologne, germany",
    "hotel in baesweiler restaurant",
    "hotel in baesweiler telefonnummer",
    "germany hotel booking website",
    "hotel baesweiler speisekarte",
    "germany hotel address",
    "hotel baesweiler öffnungszeiten"
  ]
}
Saving data in CSV format...
Data successfully saved to hotel_in.csv file
Enter fullscreen mode Exit fullscreen mode

If you want to change the default language en (English) of the search, you need to use the --language (or short: -hl) argument and set it to any of supported language codes list:

$ seo -q "anime" -e ac -hl "ja"
Enter fullscreen mode Exit fullscreen mode

Output

{
  "auto_complete": [
    "アニメ",
    "アニメイト",
    "アニメーション",
    "アニメ無料",
    "アニメ映画",
    "アニメ 英語",
    "アニメダンス",
    "アニメイト 池袋",
    "アニメ 2023",
    "アニメ人気",
    "アニメ9",
    "アニメ無料サイト",
    "anime expo",
    "anime girl",
    "anime store japan"
  ]
}
Saving data in CSV format...
Data successfully saved to anime.csv file
Enter fullscreen mode Exit fullscreen mode

The default file type to save search results is CSV. You can change it to JSON or TXT using the --save-to (or short: -st) argument:

$ seo -q "coffee" -st json
Enter fullscreen mode Exit fullscreen mode

Output

{
  "auto_complete": [
    "coffee near me",
    "coffee belvidere il",
    "coffee rockford",
    "coffee crush",
    "coffee nearby",
    "coffee table",
    "coffee crush menu",
    "coffee maker",
    "coffee shop",
    "coffee cake recipe",
    "coffee shops rockford il",
    "coffee grinder",
    "coffee shops rockford",
    "coffee meets bagel",
    "coffee creamer"
  ],
  "related_searches": [
    "coffee near me",
    "coffee types",
    "coffee beans",
    "coffee benefits",
    "coffee starbucks",
    "coffee origin",
    "coffee plant",
    "coffee tree"
  ],
  "related_questions": [
    "Is coffee good for health?",
    "How coffee affects the body?",
    "What are the 4 types of coffee?",
    "What are 10 benefits of coffee?"
  ]
}
Saving data in JSON format...
Data successfully saved to coffee.json file
Enter fullscreen mode Exit fullscreen mode

When you open People Also Ask result, it loads more results.

  • Default:

image

  • One of the results is open:

image

So if you open all initial results that was mean you see next-depth results.

The --depth-limit (or short: -dl) argument for People Also Ask can be set from 0 to 4. For each depth limit value, the number of results returned grows exponentially. Below is a table showing how the depth limit argument is affected:

Depth limit Number of results Explanation
0 4 Standard results
1 12 4*2 = 8 + 4 = 12
2 36 8*3 = 24 + 12 = 36
3 108 24*3 = 72 + 36 = 108
4 324 72*3 = 216 + 108 = 324

📌Note: This is how the logic works for the google.com domain, on other domains the results may differ.

The example for depth 2 means that we open all standard results, then after the new results are loaded, we open them and wait until the new results are loaded:

$ seo -q "minecraft" -e rq -st txt -dl 2
Enter fullscreen mode Exit fullscreen mode

Output

{
  "related_questions": [
    "Is Minecraft OK for a 7 year old?",
    "Is Minecraft appropriate for a 5 year old?",
    "Can I play Minecraft for free?",
    "Is Roblox better than Minecraft?",
    "Is Minecraft good for kids?",
    "Is Minecraft appropriate for 4 year olds?",
    "Is Minecraft OK for 18 year olds?",
    "Is 15 too old to play Minecraft?",
    "Why is my son obsessed with Minecraft?",
    "Is Minecraft OK for 2 year olds?",
    "What is Roblox age limit?",
    "Do 30 year olds play Minecraft?",
    "What is Roblox age limit?",
    "Is Minecraft OK for 2 year olds?",
    "What is Roblox age limit?",
    "Is 15 too old to play Minecraft?",
    "Is Roblox ok for a 40 year old?",
    "Are there inappropriate games on Roblox?",
    "Why is Fortnite not for kids?",
    "Do 30 year olds play Minecraft?",
    "Is Minecraft OK for 2 year olds?",
    "Is Minecraft OK for 18 year olds?",
    "How long is Minecraft free?",
    "How to get Minecraft for free legally?",
    "How long is 100 Minecraft day?",
    "Does Minecraft cost per month?",
    "How long does 1 days in Minecraft last?",
    "What is the free version of Minecraft called?",
    "How to download real Minecraft without money?",
    "How much does Minecraft cost?",
    "Which is bigger Minecraft or Roblox?",
    "Who is the best Minecraft player alive?",
    "How do I get Robux for free?",
    "What parents need to know about Roblox?",
    "What is better fortnite or Roblox?",
    "Who is PvP God in Minecraft?",
    "Who is the clutch god in Minecraft?",
    "Who is the Minecraft King?",
    "Is Minecraft good for ADHD?",
    "What is the appropriate age to play Minecraft?",
    "Why do autistic kids like Minecraft so much?",
    "Why is my son obsessed with Minecraft?",
    "Why is my kid addicted to Minecraft?",
    "Is Minecraft good for the brain?",
    "Is 16 too old to play Minecraft?",
    "Is Minecraft addictive game?",
    "What is Roblox age limit?",
    "What age is appropriate for Roblox?",
    "Is Roblox ok for a 40 year old?",
    "Are there inappropriate games on Roblox?",
    "Why is Fortnite not for kids?",
    "How do I make Roblox safe for my child?",
    "Should I let my child chat on Roblox?",
    "Is Roblox ok for a 40 year old?"
  ]
}
Saving data in TXT format...
Data successfully saved to minecraft.txt file
Enter fullscreen mode Exit fullscreen mode

In-code Usage

You can also use this tool in code. In fact, the arguments that you enter in the CLI are passed to the SeoKeywordResearch class.

Import the SeoKeywordResearch class to your file:

from SeoKeywordResearch import SeoKeywordResearch
Enter fullscreen mode Exit fullscreen mode

After that, you need to create an object of this class and pass all the necessary parameters:

keyword_research = SeoKeywordResearch(
    query='starbucks coffee',
    api_key='<your_serpapi_api_key>',
    lang='en',
    country='us',
    domain='google.com'
)
Enter fullscreen mode Exit fullscreen mode

To extract data from a particular engine, you need to use the appropriate methods. In the example below, I show how you can create a dictionary and add the required data by keys:

data = {
    'auto_complete': keyword_research.get_auto_complete(),
    'related_searches': keyword_research.get_related_searches(),
    'related_questions': keyword_research.get_related_questions(depth_limit=1)
}
Enter fullscreen mode Exit fullscreen mode

📌Note: The get_related_questions() method has the depth_limit parameter that allows you to load more results from People Also Ask.

You can use a convenient format to save data using the following methods:

keyword_research.save_to_csv(data)
keyword_research.save_to_json(data)
keyword_research.save_to_txt(data)
Enter fullscreen mode Exit fullscreen mode

If you need to print data in JSON format, then use the following method:

keyword_research.print_data(data)
Enter fullscreen mode Exit fullscreen mode

Output

{
  "auto_complete": [
    "starbucks coffee menu",
    "starbucks coffee cups",
    "starbucks coffee traveler",
    "starbucks coffee gear",
    "starbucks coffee drinks",
    "starbucks coffee pods",
    "starbucks coffee sizes",
    "starbucks coffee creamer",
    "starbucks coffee cake",
    "starbucks coffee traveler price",
    "starbucks coffee bags",
    "starbucks coffee beans",
    "starbucks coffee near me",
    "starbucks coffee mugs",
    "starbucks coffee press"
  ],
  "related_searches": [
    "starbucks coffee menu",
    "starbucks coffee price",
    "starbucks near me",
    "starbucks coffee beans",
    "starbucks coffee cups",
    "starbucks coffee bag size",
    "starbucks coffee on sale this week",
    "amazon starbucks coffee instant"
  ],
  "related_questions": [
    "Which one is the best coffee in Starbucks?",
    "Which coffee is Starbucks famous for?",
    "What coffee is sold most at Starbucks?",
    "What is the Starbucks drink for St Patrick's Day?",
    "Which Starbucks coffee is best for beginners?",
    "How do I choose a Starbucks coffee?",
    "Why is Starbucks coffee so popular?",
    "Who is known for the best coffee?",
    "What is the most ordered drink in the world?",
    "What is the most expensive drink at Starbucks?",
    "What is the name of the green coffee at Starbucks?",
    "Does Starbucks have a healthy green drink?"
  ]
}
Enter fullscreen mode Exit fullscreen mode

Feel free to open bug issue, something isn't working, what feature to add, or anything else related to Google autocomplete, related searches or people also ask.

If you want other functionality added to this demo project or if you want to see some other projects made with SerpApi, write me a message.

Join us on Twitter | YouTube

Add a Feature Request💫 or a Bug🐞

Top comments (0)

Read next

saint_vandora profile image

How to Use JSONB in PostgreSQL with Step-by-Step Instructions

FOLASAYO SAMUEL OLAYEMI -

bascodes profile image

This Week In Python

Bas Steins -

edvichuki profile image

Comprehensive Weather Data Analysis Using Python: Temperature, Rainfall Trends, and Visualizations

Gichuki Edwin -

serhii_kolodych profile image

🛠️ Website Development Tools I use every day 💻

Serhii Kolodych -