Develop a Keyword Research Tool in Minutes (Python and C#)

#dotnet #python #csharp #beginners

I reverse engineered 2 Big Keyword Research Tools, and I found out how they work and how you can build the same on your own.

The Sites Are:

Answerthepublic: a website that allows users to type in a keyword or phrase and receive a list of related questions that people have searched for on the internet.
Keywordtool.io: a keyword research and analysis tool that helps users find and analyze the best keywords for their website or online business.

Even if you are not so experienced in programming, you can still easily follow up with this tutorial and Develop a Keyword Research Tool with Python or C#.

How do Keyword Research Tools Work?

Well, it depends on the scope and functionality of the Tools you’re making, and the SEO metrics you want your Tool to Provide.

In our case, we will focus on “google search suggestions” and “Question Suggestions.” I will also tell you later how you can get more metrics.

Let’s see how these tools are getting results, Open Google Search, and Type “Python Tutorial” press space and type the letter “f”, you will see Google suggesting keyword ideas and search queries!

And if you Add “how” before the “python tutorial” keyword in Google Search. you will get question suggestions.

That's it!

How To Read Google autosuggestions programmatically?

We have two approaches:

Using Web Automation and Scraping
Using Google Search Free API Call

I chose the second approach simply because with web automation, the operation will be slower, and you may get Re-captcha to solve, and things will be complicated.

API Call for Google Keyword Suggestions:

open your browser, and paste this url:

http://google.com/complete/search?output=toolbar&gl=COUNTRY&q=Your_QUERY

You can change the country and the query to get google autosuggestions in XML format using this call as shown below:

I think you got the point, we will call the Google Suggestions API, then fetch the results from our code.

Python code:


import requests
import xml.etree.ElementTree as ET

# Define Class

class QuestionsExplorer:
    def GetQuestions(self, questionType, userInput, countryCode):
        questionResults = []
        # Build Google Search Query
        searchQuery = questionType + " " + userInput + " "
        # API Call
        googleSearchUrl = "http://google.com/complete/search?output=toolbar&gl=" + \
            countryCode + "&q=" + searchQuery

        # Call The URL and Read Data
        result = requests.get(googleSearchUrl)
        tree = ET.ElementTree(ET.fromstring(result.content))
        root = tree.getroot()
        for suggestion in root.findall('CompleteSuggestion'):
            question = suggestion.find('suggestion').attrib.get('data')
            questionResults.append(question)

        return questionResults


# Get a Keyword From The User
userInput = input("Enter a Keyword: ")

# Create Object of the QuestionsExplorer Class
qObj = QuestionsExplorer()

# Call The Method and pass the parameters
questions = qObj.GetQuestions("is", userInput, "us")

# Loop over the list and pring the questions
for result in questions:
    print(result)

print("Done")

C# code:

//start

using System;
using System.Collections.Generic;
using System.Net.Http;
using System.Xml.Linq;

namespace GoogleSuggestion
{
    //Define Class
    class QuestionsExplorer
    {
        public List<string> GetQuestions(string questionType, string userInput, string countryCode)
        {
            var questionResults = new List<string>(); //List To Return

            //Build Google Search Query
            string searchQuery = questionType + " " + userInput + " ";
            string googleSearchUrl =
            "http://google.com/complete/search?output=toolbar&gl=" + countryCode + "&q=" + searchQuery;


            //Call The URL and Read Data
            using (HttpClient client = new HttpClient())
            {
                string result = client.GetStringAsync(googleSearchUrl).Result;

                //Parse The XML Documents
                XDocument doc = XDocument.Parse(result);

                foreach (XElement element in doc.Descendants("CompleteSuggestion"))
                {
                    string question = element.Element("suggestion")?.Attribute("data")?.Value;
                    questionResults.Add(question);
                }

            }

            return questionResults;

        }



    }

    class Program
    {
        static void Main(string[] args)
        {
            //Get a Keyword From The User
            Console.WriteLine("Enter a Keyword:");
            var userInput = Console.ReadLine();

            //Create Object of the QuestionsExplorer Class
            var qObj = new QuestionsExplorer();

            //Call The Method and pass the parameters
            var questions = qObj.GetQuestions("what", userInput, "us");

            //Loop over the list and pring the questions
            foreach (var result in questions)
            {
                Console.WriteLine(result);
            }

            //Finish
            Console.WriteLine();
            Console.WriteLine("---Done---");
            Console.ReadLine();
        }
    }



}

//end

Get Source Codes on Github:

Conclusion

In this post, I showed you how you can build a keyword research tool based on Google's Suggestion API with the power of two popular programming languages, C# and Python.

Maybe you are wondering how to get search metrics, like CPC, Monthly Search Volume, and Competition. The answer is in this post.

Take care, and Happy Coding!