DEV Community

Cover image for Build a SERP rank tracker app with this API
SerpApi profile image hil
hil for SerpApi

Posted on • Originally published at serpapi.com

Build a SERP rank tracker app with this API

#api #tutorial

If you're interested in building a SERP ranking tracker app, you'll love our API. SerpApi provides a simple API to access live data from various search engines, including Google, Bing, DuckDuckGo, Yahoo, and others. It enables you to build an app like a SERP ranking tracker.

Rank tracker API illustration.

The idea

To get the ranking position of a website, we need to access the organic results and check where the domain first appears. The organic results data is available through our API. Here are the three APIs we're going to use:

API Design

We'll create a single endpoint where people can receive the ranking results from the above search engine using this parameter:

  • Domain name (String): The website domain we want to track.
  • Keywords (Array): List of keywords that we want to search for.
  • Engines (Object): List of search engines we want to search on. We can also adjust the parameter details based on the API. Refer to the relevant documentation to check the available parameters for each APIs.
POST: /api/rankings

Data:
 - domain (string)
 - keywords (array[string])
 - engines ((array[name, params]))

Example:
{
  "domain": "archive.org",
  "keywords": ["internet archive"],
  "engines": [
    {
      "name": "google",
      "params": {
        "domain": "google.com",
        "gl": "es"
      }
    }
  ]
}
Enter fullscreen mode Exit fullscreen mode

The source code is available on GitHub; feel free to take a look at the detailed implementation here:
https://github.com/hilmanski/rank-tracker-api/

Let's write the code!

I'll use Nodejs for this API; feel free to use other languages/frameworks.

Install Express

Let's use Express to help us clean up the code structure.

npm i express --save
Enter fullscreen mode Exit fullscreen mode

Export your API Key

You can either export your API key in a terminal like the sample below or save it on an .env file.

export SERPAPI_API_KEY=YOUR_ACTUAL_API_KEY
Enter fullscreen mode Exit fullscreen mode

Basic route

Prepare the POST endpoint with relevant parameters

const express = require('express')
const app = express()
const port = 3000

app.use(express.json());

app.post('/api/rankings', async(req, res) => {
  const { keywords, engines, domain } = req.body;

  // detail implementation later

  res.json({ keywords, engines });
})

app.listen(port, () => {
  console.log(`Example app listening on port ${port}`)
})
Enter fullscreen mode Exit fullscreen mode

Validate the input type

Make sure API users use the correct types.

app.post('/api/rankings', async(req, res) => {
  const { keywords, engines, domain } = req.body;

   // Validate keywords
  if (!Array.isArray(keywords) || !keywords.length) {
    return res.status(400).json({ error: 'Keywords and engines must be arrays.' });
  }

  // Validate engines
  for (const engine of engines) {
    if (typeof engine !== 'object' || !engine.name) {
      return res.status(400).json({ error: 'Each engine must be an object with a "name" property.' });
    }
    if (engine.params && typeof engine.params !== 'object') {
      return res.status(400).json({ error: 'Engine "params" must be an object.' });
    }
  }

  // coming soon

})
Enter fullscreen mode Exit fullscreen mode

Run parallel search

Since we're enabling multiple keywords and multiple search engines, we need to run the function in parallel to save us some time.

// Parallel search
  const results = await Promise.all(engines.map(async engine => {
    const rankings = await Promise.all(keywords.map(async keyword => {
      return await getRanking(keyword, engine, cleanDomain);
    }));

    // map keywords - rankings in one array
    const rankingResults = keywords.map((keyword, index) => {
      return [keyword, rankings[index]];
    });

    console.log(rankingResults);

    return { domain, engine, rankingResults };
  }))
Enter fullscreen mode Exit fullscreen mode

*getRanking method coming soon.

GetRanking method implementation

Here is the function that is responsible for running the search for each search engine.

const suportEngines = ['google', 'bing', 'duckduckgo'];

async function getRanking(keyword, engine, domain) {
  const engineName = engine.name.toLowerCase();

  if(!suportEngines.includes(engineName)) {
      console.error(`Error: Engine ${engineName} is not supported.`);
      return;
  }

  return new Promise(async (resolve, reject) => {
      switch(engineName) {
          case 'google':
            resolve(await searchGoogle(keyword, engine.params, domain))
          break;
          case 'bing':
            resolve(await searchBing(keyword, engine.params, domain))
          break;
          case 'duckduckgo':
            resolve(await searchDuckDuckGo(keyword, engine.params, domain))
          break;
          default:
          break;
      }
  })
}
Enter fullscreen mode Exit fullscreen mode

We'll use the native fetch method in NodeJS to request the actual SerpApi endpoint for each search engine.

API to access ranking position in Google

function searchGoogle(keyword, params, domain) {
  let endpoint = `https://serpapi.com/search?q=${keyword}&engine=google&num=100&api_key=${SERPAPI_API_KEY}`
  if(params) {
      endpoint += `&${new URLSearchParams(params).toString()}`
  }

  return fetch(endpoint)
    .then(response => response.json())
    .then(data => {
      const organic_results = data.organic_results;
      let ranking = organic_results.findIndex(result => result.link.includes(domain))
      return ranking + 1;
    })
    .catch(error => {
      console.error(error);
    });
}
Enter fullscreen mode Exit fullscreen mode

API to access ranking position in Bing

function searchBing(keyword, params, domain) {
  let endpoint = `https://serpapi.com/search?q=${keyword}&engine=bing&count=50&api_key=${SERPAPI_API_KEY}`
  if(params) {
      endpoint += `&${new URLSearchParams(params).toString()}`
  }

  return fetch(endpoint)
    .then(response => response.json())
    .then(data => {
      const organic_results = data.organic_results;
      let ranking = organic_results.findIndex(result => result.link.includes(domain))
      return ranking + 1;
    })
    .catch(error => {
      console.error(error);
    });
}
Enter fullscreen mode Exit fullscreen mode

API to access ranking position in DuckDuckGo

function searchDuckDuckGo(keyword, params, domain) {
  let endpoint = `https://serpapi.com/search?q=${keyword}&engine=duckduckgo&api_key=${SERPAPI_API_KEY}`
  if(params) {
      endpoint += `&${new URLSearchParams(params).toString()}`
  }

  return fetch(endpoint)
    .then(response => response.json())
    .then(data => {
      const organic_results = data.organic_results;
      let ranking = organic_results.findIndex(result => result.link.includes(domain))
      return ranking + 1;
    })
    .catch(error => {
      console.error(error);
    });
}
Enter fullscreen mode Exit fullscreen mode

Final Endpoint

Here's what our endpoint looks like

app.post('/api/rankings', async(req, res) => {
  const { keywords, engines, domain } = req.body;

  // Validate keywords
  if (!Array.isArray(keywords) || !keywords.length) {
    return res.status(400).json({ error: 'Keywords and engines must be arrays.' });
  }

  // Validate engines
  for (const engine of engines) {
    if (typeof engine !== 'object' || !engine.name) {
      return res.status(400).json({ error: 'Each engine must be an object with a "name" property.' });
    }
    if (engine.params && typeof engine.params !== 'object') {
      return res.status(400).json({ error: 'Engine "params" must be an object.' });
    }
  }

  // CLean up domain
  // Since people can include https:// or http:// or a subdomain, strip all of it?
  const cleanDomain = domain.replace(/^https?:\/\//, '').replace(/\/$/, '');

  // Parallel search
  const results = await Promise.all(engines.map(async engine => {
    const rankings = await Promise.all(keywords.map(async keyword => {
      return await getRanking(keyword, engine, cleanDomain);
    }));

    // map keywords - rankings in one array
    const rankingResults = keywords.map((keyword, index) => {
      return [keyword, rankings[index]];
    });

    console.log(rankingResults);

    return { domain, engine, rankingResults }
  }))

  res.json(results);
})
Enter fullscreen mode Exit fullscreen mode

Test the API

Let's try out this API via cURL.

No parameter example

curl -X POST http://localhost:3000/api/rankings \
  -H 'Content-Type: application/json' \
  -d '{
    "keywords": ["internet archive"],
    "domain": "archive.org",
    "engines": [
      {
       "name": "google"
     }
    ]
  }'
Enter fullscreen mode Exit fullscreen mode

Using multiple keywords and search engine parameter sample

curl -X POST http://localhost:3000/api/rankings \
  -H 'Content-Type: application/json' \
-d '{
    "keywords": ["internet archive", "digital library archived internet"],
    "domain": "archive.org",
    "engines": [
      {
        "name": "google",
        "params": {
            "google_domain": "google.co.id",
            "gl": "id"
        }
      }
    ]
  }
Enter fullscreen mode Exit fullscreen mode

Using multiple search engines

curl -X POST http://localhost:3000/api/rankings \
  -H 'Content-Type: application/json' \
  -d '{
    "keywords": ["internet archive", "digital archive", "internet library"],
    "domain": "archive.org",
    "engines": [
    {
      "name": "google",
      "params": {
        "domain": "google.com",
        "gl": "es"
      }
    },
    {
      "name": "Bing",
      "params": {
        "cc": "gb"
      }
    },
{
      "name": "duckduckgo",
      "params": {
        "kl": "uk-en"
      }
    }
  ]
  }'
Enter fullscreen mode Exit fullscreen mode

That's it! I hope you like this post. Please let us know if you have any questions. Feel free to also contribute to this project on GitHub.

Top comments (0)

Read next

jamesbmour profile image

Streamlit Part 4: Mastering Media Elements - Logos, Images, Videos, and Audio

James -

nevodavid profile image

40 open-source gems to become the ultimate developer🔥 🚀

Nevo David -

trish_07 profile image

Rust TUI Chat Application - Mastering Terminal User Interfaces

Trish -

robertobutti profile image

Building a website using Markdown content with Next.js App Router and Fusionable

Roberto B. -