DEV Community

Joel Jaison
Joel Jaison

Posted on

Scraping GeeksforGeeks User Profiles with Node.js

In this tutorial, we will explore how to scrape user profiles on GeeksforGeeks using Node.js. We'll utilize the Express framework, Axios for making HTTP requests, Cheerio for parsing HTML, and Mongoose for working with MongoDB.

Project Setup

To get started, make sure you have Node.js and npm (Node Package Manager) installed on your machine. Then, follow these steps:

  1. Create a new directory for your project and navigate to it in your terminal.
  2. Initialize a new Node.js project by running the following command:
npm init -y
Enter fullscreen mode Exit fullscreen mode
  1. Install the necessary dependencies:
npm install express axios cheerio cors dotenv mongoose

Enter fullscreen mode Exit fullscreen mode

Setting up the Express Server

Let's start by setting up an Express server that will handle the scraping and API requests.

Create a new file called server.js and add the following code:

const express = require('express');
const axios = require('axios');
const cheerio = require('cheerio');
const cors = require('cors');
const mongoose = require("mongoose");

const app = express();


app.get('/profile/:username', async (req, res) => {


const port = process.env.PORT || 3000;
app.listen(port, () => {
  console.log(`Server running on port ${port}`);

Enter fullscreen mode Exit fullscreen mode

Scraping GeeksforGeeks User Profiles

To scrape user profiles on GeeksforGeeks, we'll use Axios to make HTTP requests and Cheerio to parse the HTML response.

Within the /profile/:username route, add the following code:

app.get('/profile/:username', async (req, res) => {
  const { username } = req.params;
  try {
    const url = `${username}`;
    const response = await axios.get(url);
    const $ = cheerio.load(;

    const name = $('.profile_name').text().trim();
    const institution = $('.basic_details_data a').text().trim();
    const longestStrek = $('.streakCnt.tooltipped').text().trim();
    const rank = $('.rankNum').text().trim();
    const coddingScore = $('.score_card_value').text().trim();
    const overallScore = coddingScore.slice(0, 3);
    const solvedProblemsCount = coddingScore.slice(3).replace(/_/g, '');
    const totalSubmissions = $('.heatmap_header .numProblems').text().trim();

    const profile = {

  } catch (error) {
    res.status(500).json({ error: 'Failed to fetch profile data' });

Enter fullscreen mode Exit fullscreen mode

In the above code, we make a GET request to the GeeksforGeeks user profile URL, load the HTML response using Cheerio, and then scrape the required profile information.

Building the Frontend

Now that we have the backend set up to scrape and store profiles, let's build a simple frontend to display the profile data.

Create a new file called index.html and add the following code:

<!DOCTYPE html>
<html lang="en">
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>GeeksforGeeks User Profiles</title>
  <h1>GeeksforGeeks User Profiles</h1>
  <input type="text" id="username" placeholder="Enter a username">
  <button onclick="fetchProfile()">Fetch Profile</button>
  <div id="profile"></div>

  <script src="script.js"></script>

Enter fullscreen mode Exit fullscreen mode

In the above HTML, we have a simple form to enter a username and a button to fetch the profile data.

Create a new file called script.js and add the following code:

async function fetchProfile() {
  const username = document.getElementById('username').value;
  const response = await fetch(`http://localhost:3000/profile/${username}`);
  const data = await response.json();
  const profileDiv = document.getElementById('profile');
  profileDiv.innerHTML = JSON.stringify(data, null, 2);

Enter fullscreen mode Exit fullscreen mode

This JavaScript code fetches the profile data from the server and displays it in the profile div.

Running the Project

To run the project, start the Express server by running the following command:

node server.js
Enter fullscreen mode Exit fullscreen mode

Open index.html in a web browser, enter a GeeksforGeeks username, and click the "Fetch Profile" button. The profile data will be displayed on the page.

That's it! You have successfully created a Node.js project that scrapes user profiles on GeeksforGeeks. You can now deploy it to a hosting platform and share it with others.

Happy coding!

Top comments (3)

joeljaison394 profile image
Joel Jaison

Thank you for sharing your experience with frontend scraping. I completely agree with you.

While frontend scraping can be a convenient approach, it often leads to slower loading times and performance issues. Moreover, when dealing with websites that require authentication or user credentials for scraping, exposing sensitive information on the client-side can pose significant security risks. That's why server-side scraping is a preferred solution, providing better efficiency, security, and a smoother user experience.