DEV Community

Cover image for How to Fetch Wikipedia Articles Using Node.js (Express, TypeScript, Axios)
PO
PO

Posted on • Edited on

How to Fetch Wikipedia Articles Using Node.js (Express, TypeScript, Axios)

In this blog, we'll create a simple web app that fetches Wikipedia articles using Node.js. We'll use Express for the server, TypeScript for type safety, and Axios to make HTTP requests to the Wikipedia API. We'll also create a basic HTML page to display the fetched articles.


Step 1: Setting Up the Project

First, create a new directory for your project and navigate into it:

mkdir wikipedia-fetcher
cd wikipedia-fetcher
Enter fullscreen mode Exit fullscreen mode

Initialize a new Node.js project:

npm init -y
Enter fullscreen mode Exit fullscreen mode

Install the necessary dependencies:

npm install express axios
npm install --save-dev typescript @types/node @types/express ts-node
Enter fullscreen mode Exit fullscreen mode

Create a tsconfig.json file in the root directory for TypeScript configuration:

{
  "compilerOptions": {
    "target": "ES6",
    "module": "commonjs",
    "outDir": "./dist",
    "rootDir": "./src",
    "strict": true,
    "esModuleInterop": true
  }
}
Enter fullscreen mode Exit fullscreen mode

Step 2: Setting Up Express Server

Create a src directory and inside it, create an index.ts file:
In src/index.ts, set up a basic Express server:

import express from 'express';
import axios from 'axios';

const app = express();
const PORT = 3000;

app.use(express.static('public'));

app.get('/', (req, res) => {
  res.sendFile(__dirname + '/public/index.html');
});

app.get('/api/wiki', async (req, res) => {
  const { query } = req.query;
  if (!query) {
    return res.status(400).send('Query parameter is required');
  }

  try {
    const response = await axios.get(`https://en.wikipedia.org/api/rest_v1/page/summary/${query}`);
    res.json(response.data);
  } catch (error) {
    res.status(500).send('Error fetching Wikipedia article');
  }
});

app.listen(PORT, () => {
  console.log(`Server is running on http://localhost:${PORT}`);
});
Enter fullscreen mode Exit fullscreen mode

Step 3: Creating the HTML Page

Create a public directory in the root directory and inside it, create an index.html file:
In public/index.html, add the following basic HTML:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Wikipedia Article Fetcher</title>
</head>
<body>
    <h1>Wikipedia Article Fetcher</h1>
    <input type="text" id="searchQuery" placeholder="Enter search term">
    <button onclick="fetchArticle()">Fetch Article</button>
    <div id="articleContent">
        <h2 id="articleTitle"></h2>
        <p id="articleExtract"></p>
        <p><strong>Last Updated:</strong> <span id="articleLastUpdated"></span></p>
        <p><strong>Page ID:</strong> <span id="articlePageID"></span></p>
        <p><strong>Length:</strong> <span id="articleLength"></span> characters</p>
        <p><a id="desktopLink" href="" target="_blank">Read more on Wikipedia (Desktop)</a></p>
        <p><a id="mobileLink" href="" target="_blank">Read more on Wikipedia (Mobile)</a></p>
    </div>

    <script>
        async function fetchArticle() {
            const query = document.getElementById('searchQuery').value;
            const response = await fetch(`/api/wiki?query=${encodeURIComponent(query)}`);
            const data = await response.json();

            document.getElementById('articleTitle').innerText = data.title;
            document.getElementById('articleExtract').innerText = data.extract;
            document.getElementById('articleLastUpdated').innerText = new Date(data.last_updated).toLocaleString();
            document.getElementById('articlePageID').innerText = data.pageid;
            document.getElementById('articleLength').innerText = data.length;
            document.getElementById('desktopLink').href = data.content_urls.desktop.page;
            document.getElementById('mobileLink').href = data.content_urls.mobile.page;
        }
    </script>
</body>
</html>

Enter fullscreen mode Exit fullscreen mode

Step 4: Running the Application

To run the application, add the following script to your package.json:

"scripts": {
  "start": "ts-node src/index.ts"
}
Enter fullscreen mode Exit fullscreen mode

Now, start the server:

npm start
Enter fullscreen mode Exit fullscreen mode

Open your browser and navigate to http://localhost:3000. You should see a simple form where you can enter a search term to fetch Wikipedia articles.

Project Structure

wikipedia-fetcher/
├── node_modules/
├── public/
│   └── index.html
├── src/
│   └── index.ts
├── .gitignore
├── package.json
├── tsconfig.json
└── README.md
Enter fullscreen mode Exit fullscreen mode

Conclusion

In this post, we learned how to fetch and display Wikipedia articles using the MediaWiki API. This is just the beginning, and there's a lot more you can explore on your own. Here are some ideas for further exploration:

  1. Search Suggestions: Implement a feature that provides search suggestions as the user types.
  2. Error Handling: Improve error handling to provide more user-friendly messages.
  3. Styling: Enhance the styling of the HTML page using CSS.
  4. Pagination: Add pagination to handle multiple search results.
  5. Caching: Implement caching to reduce the number of API calls for frequently searched terms.
  6. Advanced Queries: Explore more advanced queries using the MediaWiki API. You can find more information here.

Let me know if you have any questions!
Thanks for reading!
PO.

Top comments (0)