Build YouTube API Data Fetching Using Rust

Are you looking to automate fetching your YouTube video data for analytics, content management, or other purposes?

In this guide, we'll create a Rust program that leverages the YouTube Data API to fetch videos from a specified YouTube channel and save the information to a CSV file.

This step-by-step tutorial will cover everything you need:

• Setting up the YouTube Data API key on Google Cloud Console.
• Creating and configuring a new Rust project.
• Writing Rust code to fetch videos and save them to a CSV file.
• Running the program and seeing the results.
• Get your file!

If you prefer a video version

All the code is available in the video description: YouTube API Data Fetching Using Rust - Step-by-Step Guide

Agenda

• Section 1: Set up the YouTube Data API key on Google Cloud Console.
• Section 2: Create and configure a new Rust project.
• Section 3: Write Rust code to fetch videos and save them to a CSV file.
• Section 4: Run the program and get your file!

Section 1: Setting Up the YouTube Data API Key

Before diving into the Rust code, we must set up access to the YouTube Data API by obtaining an API key. This key will allow our Rust program to interact with the YouTube API and fetch data from your YouTube channel. Follow these steps to create a project on the Google Cloud Platform and enable the YouTube Data API.

Step 1: Go to Google Cloud Platform

To start, go to the Google Cloud Console. This is where you will create a new project and manage your API access.

Step 2: Create a New Project

Once on the Google Cloud Console, click the New Project button in the top navigation bar. Give your project a name that is meaningful to you (e.g., "YouTube Data Fetcher") and click Create.

Step 3: Enable the YouTube Data API v3

Next, we need to enable the YouTube Data API for the project you just created.

Click on the Enable APIs and Services button at the dashboard's top.

• In the API Library, search for YouTube Data API v3.

• Click on the Enable button to activate it for your project.

Step 4: Create API Credentials

Now that the API is enabled, we need to create credentials to access it.

Click on Create Credentials at the top of the page.

In the sidebar, select Credentials and then click on API Key.

Step 5: Save Your API Key

After creating the API key, a popup will appear with your new API key. Copy this key and save it somewhere safe, as you’ll need it later in your Rust project.

Important: Treat your API key like a password. Do not share it publicly or commit it to any public repositories.

Next Steps
Now that you have your API key, you're ready to set up the Rust project that will use this key to fetch your YouTube videos. In the next section, we will create a new Rust project and start writing the code.

Section 2: Creating and Configuring a New Rust Project

Now that you have your YouTube Data API key, it's time to set up your Rust project. In this section, we will create a new Rust project using Cargo, the Rust package manager, and add the necessary dependencies to interact with the YouTube API and handle CSV files.

Step 1: Create a New Rust Project

First, create a new Rust project by running the following commands in your terminal:

cargo new youtube_video_fetcher
cd youtube_video_fetcher

This will create a new directory named youtube_video_fetcher containing all the files you need for a basic Rust project. The cd command changes your current directory to the newly created project directory.

Step 2: Add Required Dependencies

Next, open the Cargo.toml file in the root of your project directory. This file is where you'll specify the dependencies your project needs.

Add the following dependencies under the [dependencies] section:

[dependencies]
reqwest = { version = "0.11", features = ["json"] }
serde_json = "1.0"
csv = "1.1"
dotenv = "0.15"
tokio = { version = "1", features = ["full"] }

Explanation of the Dependencies:

• reqwest: A popular HTTP client library in Rust that we'll use to send requests to the YouTube Data API.
• serde_json: A library for parsing JSON data, which will help us handle the responses from the API.
• csv: A library for reading and writing CSV files, allowing us to save the fetched video data.
• dotenv: A library to load environment variables from a .env file, which is useful for securely handling API keys.
• tokio: An asynchronous runtime for Rust, enabling us to handle asynchronous operations like HTTP requests efficiently.

You project should look like this:

Step 3: Set Up Your .env File

To keep your API key secure, create a .env file in the root directory of your project. Add your YouTube API key to this file:

YOUTUBE_API_KEY=YOUR_API_KEY_HERE

Replace YOUR_API_KEY_HERE with the actual API key you obtained in Section 1. This file will be read by the Rust program to access your API key securely.

Note: Make sure to add .env to your .gitignore file if you are using version control to avoid accidentally exposing your API key.

Step 4: Confirm Your Setup

Your project is now set up and ready for coding! You should have:

A new Rust project directory named youtube_video_fetcher.
Updated the Cargo.toml file with the necessary dependencies.
Created a .env file with your API key.
In the next section, we will begin writing the Rust code to fetch videos from YouTube and save them to a CSV file.

Section 3: Writing the Rust Code

With your project set up, it's time to start coding! In this section, we'll implement the Rust functions necessary to fetch video data from YouTube using the YouTube Data API and save the results to a CSV file. We'll go step-by-step, explaining each function to ensure you understand how everything works together.

Step 1: Create the Main Function

Start by writing the main function in your src/main.rs file. This function will initialize your project by loading environment variables, setting up the API key, and calling the functions to fetch videos and write them to a CSV file.

Add the following code to your src/main.rs:

use dotenv::dotenv;
use std::env;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    dotenv().ok(); // Load environment variables from .env file

    // Fetch API key from environment variables
    let api_key = env::var("YOUTUBE_API_KEY").expect("YOUTUBE_API_KEY must be set");

    // Set the channel ID (change this to your desired YouTube channel)
    let channel_id = "UCBRxDSTfr2aJVODDh4WG_7g"; 

    // Placeholder for function calls (to be implemented)
    println!("API Key: {}, Channel ID: {}", api_key, channel_id);

    Ok(())
}

Explanation:

• dotenv().ok();: Loads environment variables from the .env file, making the API key accessible within the program.
• env::var("YOUTUBE_API_KEY"): Retrieves the API key from the environment variables. Make sure your .env file is correctly set up.
• channel_id: This is the ID of the YouTube channel from which you want to fetch videos. You can change this to any channel ID you wish.

Note: The main function is marked with #[tokio::main] to allow asynchronous execution, which is necessary for performing asynchronous HTTP requests.

Step 2: Implement the Function to Fetch Videos

Next, implement the fetch_videos function. This function will make an HTTP request to the YouTube Data API to retrieve videos from the specified channel.

Add the following code below your main function in src/main.rs:

use reqwest::Client;
use serde_json::Value;

async fn fetch_videos(api_key: &str, channel_id: &str) -> Result<Vec<Value>, Box<dyn std::error::Error>> {
    let client = Client::new(); // Create a new HTTP client
    let mut videos = Vec::new(); // Initialize a vector to store videos
    let mut page_token = String::new(); // Token to handle pagination

    loop {
        // Build the API request URL
        let url = format!(
            "https://www.googleapis.com/youtube/v3/search?key={}&channelId={}&part=snippet,id&order=date&maxResults=50&type=video&pageToken={}",
            api_key, channel_id, page_token
        );

        let response = client.get(&url).send().await?; // Send the HTTP GET request

        // Check if the response was successful
        if !response.status().is_success() {
            println!("API request failed with status: {}", response.status());
            println!("Response body: {}", response.text().await?);
            return Err("API request failed".into());
        }

        let json: Value = response.json().await?; // Parse the response body as JSON

        // Check for API errors
        if let Some(error) = json.get("error") {
            print!("API returned an error: {:?}", error);
            return Err("API returned an error".into());
        }

        // Extract video items and add to the videos vector
        if let Some(items) = json["items"].as_array() {
            videos.extend(items.clone());
        }

        // Handle pagination by checking for the nextPageToken
        if let Some(next_page_token) = json["nextPageToken"].as_str() {
            page_token = next_page_token.to_string();
        } else {
            break;
        }
    }

    Ok(videos) // Return the list of videos
}

Explanation:

• Client::new(): Creates a new HTTP client for sending requests.
• loop { … }: A loop that continues until all video pages are fetched. The YouTube API returns a maximum of 50 results per request, so multiple requests may be needed.
• format!(…): Constructs the API request URL using the API key, channel ID, and other parameters.
• response.status().is_success(): Checks if the HTTP response status indicates success.
• json["items"].as_array(): Extracts the array of video items from the JSON response.
• nextPageToken: Handles pagination by continuing to fetch more pages if available.

Step 3: Write the Function to Save Data to CSV

Now, let's implement the write_to_csv function. This function will save the fetched video data into a CSV file.

Add the following code below the fetch_videos function:

use csv::Writer;

fn write_to_csv(videos: Vec<Value>) -> Result<(), Box<dyn std::error::Error>> {
    // Create a new CSV writer and specify the output file name (change if desired)
    let mut wtr = Writer::from_path("youtube_videos.csv")?;

    // Write the header row
    wtr.write_record(&["Video ID", "Title", "Description", "Published At"])?;

    for video in videos {
        let snippet = &video["snippet"];

        // Write each video's data to the CSV file
        wtr.write_record(&[
            video["id"]["videoId"].as_str().unwrap_or(""),
            snippet["title"].as_str().unwrap_or(""),
            snippet["description"].as_str().unwrap_or(""),
            snippet["publishedAt"].as_str().unwrap_or(""),
        ])?;
    }

    wtr.flush()?; // Ensure all data is written to the file
    Ok(())
}

Explanation:

• Writer::from_path(…): Creates a new CSV writer that writes to a file named youtube_videos.csv. You can change the file name here if needed.
• wtr.write_record(…): Writes the header row and each video's data to the CSV file.
• wtr.flush(): Ensures all buffered data is written to the file.

Step 4: Update the Main Function to Call Other Functions

Finally, update the main function to call fetch_videos and write_to_csv:

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    dotenv().ok(); // Load environment variables from .env file

    let api_key = env::var("YOUTUBE_API_KEY").expect("YOUTUBE_API_KEY must be set");
    let channel_id = "UCBRxDSTfr2aJVODDh4WG_7g"; // Change to your desired channel ID

    // Fetch videos using the YouTube API
    match fetch_videos(&api_key, channel_id).await {
        Ok(videos) => {
            println!("Fetched {} videos", videos.len());

            if videos.is_empty() {
                println!("No videos found");
            } else {
                // Call the function to write videos to a CSV file
                write_to_csv(videos)?;
                println!("Videos written to CSV");
            }
        }
        Err(e) => {
            println!("Error fetching videos: {}", e);
            if let Some(reqwest_err) = e.downcast_ref::<reqwest::Error>() {
                if let Some(status) = reqwest_err.status() {
                    println!("HTTP Status {}", status);
                }
            }
        }
    }

    Ok(())
}

Explanation:

• fetch_videos(…): Asynchronously fetches videos using the provided API key and channel ID.
• write_to_csv(…): Writes the fetched videos to a CSV file.

Now you have all the code needed to fetch your YouTube videos and save them to a CSV file. In the next section, we will see how to run the program and verify the results.

Section 4: Running Your Program

Now that we've written all the necessary Rust code, it's time to run the program and see it in action! In this section, we will set up the environment variables securely and execute the Rust program to fetch videos from your YouTube channel and save them to a CSV file.

Step 1: Set Up Environment Variables

To securely handle your API key, you need to create a .env file in the root directory of your project. This file will store your API key so that it can be accessed by your Rust program without hardcoding it directly in your code.

• Create a .env File: In the root directory of your project, create a new file named .env.

• Add Your API Key: Open the .env file in your text editor and add the following line:
  

YOUTUBE_API_KEY=YOUR_API_KEY_HERE

Replace YOUR_API_KEY_HERE with the API key you obtained from the Google Cloud Console in Section 1.

Note: Ensure that your .env file is included in your .gitignore file if you are using version control to prevent accidentally exposing your API key to the public.

Step 2: Run Your Rust Program

To run your Rust program, use the following command in your terminal:

cargo run

If everything is set up correctly, the program will:

• Load the API key from the .env file.
• Fetch the videos from the specified YouTube channel using the YouTube Data API.
• Save the fetched data to a CSV file named youtube_videos.csv.

Expected Output

In our case, a new file named youtube_videos.csv will be created in your project directory. Open this file with any text editor or spreadsheet software (like Microsoft Excel or Google Sheets) to see the videos fetched from your YouTube channel.

Troubleshooting Tips:

• Error Fetching Videos: If the program displays an error message like Error fetching videos, ensure that your API key is correct and that the YouTube Data API is enabled for your project.

• API Key Not Found: If you see an error like YOUTUBE_API_KEY must be set, double-check that your .env file is properly formatted and located in the root of your project directory.

• Permission Errors: If you encounter permission errors when creating or writing to the CSV file, ensure you have the correct write permissions for the directory.

Step 3: Verify the Results

Open the generated youtube_videos.csv file. You should see columns like "Video ID," "Title," "Description," and "Published At," with the corresponding data for each video fetched from the YouTube channel.

The output should look similar to the following example (the name of the file is different in the screenshot):

The File should also be visible at the root of your project directory.

Customize Your Program:
Feel free to customize the program further:

Change the Channel ID: To fetch videos from a different channel, replace the channel_id variable in the main function with another YouTube channel ID.
Modify the CSV Output: Edit the write_to_csv function to include additional fields from the YouTube API response, such as video tags, thumbnails, or view counts.

Conclusion

Congratulations!

You have successfully created a Rust program that interacts with the YouTube Data API to fetch videos from a specific channel and save them to a CSV file. This setup can be a great starting point for more advanced automation tasks, data analysis, or content management projects involving YouTube.

If you prefer a video version

All the code is available in the video description: YouTube API Data Fetching Using Rust - Step-by-Step Guide

Leave a comment if you want to leave feedback or need help!

Francesco