Using Google Drive with Perl

I often use Google Drive to transfer files between devices, for example from my Mac to my Linux laptop. It works great, but there can be some boring repetitive work involved depending on how often you do the same operation. For example, when I want to transfer a file from the Mac, I open up a web browser (usually Google Chrome) and point it to drive.google.com, the Google Drive web interface. From here, I try to locate the directory I want to upload the file to which might involve some scrolling in the directory listing. After that, I need to click on the directory name to view its content. Next, I right-click on an empty area in the directory list and select "Upload files" from a drop-down menu. Then a file selection dialog appears and I have to locate the directory and file I want to upload in the dialog, and after some more mouse movements and clicks I finally press the "Ok" button to select the file. Similarly, on the other device (like my Linux laptop) where I want to download the file, I need to locate the file in web UI and then right-click it and select "Download" from the drop-down menu to download the file.

Luckily, Google has provided the Google Drive API that can be used by user scripts to communicate with the cloud server. So the first thing I wanted to check was if someone already had written Perl code that made use of this API. After a quick search on MetaCPAN I found a module Net::Google::Drive::Simple that looked promising. When reading through the module's documentation, I realized that the tricky part could be to get hold of an "access token". The script needs this token to be able to connect to the Google Drive API server. The module claimed that it could generate the access token from a secret credential token which I first had to obtain from Google Cloud. Then, when the module had generated this access token the rest should be fairly simple was my impression. Well.., let's see what happend 👾

Creating a Google account

In order to get the Google Drive credentials, the first thing you need to have is a Google account (obviously). This allows you to use Google developer products, including Google Cloud Console, Cloud SDK, Cloud Logging, and Cloud Monitoring. If you don't have a Google account, you can visit https://cloud.google.com/apis/docs/getting-started to get started. Creating a Google account, with some amount of cloud storage is free, see https://cloud.google.com/free for more information.

Creating a Google project

Next, to use the Cloud APIs you need to set up a Google project. A project is equivalent to a developer account. It serves as a resource container for your Google Cloud resources and provides an isolation boundary for your usage of Google Cloud services, so you can manage quota limits and billing independently at the project level. Usage telemetry and dashboards are grouped by projects as well. If you don't already have a project, you can create one using the Cloud Console, see this link for more information. Here is what I did to create a new project:

• In the Google Cloud console, click the drop down menu in top left corner, and select "IAM & Admin" -> "Manage Resources",

• from the "Select organization" drop-down list at the top of the page, you can select the organization in which you want to create a project. If you are a free trial user, skip this step, as this list does not appear.

• Then click "Create Project", and in the "New Project" window that appears, enter a project name.
  

  A project name can contain only letters, numbers, single quotes, hyphens, spaces, or exclamation points, and must be between 4 and 30 characters. I chose "DesktopApp" for the project name, and will use this name in the further discussion.

• Enter the parent organization or folder in the "Location" box. That resource will be the hierarchical parent of the new project. If you do not have a parent folder, just leave "No organization" (as I did) in the "Location" box.

• When you're finished entering new project details, click the "CREATE" button.

Enabling the Google Drive API

Next, in order to use Google Drive API from a script you need to enable it.

• In the Google Cloud console, click the drop down menu in top left corner, and select "APIs & Services" -> "Library". Type "Google Drive" in the search bar. Select "Google Drive API" from the result list.

• Select the "DesktopApp" project, by clicking project button in the top menu bar. If it already shows "DesktopApp", you can skip this step.

• Click the "ENABLE" button.

Configure the app's scopes and the consent screen

The script that we will write to access Google Drive is called an "app". The "user" of the script is in our case the same as the writer (ourselves) of the script, but it need not be. You could send the script to someone else or publish it on the internet. In order for the app to get access to the user's Google Drive, the user must approve that script can access his drive under a certain "scope". For example, the user may give the script read-only access.

The first time the script is trying to access the drive, the user is presented with a "consent screen" where they can select which scopes the script can use to access their drive. After the user has given their consent, the Google Cloud server returns an access token to the script which it can use to access the drive without asking the user for permission the next time. The access token will expire after a certain time, and then the script needs to refresh the token. The user can also revoke the access right of the script at a later time if they so wishes, then the script's access token becomes invalid and it cannot be refreshed. The script must then repeat the process with presenting the user with the consent screen in order to get further access. Read more about the OAuth2 protocol for authentication here.

Now, in order to setup what scopes should be presented to the user in the consent screen (see workspace/guides/create-credentials) do the following:

• In the Google Cloud console, click the drop down menu in top left corner, and select "APIs & Services" -> "Credentials". The credential page for your project appears.

• Select the "DesktopApp" project, by clicking project button in the top menu bar. If it already shows "DesktopApp", you can skip this step.

• Click the "CONFIGURE CONSENT SCREEN" button on the right side. (If you have already configured the consent screen and want to modify it, you should instead click on "OAuth consent screen" in the left side bar)

• Select the user type for the app. Click "External".
  (If you are a google workspace user and wants the app to be internal to your organization, you could select "Internal" instead.)

• Click the "CREATE" button

• In the App information screen, type in a name for the app. This name will appear in the consent screen that is presented to the user during the authorization procedure.

• Enter your email in the "User support email" field.

• Skip the "App logo" field.

• Also skip the "App domain" fields.

• Also skip the "Authorized domains" field.

• Enter your email address in the "Developer contact information" field.

• Click the "SAVE AND CONTINUE" button

• In the "Edit app registration" screen, click the "ADD OR REMOVE SCOPES" button.

• From the popup window "Update selected scopes" click the checkmark next to the following scopes:

  • .../auth/drive.file
  • .../auth/drive.metadata.readonly
  • .../auth/drive.readonly

  Note that only the scopes you select in the popup window can be requested by the app. For instance, if the app later requests all of the above three scopes, the user will be presented with a consent screen with the three alternatives, like this:
  
  The first alternative in the screen shot above corresponds to the drive.readonly scope, the second corresponds to the drive.metadata.readonly scope, and the third one corresponds to the drive.file scope.

• Click the "UPDATE" button at the bottom of the window

• Click "SAVE AND CONTINUE" in the "Edit app registration" screen.

• In the "Test users" screen, click the "+ ADD USERS" button, and add yourself (your gmail address) to the list.
  

• Click "SAVE AND CONTINUE".

• The Summary screen is shown. Click the "BACK TO DASHBOARD" button.

Create credentials

In order for the app (the script) to obtain an access token, it needs to pass a set of credentials to to the Google authorization server. Together with the credentials, the script passes (as a parameter) the set of scopes that it would like to access. The authorization server then present the user of the script with a screen where they can log in with their Google account. After logging in, the user is asked whether they are willing to grant one or more permissions that your app is requesting. This process is called user consent as discussed in the previous section.

To obtain the set of credentials that the script should present to the authorization server (see: workspace/guides/create-credentials), we do the following:

• In the Google cloud console, click the drop down menu in top left corner, and select "APIs & Services" -> "Credentials". The credential page for your project appears.
  

• Select the "DesktopApp" project, by clicking project button in the top menu bar. If it already shows "DesktopApp", you can skip this step.

• Click the "+ CREATE CREDENTIALS" button on the top of the
  page. Select "OAuth Client ID" from the drop down menu.

• Select "Desktop app" from the "Application type" drop down menu.

• Select a client name in the "Name" field, or use the suggested name.

• Click the "CREATE" button at the bottom of the page.

• Click "OK" in the "OAuth client created" dialog box.

• Next, still in the Google cloud console: click the drop down menu in top left corner, and select "APIs & Services" -> "Credentials". The credential page for your project appears.

• Select the "DesktopApp" project, by clicking project button in the top menu bar. If it already shows "DesktopApp", you can skip this step.

• Under "OAuth 2.0 Client IDs", click the download button under the "Actions" heading.
  

• Select "DOWNLOAD JSON" in the popup window.

• Rename the file as "credentials.json" on your computer.

Obtaining the access token

Now as we have the credentials, we can obtain the access token. I have rewritten the google-drive-init script provided by Net::Google::Drive::Simple slightly:

use v5.26; # Indented here-docs introduced in 5.26
use feature qw(say);
use strict;
use warnings;
use experimental qw(signatures);
use open qw(:std :encoding(utf-8));
use OAuth::Cmdline::GoogleDrive;
use OAuth::Cmdline::Mojo;
use JSON;
{
    my $credentials = read_credentials();
    my $token_dir = '.';
    # NOTE: if you change the scopes below, you should delete the previous
    #  access token file (.google-drive.yml) and then rerun this script..
    my @scopes  = (
        "drive.file",
        "drive.metadata.readonly",
        "drive.readonly"
    );
    my $scopes = join ' ', map { "https://www.googleapis.com/auth/$_" } @scopes;
    my $oauth = OAuth::Cmdline::GoogleDrive->new(
        client_id     => $credentials->{client_id},
        client_secret => $credentials->{client_secret},
        login_uri     => $credentials->{auth_uri},
        token_uri     => $credentials->{token_uri},
        scope         => $scopes,
        homedir       => $token_dir
    );
    print <<~'INFO_STR';
    Running the token collector web server at http://localhost:8082
    - Open your web browser and go to http://localhost:8082, then click the link
      "Login on google-drive" and fill out the OAuth consent screen granting
      your Perl scripts access to your google drive.
    - Click the check boxes for what the app can access (the scopes)
    - Then click "Continue"
    - When the message "Tokens saved in ./.google-drive.yml" appears, the
      access token was successfully generated and saved to the local file
      .google-drive.yml.
    - You can then quit the server (this script) by pressing CTRL-C.
    ----------------------------------------------------------------
    INFO_STR
    my $app = OAuth::Cmdline::Mojo->new(
        oauth => $oauth,
    );
    $app->start( 'daemon', '-l', $oauth->local_uri );
    # After you filled out the consent screens, you need to press CTRL-C in the shell
    #  window to exit this script.
    say "Server finished.";
    say "Done.";
}


sub read_credentials {
    my $fn = 'credentials.json';
    open ( my $fh, '<', $fn ) or die "Could not open file '$fn': $!";
    my $str = do { local $/; <$fh> };
    close $fh;
    my $json = JSON->new();
    my $hash = $json->decode( $str );
    die "$fn: Expected installed app" if !(exists $hash->{installed});
    return $hash->{installed};
}

If you run this script (after installing OAuth::Cmdline), the access token should be saved in a file .google-drive.yml in the current directory.

Uploading a file to Google Drive

Finally, we have the access token and we can start to use the module Net::Google::Drive::Simple. We will start by uploading a file to Google Drive. Unfortunately, when I tested the file_upload( $file, $dir_id ) method, it did not work as expected, see issue # 47. I created a pull request to fix the issue. Since the pull request has not yet been merged I suggest that you (if you want to upload files) install the module (Net::Google::Drive::Simple) like this:

git clone https://github.com/mschilli/net-google-drive-simple.git
cd net-google-drive-simple
git fetch origin pull/48/head:pr_48
git checkout pr_48
cpanm .

Now, create a dummy hello.txt file that we will upload to the root directory of Google Drive:

$ echo "Hello world!" > hello.txt 

Then save the following script as upload.pl (in the same directory as you have the access token file):

use feature qw(say);
use strict;
use warnings;
use Net::Google::Drive::Simple;
use OAuth::Cmdline::GoogleDrive;

my $token_dir = '.';
my $oauth = OAuth::Cmdline::GoogleDrive->new(
    scope         => "https://www.googleapis.com/auth/drive.file",
    homedir       => $token_dir
);
my $gd = Net::Google::Drive::Simple->new( oauth => $oauth );
my $file = 'hello.txt';
my $parent = 'root';
my $file_id = $gd->file_upload( $file, $parent ) or die "Upload failed: $!";
say "Uploaded file: $file (id: $file_id) to directory with id: $parent";

Then run the script:

$ perl upload.pl
Uploaded file: hello.txt (id: 1x5pc9_rvAFeLgkp0zLUeuKAIMWptplz9) to directory with id: root

to upload the hello.txt file to your Google Drive!

Downloading a file from Google Drive

It is not enough to simply specify the directory and filename to download a file. There can be multiple files with the same name in the same directory, however a file's ID is unique. When we uploaded the hello.txt the API returned the ID of the uploaded file, namely 1x5pc9_rvAFeLgkp0zLUeuKAIMWptplz9. Another way to determine the ID is to right click a file in the Google Drive UI and select "Get link" from the drop down menu. If you know that the file name is unique, a third way is to use the search() method. I first show how to download a file when you know the ID. The following script can download a file that the app created (if you want to download any file, you should replace the drive.file scope with the drive.readonly scope) :

use feature qw(say);
use strict;
use warnings;
use Net::Google::Drive::Simple;
use OAuth::Cmdline::GoogleDrive;

my $token_dir = '.';
my $oauth = OAuth::Cmdline::GoogleDrive->new(
    scope         => "https://www.googleapis.com/auth/drive.file",
    homedir       => $token_dir
);
my $gd = Net::Google::Drive::Simple->new( oauth => $oauth );
my $file_id = '1x5pc9_rvAFeLgkp0zLUeuKAIMWptplz9';
my $url = download_url($file_id);
say "Downloading from: $url";
my $save_fn = 'hello2.txt';
if ($gd->download($url, $save_fn)) {
    say "Downloaded file with id '$file_id' as '$save_fn'";
}
else {
    die "Failed to download $file_id: ", $gd->error();
}

sub download_url {
        'https://www.googleapis.com/drive/v2/files/'
      . $_[0]
      . '?alt=media&source=downloadUrl';
}

Alternatively, if you know that the file name is unique across the whole Google Drive and you don't want to look in the UI for the file ID, you can use the search() method like this:

use feature qw(say);
use strict;
use warnings;
use Net::Google::Drive::Simple;
use OAuth::Cmdline::GoogleDrive;

my $token_dir = '.';
my $oauth = OAuth::Cmdline::GoogleDrive->new(
    scope         =>  ( "https://www.googleapis.com/auth/drive.readonly" . ' '
                     . "https://www.googleapis.com/auth/drive.file"),
    homedir       => $token_dir
);
my $gd = Net::Google::Drive::Simple->new( oauth => $oauth );
my $fn = 'hello.txt';
my $children= $gd->search(
    { maxResults => 2 },
    { page => 1 },
    "title = '$fn'"
);
die "Multiple files matching '$fn' found." if @$children > 1;
die "No files matching '$fn' found." if @$children == 0;

my $child = $children->[0];
my $save_fn = 'hello2.txt';
if ($gd->download($child, $save_fn)) {
    say "Downloaded file with name '$fn' as '$save_fn'";
}
else {
    die "Failed to download $fn: ", $gd->error();
}