Petar G. Petrov

Posted on Oct 17, 2018

Store Encrypted Files in Google Drive

#privacy #security #backup #cryptography

Intro
Goal
Requirements
Usage
Conclusion
Annex - Metadata
References

Intro

By now I reckon you already know that Google Drive offers the astounding 15 GB of storage for free [1]. Microsoft's OneDrive offers just about 5 GB [2] for free and Dropbox, one of the pioneers in the field, about 2 GB [3]. Dropbox are kind of lagging behind and out of all three Google Drive seems the clear winner here in terms of offering a greater amount of storage for no cost at all.

(The amounts of storage described are from October 2018.)

Needless to say, free online storage is a great way to store your bulk of important data - documents, backups and maybe your large collection of pop music mp3s you downloaded back in the 90s. Don't look at me like that! When it comes to music preferences, we all have sinned. Now, if you are like me, keeping sensitive documents, data backups or SQL exports unprotected in the cloud does not really seem much comfortable. Google do offer extended security by using 2-Step verification process [4], which makes it quite hard for your account to get compromised. We must, however, add that the human factor [5] in security cannot be discarded.

Also, call me paranoid, but the following text in Google's Terms of Service [6] leaves me with a feeling of unease:

Our automated systems analyze your content (including emails) to provide you personally relevant product features, such as customized search results, tailored advertising, and spam and malware detection. This analysis occurs as the content is sent, received, and when it is stored.

See? My paranoia is a tad more justified now. So the question is can we store our data on Google Drive in an encrypted manner - and also how hard would be to automate that.

Goal

Alright, case in point - our goal here is to somehow establish a process for server or desktop located files to be encrypted and uploaded to a Google Drive folder. The synchronization will be one way only, meaning files will only be uploaded from the server or desktop machine to a designated Google Drive folder.

We needed a way to collect, encrypt and synchronize files. Collecting files may be realized via shell scripts like Bash on Un*x or Powershell on Windows.

To encrypt our precious data we'll use GnuPG [7], either version 1.4 or 2.1. It should be downloaded using your operating system's package manager, e.g., Aptitute or Brew, or alternatively from GnuPG's website.

For synchronization, there's gdrive [8], short for Google Drive CLI Client. I dig that short name. What's up G! It's written in Go and it's open source. It comes with pre-built binaries for an impressive range of platforms.

Oh, and here's a diagram of the intended workflow that I drew earlier, to make this article seem less boring. Behold!

Requirements

Generate GnuPG Keys

If you already have a GnuPG key pair that you want to use for encryption, you may skip this step completely. But you really shouldn't, because there're a bunch of lame jokes ahead.

Now, I'm going to first start with a fair warning that dealing with GnuPG could, at times, be somewhat irritating. Cryptography is generally complex and unfortunately using GPG could be a bit cumbersome, but we are going to do a minimal set of steps, so it should all be fine. Take a deep breath. Ready?

First, you need a place to store your GPG key rings. A key ring contains one or more public or private keys. By default, this would be a place in your home dir, i.e., /home/user/.gnupg. I recommend using another, separate directory for this setup. This way we can only store the synchronization keys there.

$ cd /var/local
$ mkdir mygpgkeys && chmod 700 mygpgkeys
$ export GNUPGHOME="/var/local/mygpgkeys"

Pull the curtains down and put your dark shades on, we're generating the GnuPG key pair next.

$ cd mygpgkeys
$ gpg --default-new-key-algo rsa4096 --gen-key

By default gpg generates 2048 bits keys, so we add an extra parameter to initiate a 4096 bit key generation. Fill in a name and an Email address. Although recommended, the Email address does not need to be a real one. You should then be asked about a password to secure your private key. Needless to say, choose a good, long password. Yeah, your cat's name doesn't count unless it's a concat of a bunch of Lovecraft's character names. It might take a while for your key to get generated depending on the available entropy on your system [9].

gpg (GnuPG) 2.1.21; Copyright (C) 2017 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

gpg: keybox '/var/local/mygpgkeys/pubring.kbx' created
Note: Use "gpg2 --full-generate-key" for a full featured key generation dialog.

GnuPG needs to construct a user ID to identify your key.

Real name: Max Mustermann
Email address: maxmustermann@example.org
You selected this USER-ID:              
    "Max Mustermann <maxmustermann@example.org>"

Change (N)ame, (E)mail, or (O)kay/(Q)uit? o
...
gpg: /var/local/mygpgkeys/trustdb.gpg: trustdb created
gpg: key 1A2B04F0E994DAF4 marked as ultimately trusted
gpg: directory '/var/local/mygpgkeys/openpgp-revocs.d' created
gpg: revocation certificate stored as '/var/local/mygpgkeys/openpgp-revocs.d/31208B5D45320FAE3D7E7FE21A2B04F0E994DAF4.rev'
public and secret key created and signed.

pub   rsa2048 2017-05-25 [SC] [expires: 2019-05-25]
        31208B5D45320FAE3D7E7FE21A2B04F0E994DAF4
        31208B5D45320FAE3D7E7FE21A2B04F0E994DAF4
uid                      Max Mustermann <maxmustermann@example.org>
sub   rsa2048 2017-05-25 [E] [expires: 2019-05-25]

GnuPG will generate a public and a secret key ring files named pubring.gpg and secring.gpg. (The file names may differ on macOS! Thanks Tim Cook.) You can get a glimpse on the available keys in your GPG key ring by running the following command:

$ gpg --list-keys

-------------------------------------------------------------------
pub   rsa2048 2017-05-25 [SC] [expires: 2019-05-25]
        31208B5D45320FAE3D7E7FE21A2B04F0E994DAF4
uid           [ultimate] Max Mustermann <maxmustermann@example.org>
sub   rsa2048 2017-05-25 [E] [expires: 2019-05-25]

We got our key pair, so let's board the synchronization train next. Oh, and by the way, you may now take your shades off.

Manage GnuPG Keys

I'll have to resort to bold measures for this one, so here goes - Do NOT lose your GPG key pair! I'm sure you've thought about that already, but take good care of the key you've just generated. You won't be able to decrypt your data, if you lose it!

A poor man solution could be to backup the keys on a dedicated flash drive and hide that under your pillow. If you're less of a paranoid, then you can put your keys in a certificate manager like Kleopatra [10]. Thunderbird users can just install and use the Enigmail [11] extension.

Setup Sync Folder

The first requirement is a Google Drive folder where you'll be uploading your encrypted files to. Notice that the unique id of the folder is given in the browser url.

Next, you need to initialize gdrive and configure a sync folder on the hard drive.

To create a sync folder run:

$ mkdir mysyncfolder && cd mysyncfolder

To initialize gdrive run the command below using the folder id you got from Google Drive. You'll need to manually copy and paste the authorization url in your browser and follow the steps to authorize gdrive to upload files on your behalf.

$ gdrive sync upload . 0BxNabiLkX8lpSGN3UVRmWEVQWWM -c /Users/max/.gdrive-test-sync

Authentication needed
Go to the following url in your browser:
https://accounts.google.com/o/oauth2/auth?access_type=offline&client_id=someverylongstringeg.apps.googleusercontent.com&redirect_uri=...

Enter verification code: 7/TWTDPBq32xYiRqqAlWp4tkcVBYgL3n5qsD_bfzXr8B0
Starting sync...
Collecting local and remote file information...
Found 0 local files and 0 remote files
Sync finished in 2.095311029s

Notice that the gdrive access token parameters are saved to a manually specified path, i.e., in /Users/max/.gdrive-test-sync. This should normally be a location that only your user is allowed to access. chmod 600 anyone?

Usage

Linux and macOS

Let's write a Bash script that will encrypt all files from a given folder on the hard drive and then upload them to a specified Google Drive folder. We'll call it sync.sh, because I'm out of original ideas.

$ touch sync.sh && chmod +x sync.sh

Open the file in an editor and add the following. Check the comments to adjust the source and destination paths where needed.

#!/bin/sh

GPG=$(which gpg)
GDRIVE=$(which gdrive)

# checks if the required tools are available 
if [ "XY$GPG" = "XY" ]; then
    echo "GPG not found!"
    exit 1
fi
if [ "XY$GDRIVE" = "XY" ]; then
    echo "gdrive not found!"
    exit 2
fi


####################################
## configuraitons
####################################

##### !IMPORTANT! #####
## Make sure to adjust the paths to your system
#######################

# key to encrypt with (the one we generated earlier)
GPG_KEY="maxmustermann@example.org"
export GNUPGHOME="/var/local/mygpgkeys"
# destination folder id in gdrive
GDRIVE_DEST="0BxNabiLkX8lpSGN3UVRmWEVQWWM"
# local sync folder path
GDRIVE_SYNC_DIR="/var/local/mysyncfolder"
# gdrive config
GDRIVE_CONFIG_PATH="/Users/max/.gdrive-test-sync"

####################################
## encrypt source files
####################################
echo

FOLDER="/<PATH-TO-MY-IMPORTANT-FILES-OR-BACKUPS>"

for f in $FOLDER/*; do
    echo "Encrypting $f ..."
    echo y | $GPG --recipient="$GPG_KEY" -e $f
    GPG_FILE="$f.gpg"

    BASE_NAME=$(basename $GPG_FILE)
    echo "Moving to $GDRIVE_SYNC_DIR/$BASE_NAME ..."
    mv $GPG_FILE $GDRIVE_SYNC_DIR/$BASE_NAME
done

## ...you may add more folders here

####################################
## sync everything
####################################

echo
echo "Sync files with Google Drive ..."

$GDRIVE sync upload $GDRIVE_SYNC_DIR $GDRIVE_DEST -c $GDRIVE_CONFIG_PATH

####################################
## cleanup
####################################

echo
if [ "XZ$GDRIVE_SYNC_DIR" != "XZ" ]; then
    echo "Removing encrypted files from $GDRIVE_SYNC_DIR ..."
    rm -f $GDRIVE_SYNC_DIR/*
fi

Now this is a very simple script that may be further extended. For example, traversing a folder tree or producing and encrypting a tar archive composed of many small sized files. I leave this up to you to adjust as needed.

Alright, let's run the sync.sh script.

$ ./sync.sh 

Encrypting LICENSE ...
Moving to /var/local/mysyncfolder/LICENSE.gpg ...
Encrypting Makefile ...
Moving to /var/local/mysyncfolder/Makefile.gpg ...
Encrypting MobileDevice.h ...
Moving to /var/local/mysyncfolder/MobileDevice.h.gpg ...
Encrypting README.md ...
Moving to /var/local/mysyncfolder/README.md.gpg ...

Sync files with Google Drive ...
Starting sync...
Collecting local and remote file information...
Found 4 local files and 4 remote files

4 local files has changed
[0001/0006] Updating LICENSE.gpg -> TestSync/LICENSE.gpg
[0002/0006] Updating Makefile.gpg -> TestSync/Makefile.gpg
[0003/0006] Updating MobileDevice.h.gpg -> TestSync/MobileDevice.h.gpg
[0004/0006] Updating README.md.gpg -> TestSync/README.md.gpg
Sync finished in 7.074881652s

Removing encrypted files from /var/local/mysyncfolder ...

If you now take a look in Google Drive, you'll find all the encrypted files placed in the target folder.

So what if you need this done on regular basis? Did I hear you say a cron job? Here's one that invokes the sync.sh script at 00:30 every day:

# m h  dom mon dow   command
30 0 * * * /var/local/sync.sh >> /var/log/sync.log

Windows

The sync workflow and script on Windows systems is pretty much the same. Of course, you would need the Windows version of GnuPG - Gpg4win and a Windows binary of the gdrive CLI.

@echo off
setlocal

REM ################################
REM ## configuraitons
REM ################################
SET RC=0
SET "CURRENT_DIR=%cd%"

SET "GDRIVE=C:\SYNC_TEST\gdrive-windows-x64.exe"

REM # key to encrypt with (the one we generated earlier)
SET "GPG_KEY=maxmustermann@example.org"
SET "GNUPGHOME=C:\SYNC_TEST\mygpgkeys"
REM # destination folder id in gdrive
SET "GDRIVE_DEST=0BxNabiLkX8lpSGN3UVRmWEVQWWM"
REM # local sync folder path
SET "GDRIVE_SYNC_DIR=C:\SYNC_TEST\mysyncfolder"
REM # gdrive config
SET "GDRIVE_CONFIG_PATH=C:\Users\<MY_USERNAME>\AppData\Local\gdrive-test"

REM ####################################
REM ## encrypt source files
REM ####################################

SET "FOLDER=C:\SYNC_TEST\sourcefiles"

PUSHD
CD %FOLDER%

for %%f in (*) do ( 
    echo Encrypting %%f ...
    gpg --recipient="%GPG_KEY%" -e %%f
    if errorlevel 1 (
        goto error_encrypt
    )

    echo Moving %%f.gpg to %GDRIVE_SYNC_DIR% ...
    move %%f.gpg %GDRIVE_SYNC_DIR%
)
POPD


REM ####################################
REM ## sync everything
REM ####################################

echo Sync files with Google Drive ...

PUSHD
CD %GDRIVE_SYNC_DIR%
%GDRIVE% sync upload %GDRIVE_SYNC_DIR% %GDRIVE_DEST% -c %GDRIVE_CONFIG_PATH%
POPD

REM ####################################
REM ## cleanup
REM ####################################
:cleanup

PUSHD
cd %GDRIVE_SYNC_DIR%
echo Removing encrypted files from %GDRIVE_SYNC_DIR% ...
del /F /Q *
POPD

goto end

:error_encrypt
echo Error encrypting file.
goto end

:end

Running the sync.cmd script would produce the following:

> sync.cmd
Encrypting LICENSE ...
Moving LICENSE.gpg to C:\SYNC_TEST\mysyncfolder ...
        1 file(s) moved.
Encrypting Makefile ...
Moving Makefile.gpg to C:\SYNC_TEST\mysyncfolder ...
        1 file(s) moved.
Encrypting MobileDevice.h ...
Moving MobileDevice.h.gpg to C:\SYNC_TEST\mysyncfolder ...
        1 file(s) moved.
Encrypting README.md ...
Moving README.md.gpg to C:\SYNC_TEST\mysyncfolder ...
        1 file(s) moved.
Sync files with Google Drive ...
Starting sync...
Collecting local and remote file information...
Found 4 local files and 6 remote files

4 local files has changed
[0001/0004] Updating LICENSE.gpg -> TestSync\LICENSE.gpg
[0002/0004] Updating Makefile.gpg -> TestSync\Makefile.gpg
[0003/0004] Updating MobileDevice.h.gpg -> TestSync\MobileDevice.h.gpg
[0004/0004] Updating README.md.gpg -> TestSync\README.md.gpg
Sync finished in 4.2520023s
Removing encrypted files from C:\SYNC_TEST\mysyncfolder ...

You can use the Windows Task Scheduler [12] to run the sync.cmd script at a desired time or interval.

Decrypt Files

So, the encrypt and sync workflow is ready and this is nice and all, but how does one get back their content in case they need it? Downloading the files from Google Drive is not an issue, but how about decrypting the content? Remember my all 90s mp3 file collection? Those are an important legacy I'd like my grandchildren to have.

Here is a simple bash script the decrypts all encrypted gpg files in a directory.

#!/bin/sh

if [ ! -d "$1" ]; then
    echo "No input folder specified!"
    exit 1
fi

GPG=$(which gpg)
if [ "XY$GPG" = "XY" ]; then
    echo "GPG not found!"
    exit 1
fi

FOLDER=$1

for f in $FOLDER/*.gpg; do
    echo "Decrypt $f ..."
    echo "<your-gpg-key-password>" | gpg --passphrase-fd 0 --batch --yes $f
done

Simply pass the directory path where all encrypted files reside and run the script to get all of them decrypted.

$ cd my-gdrive-downloaded-files && ./decrypt .

Decrypt ./LICENSE.gpg ...
gpg: encrypted with 2048-bit RSA key, ID E31A85E6729D5490, created 2017-05-25
    "Max Mustermann <maxmustermann@example.org>"
Decrypt ./Makefile.gpg ...
gpg: encrypted with 2048-bit RSA key, ID E31A85E6729D5490, created 2017-05-25
    "Max Mustermann <maxmustermann@example.org>"
Decrypt ./MobileDevice.h.gpg ...
gpg: encrypted with 2048-bit RSA key, ID E31A85E6729D5490, created 2017-05-25
    "Max Mustermann <maxmustermann@example.org>"
Decrypt ./README.md.gpg ...
gpg: encrypted with 2048-bit RSA key, ID E31A85E6729D5490, created 2017-05-25
    "Max Mustermann <maxmustermann@example.org>"

So, that's it. You got all your data encrypted. Sundar Pichai will never smile again.

Conclusion

In the end, what's all this good for? (I probably should have put that question at the beginning of the article.)

I personally use it to keep server backups safely stored in the cloud. Mostly binary files, PDFs, archives, etc. It probably doesn't make sense to store frequently changing documents like text or Word files using this method, although there's nothing stopping you.

Annex - Metadata

One topic that I did not consider when I was writing this article is Metadata [13]. You may have noticed that the original filenames in the scripts above are always preserved when uploaded to Google Drive. The name and size of a file may be enough information for complex algorithms to still extract quite a lot of meaningful info about what the purpose and contents of that file may be in your user context. A simple counter mechanism could be used to produce a hash value of each filename, thus at least preventing filename metadata extraction.

Here is a modification of the encryption script for Linux/macOS that hashes the filenames using arbitrary salt value and produces a CSV file index of all encrypted files.

# a registry of hashed files
INDEX="/var/local/gdrive_index.csv"
SALT="someveryveryveryveryveryveryverylongstring"

## reset index contents
echo "FILENAME;HASHED NAME" > $INDEX

####################################
## Encrypt source files
####################################
echo

FOLDER="/<PATH-TO-MY-IMPORTANT-FILES-OR-BACKUPS>"

for f in $FOLDER/*; do
    echo "Encrypting $f ..."
    echo y | $GPG --recipient="$GPG_KEY" -e $f

    BASE_NAME=$(basename $f)
    HASHED=$(echo $BASE_NAME.$SALT | openssl dgst -sha256)
    echo "$BASE_NAME;$HASHED" >> $INDEX

    GPG_FILE="$f.gpg"
    HASHED_GPG_FILE="$HASHED.gpg"

    echo "Moving to $GDRIVE_SYNC_DIR/$HASHED_GPG_FILE ..."
    mv $GPG_FILE $GDRIVE_SYNC_DIR/$HASHED_GPG_FILE
done