Let's say storage is a needed part of our application, but our hosting has limited storage size, and we really need a few gigs of storage as cheap as it is possible. So lets create a service which could access and work with Google drive.
Prerequisites
- enabled Google drive API in Google Cloud console
- registered a project
- create an OAuth 2.0 client with Google drive api permissions
- install the Google drive v3 nuget package https://www.nuget.org/packages/Google.Apis.Drive.v3/
Lets start with creating an empty class
public class DriveApiService
{
}
Auth
We have 2 options to get access to a drive storage
- service account
- user account
If you don't want any user interaction and authorization use a service account, the downsides of this account is, that you can't access the google drive UI for this account, and you cant buy additional storage for this account.
Our second option is to use our personal account, but it will need one time manual login by the user.
We will use the second option.
So in the constructor of the service we will handle the authorization with the file client_id.json which is generated for the Google OAuth client.
public class DriveApiService
{
protected static string[] scopes = { DriveService.Scope.Drive };
protected readonly UserCredential credential;
static string ApplicationName = "MyApplicationName";
protected readonly DriveService service;
protected readonly FileExtensionContentTypeProvider fileExtensionProvider;
public DriveApiService()
{
using (var stream =
new FileStream("client_id.json", FileMode.Open, FileAccess.Read))
{
string credPath = "token.json";
credential = GoogleWebAuthorizationBroker.AuthorizeAsync(
GoogleClientSecrets.Load(stream).Secrets,
scopes,
YOUR_GMAIL_EMAIL, // use a const or read it from a config file
CancellationToken.None,
new FileDataStore(credPath, true)).Result;
fileExtensionProvider = new FileExtensionContentTypeProvider();
}
service = new DriveService(new BaseClientService.Initializer()
{
HttpClientInitializer = credential,
ApplicationName = ApplicationName,
});
}
}
Let's try it out.
Let's use the dependency injection and register our service as a singleton in our application.
services.AddSingleton<DriveApiService>();
After you start the application, a browser window will open and requires you to login to your google account, if the login went well, it creates a persistent file token.json in your application which will contain the access and refresh token to access the Google drive API. You won't need to login until the generated file is in your application, and your refresh token is valid, which has no time limit, but if you change you Google account password it gets revoked and you have to log in manually again.
Listing files
Google drive has no directory structure (you can't use a path for navigation "root/parent/child.type"), it uses parent child relations, one child can be in multiple parents. In our application we store these relations in a DB and emulate a classic directory behavior with all it restrictions and we actually using paths, but this article won't cover that part.
To get files in a directory we need to know its id. We can access the root directory with the root keyword.
public IList<Google.Apis.Drive.v3.Data.File> ListEntities(string id = "root")
{
FilesResource.ListRequest listRequest = service.Files.List();
listRequest.PageSize = 100;
listRequest.Fields = "nextPageToken, files(id, name, parents, createdTime, modifiedTime, mimeType)";
listRequest.Q = $"'{id}' in parents";
return listRequest.Execute().Files;
}
The ListEntities method returns a list of files, with all the defined fields, which are in the directory with the given id. If no id is set we are using the default root keyword. A new folder needs to have a name, mime type and defined parent. Be careful Google drive allows multiple files/folders with the same name, google drive doesn't care. (we handle this in the database layer of the app, so we care :))
Creating folders
To group files we are using folders. The creation of folders is simple.
public Google.Apis.Drive.v3.Data.File CreateFolder(string name,string id = "root")
{
var fileMetadata = new Google.Apis.Drive.v3.Data.File()
{
Name = name,
MimeType = "application/vnd.google-apps.folder",
Parents = new[] { id }
};
var request = service.Files.Create(fileMetadata);
request.Fields = "id, name, parents, createdTime, modifiedTime, mimeType";
return request.Execute();
}
If the id of the parent is given we create a folder in the defined parent, if not, then we create the folder in the root. The response will contain all the defined fields.
File upload
File upload is similar to a folder creation.
public async Task<Google.Apis.Drive.v3.Data.File> Upload(IFormFile file, string documentId)
{
var name = ($"{DateTime.UtcNow.ToString()}.{Path.GetExtension(file.FileName)}");
var mimeType = file.ContentType;
var fileMetadata = new Google.Apis.Drive.v3.Data.File()
{
Name = name,
MimeType = mimeType,
Parents = new[] { documentId }
};
FilesResource.CreateMediaUpload request;
using (var stream = file.OpenReadStream())
{
request = service.Files.Create(
fileMetadata, stream, mimeType);
request.Fields = "id, name, parents, createdTime, modifiedTime, mimeType, thumbnailLink";
await request.UploadAsync();
}
return request.ResponseBody;
}
First we generate a new unique filename for our file with the right extension, we extract the right mime type from our file and we fill our file metadata. Then we create an upload request and we upload our stream. The response will contain all the defined fields.
Rename files/folders
In our application we can only edit the file or folder name, but following this sample you are able to change anything in the file entity.
public void Rename(string name, string id)
{
Google.Apis.Drive.v3.Data.File file = service.Files.Get(id).Execute();
var update = new Google.Apis.Drive.v3.Data.File();
update.Name = name;
service.Files.Update(update, id).Execute();
}
First we get our file by the given id, then we create an update request, with the metadata we want to change. If you want you could implement some error handling because if the file with the given id does not exist, then it throws an error (we have the error handling in the upper layer).
Removing File/Folder
Removing entities is straight forward.
public void Remove(string id)
{
service.Files.Delete(id).Execute();
}
File download
We can implement a file downloading which downloads a file by its id, we store its content in a memory stream we set its position to 0 so we are able to work with it in the upper layers and then we return the stream.
public async Task<Stream> Download(string fileId)
{
Stream outputstream = new MemoryStream();
var request = service.Files.Get(fileId);
await request.DownloadAsync(outputstream);
outputstream.Position = 0;
return outputstream;
}
BONUS Zip multi download
First we implement a DTO model which describes our file.
public class ZipVM
{
public string Name { get; set; }
public string DriveId { get; set; }
}
We are storing custom file names in our database, in drive we are using date based file names, to have the right filenames in the generated zip file wee need to provide the right filenames to our method, if you don't need that, you could use a simple list of ids or you could reuse the Download method.
Handling a zip item download is similar to a single file upload, but this method is working with our DTO model and returns a tuple consisted of a stream and filename. Why we need this? Actually the Google drive API doesn't provide anything to download multiple files at once.
public async Task<(Stream,string)> DownloadZipItem(ZipVM vm)
{
Stream outputstream = new MemoryStream();
var request = service.Files.Get(vm.DriveId);
await request.DownloadAsync(outputstream);
outputstream.Position = 0;
return (outputstream, vm.Name);
}
The Zip method accepts a list of the DTOs we defined. Then we create a list of DownloadZipItem task from the provided DTOs which we call in parallelly. Then we create an archive and return it as a byte array.
public async Task<byte[]> Zip(List<ZipVM> documents)
{
List<Task<(Stream,string)>> downloadTasks = new List<Task<(Stream,string)>>();
for(int i = 0; i<documents.Count; i++)
{
downloadTasks.Add(DownloadZipItem(documents[i]));
}
await Task.WhenAll(downloadTasks);
List<(Stream, string)> files = downloadTasks.Select(t => t.Result).ToList();
byte[] archiveFile;
using (var archiveStream = new MemoryStream())
{
using (var archive = new ZipArchive(archiveStream, ZipArchiveMode.Create, true))
{
int i = 0;
foreach (var file in files)
{
var zipArchiveEntry = archive.CreateEntry(file.Item2, CompressionLevel.Fastest);
using (var zipStream = zipArchiveEntry.Open())
file.Item1.CopyTo(zipStream);
}
}
archiveFile = archiveStream.ToArray();
}
return archiveFile;
}
Be careful, to provide the file extension inside the filename for the entry creation inside the zipping method.
English isn’t my first language, so please excuse any mistakes.
Top comments (5)
Your article is super interesting, you have explained it very clearly, and I think it is very useful for file management.
Still I always think that manipulating API key, or as in this case google credentials are insecure, the article is perfect, but it would be nice to talk about how to make those credentials secure, because in this current form, anyone can get them and use your drive account.
Still a very good article, congratulations 🥳!
Thank you, I think you are right, but the package has no other option to load the credentials, or I didn't find any other way. But for example, we can extend and override the FileDataStore class (github.com/googleapis/google-api-d...) where we can implement some custom logic to encrypt/decrypt our file, but that's just an idea. I have to be honest I am more a frontend guy than a backend guy and I think security is one of the topics where I need to improve.
please how do i get token.json and config.json
Config is created by you, token json will be created after you launch the app the first time, it then opens the browser and requires your login to gdirve, after that the token.json file is created and stored automatically
hi..i am getting the following error when i try to download google document file.
Only files with binary content can be downloaded. Use Export with Google Docs files
any advice on that ?