DEV Community

Antonio Sánchez
Antonio Sánchez

Posted on • Originally published at asanchez.dev

Upload files to Amazon S3 with Golang

Connecting your Go application with Amazon S3 is pretty easy and requires only a few lines of code. But if you have no experience with Amazon Web Services, it can get pretty tricky at the beginning to figure out even the most straightforward task.

We are going to see here how to connect to S3 with Golang, upload a file from a form to an S3 bucket, download it, and list all items saved on this bucket.

I am going to suppose that you have already:

  • created an AWS account
  • created an Amazon Simple Storage (S3) Bucket
  • generated the credentials to access it (Access key ID and Secret access key)

I am not going to explain those things here since there are already plenty of good tutorials on the Internet about it.

Set up your credentials

After having created the S3 Bucket, the very first thing that we are going to do is to save the credentials.

There are a few ways of doing this as described here https://docs.aws.amazon.com/es_es/sdk-for-go/v1/developer-guide/configuring-sdk.html

We are going to use the easiest one using a shared credential file. For this, create a file like described below on ~/.aws/credentials on Linux, macOS or Unix. If using windows, create it on C:\Users\USERNAME\.aws\credential.

[default] 
aws_access_key_id = your_access_key_id 
aws_secret_access_key = your_secret_access_key
Enter fullscreen mode Exit fullscreen mode

Install the SDK

After this, we need to install de AWS SDK for Go:

go get -u github.com/aws/aws-sdk-go/...
Enter fullscreen mode Exit fullscreen mode

and import the AWS packages into your Go application:

import ( 
    "github.com/aws/aws-sdk-go/aws" 
    "github.com/aws/aws-sdk-go/aws/session" 
    "github.com/aws/aws-sdk-go/service/s3"
)
Enter fullscreen mode Exit fullscreen mode

The Code

You can skip all explanations below and check the code directly on https://github.com/antsanchez/goS3example.

I have tried to keep the code as minimal as possible.

Connect to AWS S3

I like to use this code on my main.go file or whatever other file where I can share the session (sess variable) to reuse it.

const (
    AWS_S3_REGION = ""
    AWS_S3_BUCKET = ""
)

var sess = connectAWS()

func connectAWS() *session.Session {
    sess, err := session.NewSession(
            &aws.Config{
                Region: aws.String(AWS_S3_REGION)
            }
        )
    if err != nil {
        panic(err)
    }
    return sess
}
Enter fullscreen mode Exit fullscreen mode

Upload a file from a form to Amazon S3

First, get the file from the form:

file, header, err := r.FormFile("file")
if err != nil {
    // Do your error handling here
    return
}
defer file.Close()

filename := header.Filename
Enter fullscreen mode Exit fullscreen mode

Then, upload it to S3:

uploader := s3manager.NewUploader(sess)

_, err = uploader.Upload(&s3manager.UploadInput{
    Bucket: aws.String(AWS_S3_BUCKET), // Bucket to be used
    Key:    aws.String(filename),      // Name of the file to be saved
    Body:   file,                      // File
})
if err != nil {
    // Do your error handling here
    return
}
Enter fullscreen mode Exit fullscreen mode

Download a file from Amazon S3

First, create a temporary file where to store the downloaded file:

f, err := os.Create(filename)
if err != nil {
    // Do your error handling here
    return
}
Enter fullscreen mode Exit fullscreen mode

Now, download the file and do whatever you want with it (f)

downloader := s3manager.NewDownloader(sess)
_, err = downloader.Download(f, &s3.GetObjectInput{
    Bucket: aws.String(AWS_S3_BUCKET),
    Key:    aws.String(filename),
})
if err != nil {
    // Do your error handling here
    return
}
Enter fullscreen mode Exit fullscreen mode

List the items of an AWS S3 Bucket

svc := s3.New(sess)
input := &s3.ListObjectsInput{
    Bucket: aws.String(AWS_S3_BUCKET),
}

result, err := svc.ListObjects(input)
if err != nil {
    if aerr, ok := err.(awserr.Error); ok {
        switch aerr.Code() {
        case s3.ErrCodeNoSuchBucket:
            fmt.Println(s3.ErrCodeNoSuchBucket, aerr.Error())
        default:
            fmt.Println(aerr.Error())
        }
    } else {
        // Print the error, cast err to awserr.Error to get the Code and
        // Message from an error.
        fmt.Println(err.Error())
    }
    // Do your error handling here
    return
}
Enter fullscreen mode Exit fullscreen mode

You could now list the items showing the name like this:

w.Header().Set("Content-Type", "text/html")

for _, item := range result.Contents {
    fmt.Fprintf(w, "<li>File %s</li>", *item.Key)
}
Enter fullscreen mode Exit fullscreen mode

More information

There are plenty of tutorials on internet, but not all of them are updated or following the best practices.
I would recommend you to check those links:

And that's all.

Discussion (0)