Building a File Upload Service in Go

#aws #go #backend #beginners

Let’s talk file uploads. Whether you’re building the next Instagram, a CMS, or any app that involves user-generated content, you’re going to need to handle files. Today, we’re diving into the world of file uploads with Go. We’ll set up a simple file upload service that can store files locally and, for a little extra flavor, we’ll even connect it to Amazon S3 so you can go full cloud mode. 🌥️

Here’s the game plan:

Setting up a simple file upload endpoint.
Handling files, storing them locally, and making sure everything works.
Adding some basic validation to keep things secure.
And then, we’re taking it up a notch with S3 storage.

Grab your coffee, and let’s go! ☕

Step 1: Building the File Upload Endpoint

First things first—let’s set up a basic HTTP server with a /upload endpoint. For this, we’re sticking to Go’s built-in net/http package because it’s straightforward and easy to use.

Server Setup

Pop open your favorite editor, create a main.go file, and set up a basic server:

package main

import (
    "fmt"
    "log"
    "net/http"
)

func main() {
    http.HandleFunc("/upload", fileUploadHandler)

    fmt.Println("Server running on :8080")
    log.Fatal(http.ListenAndServe(":8080", nil))
}

Creating the Upload Handler

Now let’s get to the fun part: handling file uploads! We’ll create a fileUploadHandler function that’s going to handle incoming files and store them in a local directory.

func fileUploadHandler(w http.ResponseWriter, r *http.Request) {
    // Limit file size to 10MB. This line saves you from those accidental 100MB uploads!
    r.ParseMultipartForm(10 << 20)

    // Retrieve the file from form data
    file, handler, err := r.FormFile("myFile")
    if err != nil {
        http.Error(w, "Error retrieving the file", http.StatusBadRequest)
        return
    }
    defer file.Close()

    fmt.Fprintf(w, "Uploaded File: %s\n", handler.Filename)
    fmt.Fprintf(w, "File Size: %d\n", handler.Size)
    fmt.Fprintf(w, "MIME Header: %v\n", handler.Header)

    // Now let’s save it locally
    dst, err := createFile(handler.Filename)
    if err != nil {
        http.Error(w, "Error saving the file", http.StatusInternalServerError)
        return
    }
    defer dst.Close()

    // Copy the uploaded file to the destination file
    if _, err := dst.ReadFrom(file); err != nil {
        http.Error(w, "Error saving the file", http.StatusInternalServerError)
    }
}

Here’s the lowdown:

We’re grabbing the file from the form with r.FormFile("myFile").
After we get the file, we’re opening (or creating) a local file and copying the contents over.
This setup is great for local storage, quick prototypes, or projects that aren’t ready for cloud storage yet.

Saving the File Locally

Let’s create the helper function createFile that handles where our files go:

import (
    "os"
    "path/filepath"
)

func createFile(filename string) (*os.File, error) {
    // Create an uploads directory if it doesn’t exist
    if _, err := os.Stat("uploads"); os.IsNotExist(err) {
        os.Mkdir("uploads", 0755)
    }

    // Build the file path and create it
    dst, err := os.Create(filepath.Join("uploads", filename))
    if err != nil {
        return nil, err
    }

    return dst, nil
}

Step 2: Validating and Securing Your Files 🛡️

Security is key! Let’s add a little validation so only approved file types make it through.

Validating the MIME Type

Want to keep it safe? Let’s restrict uploads to images. Here’s how:

import (
    "io/ioutil"
    "strings"
)

func isValidFileType(file []byte) bool {
    fileType := http.DetectContentType(file)
    return strings.HasPrefix(fileType, "image/") // Only allow images
}

func fileUploadHandler(w http.ResponseWriter, r *http.Request) {
    // [Existing code here]

    // Read the file into a byte slice to validate its type
    fileBytes, err := ioutil.ReadAll(file)
    if err != nil {
        http.Error(w, "Invalid file", http.StatusBadRequest)
        return
    }

    if !isValidFileType(fileBytes) {
        http.Error(w, "Invalid file type", http.StatusUnsupportedMediaType)
        return
    }

    // Proceed with saving the file
    if _, err := dst.Write(fileBytes); err != nil {
        http.Error(w, "Error saving the file", http.StatusInternalServerError)
    }
}

Step 3: Taking It to the Cloud with S3 ☁️

Local storage is fine and all, but if you want to scale, S3 is where it’s at! Let’s connect your file upload service to Amazon S3 so you can store files in the cloud.

Install the AWS SDK

To work with S3, you’ll need the AWS SDK:

go get -u github.com/aws/aws-sdk-go/aws
go get -u github.com/aws/aws-sdk-go/service/s3

Configure the S3 Client

Let’s set up a function to connect to your S3 bucket:

import (
    "bytes"
    "github.com/aws/aws-sdk-go/aws"
    "github.com/aws/aws-sdk-go/aws/session"
    "github.com/aws/aws-sdk-go/service/s3"
)

func uploadToS3(file []byte, filename string) error {
    sess, err := session.NewSession(&aws.Config{
        Region: aws.String("us-west-1"), // Your AWS region
    })
    if err != nil {
        return err
    }

    s3Client := s3.New(sess)
    _, err = s3Client.PutObject(&s3.PutObjectInput{
        Bucket: aws.String("your-bucket-name"),
        Key:    aws.String(filename),
        Body:   bytes.NewReader(file),
        ACL:    aws.String("public-read"),
    })
    return err
}

Replace "your-bucket-name" with your actual S3 bucket name. Now, let’s tweak our upload handler to use this function.

Modify the Upload Handler

Update fileUploadHandler so we store the file in S3 instead of locally:

func fileUploadHandler(w http.ResponseWriter, r *http.Request) {
    // [Existing code here]

    if err := uploadToS3(fileBytes, handler.Filename); err != nil {
        http.Error(w, "Error uploading to S3", http.StatusInternalServerError)
        return
    }
    fmt.Fprintf(w, "File successfully uploaded to S3!")
}

And that’s it! You’ve now got a file upload service that supports both local storage and cloud storage via Amazon S3. 🔥

Testing It Out 🧪

To test the upload service, you can use curl:

curl -X POST http://localhost:8080/upload -F "myFile=@path/to/your/file.jpg"

Or, if you prefer a graphical interface, create a quick HTML form:

<form enctype="multipart/form-data" action="http://localhost:8080/upload" method="POST">
  <input type="file" name="myFile">
  <input type="submit" value="Upload">
</form>

Upload a file, and you should see it saved locally or in your S3 bucket.

Wrapping Up

Building a file upload service is a great way to add functionality and learn about file handling, validation, and even cloud storage. Now that you’ve got the basics down, think about what’s next—whether it’s image resizing, video processing, or handling larger file types, the sky’s the limit!

Have you built a file upload service before? Drop a comment below with your tips, or let me know what you’d like to see next. Happy coding!