Golang Data Connector Part 1

Intro

There are a handful of tools out there like Zapier (which just hit a $5 billion valuation) which connect disparate systems together. It's an interesting idea but, how does one actually create a connector (or integration) without Zapier? In this series of posts, we'll step through how I might go about building something.

Getting Spooky

Let's pretend that we want to move data from a Ghost blog over to another service. That sounds simple enough, for this post we're going to not worry about a continued connection we're just going to get our content from Point A. We'll worry about Point B next time.

OK, that's well and good but where do we start? The best place to start is the Ghost content API documentation at https://ghost.org/docs/content-api/. This will give us all the information we need to start planning. There is even a live content/posts endpoint we can call at https://demo.ghost.io/ghost/api/v4/content/posts/?key=22444f78447824223cefc48062, which will show us what we can expect to receive back.

Go Get It

Phase one of this simple connector is relatively easy thanks to Go's http package. We use it to reach out and request the first page of post content. We'll start by creating a new request. The full code listing is at the bottom of this post.

    req, err := http.NewRequest(http.MethodGet, "https://demo.ghost.io/ghost/api/v4/content/posts/?key=22444f78447824223cefc48062", nil)

http.NewRequest takes in the method of the HTTP request, in this case, it is a GET, the URL of the request, and a body. This time we don't have a body so we'll just use nil. OK, great we have our request all set up but how do we use it? Again we're back to http, we can use http.DefaultClient method Do() to actually make our request. We simply pass in our request and capture the response in a new variable.

    resp, err := http.DefaultClient.Do(req)

And that's it! We have our data, well as long as the call was OK that is. We aren't really doing anything with errors instead we'll just bail out. I imagine that we can have some sort of retry instead of just exiting.

From here we'll log the response body to the console.

    body, err := io.ReadAll(resp.Body)
    if err != nil {
        log.Fatal(err)
    }

    fmt.Println(string(body))

Next Time

In our next post, we'll modify our project to allow us to actually do something with the data we have received

Code Listing

package main

import (
    "encoding/json"
    "fmt"
    "io"
    "log"
    "net/http"
)

func main() {
    req, err := http.NewRequest(http.MethodGet, "https://demo.ghost.io/ghost/api/v4/content/posts/?key=22444f78447824223cefc48062", nil)
    if err != nil {
        log.Fatal(err)
    }

    resp, err := http.DefaultClient.Do(req)
    if err != nil {
        log.Fatal(err)
    }

    defer resp.Body.Close()

    body, err := io.ReadAll(resp.Body)
    if err != nil {
        log.Fatal(err)
    }

    fmt.Println(string(body))
}