I'm building a trivial Go app that requests data from multiple API endpoints and renders them in a template. The requests are completely independent from each other: neither request relies on data from the other.
I've written a lot of JavaScript over the past 7 years, and JavaScript's promises and async/await make concurrency trivial. I wanted to see if I could use Go's concurrency primitives to implement JS-inspired async/await patterns in Go.
Note: This post requires some familiarity with Go's concurrency primitives, and this isn't a recommendation for how to write concurrent Go code.
JavaScript concurrency
Here's an example of concurrent requests using promises and async/await:
// Async/await-based concurrency:
async function makeRequests() {
let postsResponse, tagsResponse;
try {
const postsPromise = fetch("https://example.com/api/posts");
const tagsPromise = fetch("https://example.com/api/tags");
postsResponse = await postsPromise;
tagsResponse = await tagsPromise;
} catch (e) {
console.error("At least one request failed", e);
}
// Do more stuff...
}
Here's what it looks like to make concurrent requests using promise.all():
// promise.all()-based concurrency:
Promise.all([
fetch("https://example.com/api/posts"),
fetch("https://example.com/api/tags"),
]).then(([postsResponse, tagsResponse]) => {
// Do more stuff...
}).catch((error) => {
console.error("At least one request failed", error);
});
Both of these methods appeal to me because they're simple and require very little code.
Implementing awaitable HTTP requests in Go
My attempt at implementing a promise & async/await-like API called async.Get() for HTTP requests in Go is below.
// main.go
package main
import (
"app/async"
)
func main() {
postPromise := async.Get("https://example.com/api/posts")
tagPromise := async.Get("https://example.com/api/tags")
postRes, postErr := postPromise.AwaitResponse()
tagRes, tagErr := tagPromise.AwaitResponse()
// Do more stuff...
}
The following code powers this API.
// async/get.go
package async
import (
"net/http"
"sync"
)
type response struct {
res *http.Response
err error
}
type ResponsePromise struct {
channel chan *response
result *response
once *sync.Once
}
func (r *ResponsePromise) AwaitResponse() (resp *http.Response, err error) {
r.once.Do(func() {
r.result = <-r.channel
})
return r.result.res, r.result.err
}
func Get(url string) ResponsePromise {
c := make(chan *response)
go func(url string) {
resp, err := http.Get(url)
c <- &response{resp, err}
}(url)
return newResponsePromise(c)
}
func newResponsePromise(c chan *response) ResponsePromise {
return ResponsePromise{
channel: c,
once: &sync.Once{},
}
}
How it works
Here's what happens when you call async.Get(url):
- A channel is created within
Get()to store the result of the http request. - A goroutine asynchronously fires off an
http.Get()request to the URL provided as an argument toGet(). Its return values will be added to arequeststruct when the request completes, and these will be sent the the channel. - A
ResponsePromisestruct with a reference to the channel is returned to the caller ofGet(). -
The first time that the
AwaitResponse()method is called on theResponsePromisestruct, all other consumers of theAwaitResponse()method will be blocked bysync.Once.Dountil the channel that was created withinGet()resolves aresultstruct. Theresultwill be cached on theResultPromise.Caching the
resulton theResponsePromiseis wrapped insync.Onceto ensure that unsafe concurrent writes to theResultPromiseare impossible, and therefore won't crash the app. Once the cached result is stored on the
ResultPromisestruct, it will return both the response and error from the previously calledhttp.Get()request.
This implementation has a little bit of indirection, but it has some nice properties. Because the channel isn't returned directly by async.Get(), it's impossible for Go to block permanently by waiting for another channel value if the consumer so happens to call <- returnedChannel twice. Also, calling AwaitResponse() multiple times is idempotent after the first call: the result is cached so no further requests will be made.
One noteworthy part of the implementation is the intermediate response struct. This is necessary because Go does not have a tuple type, so either multiple channels or an intermediate struct were necessary to return both the response and error from http.Get().
Should you do this? Maybe not.
I really like my implementation: I think that the code reads well. But the code hides inefficiency and rigidness.
My code allocates more memory than is strictly necessary. Behind the scenes, each call to the async.Get() function allocates a channel and 3 structs (ResponsePromise, response and sync.Once) that the caller doesn't care about: the caller only wants the response and error from each request.
Compare my awaitable HTTP request to a WaitGroup-based implementation:
// Concurrent requests using a WaitGroup:
package main
import (
"net/http"
"sync"
)
func main() {
var pRes, tRes *http.Response
var pErr, tErr error
wg := sync.WaitGroup{}
wg.Add(2)
go func() {
defer wg.Done()
pRes, pErr = http.Get("https://example.blog/api/posts")
}()
go func() {
defer wg.Done()
tRes, tErr = http.Get("https://example.blog/api/tags")
}()
wg.Wait()
// Do more stuff...
}
Compared to my solution, this allocates 2 fewer channels and 6 fewer structs to the heap. It's slightly more verbose, but much more flexible, allowing other concurrent processes to happen that might not be HTTP requests.
If you had a database call that you wanted to make concurrently with the awaitable HTTP requests above, you'd need to wrap that call in a similar awaitable structure, or you'd need to do something hacky to make it play nice with the awaitable HTTP requests.
If you instead use a WaitGroup for the HTTP requests, it's trivial to add a concurrent database call using that same WaitGroup:
// Add concurrent database call with imaginary
// `app/db` and `app/models` packages.
func main() {
var pRes, tRes *http.Response
var pErr, tErr error
// NEW!
var q []models.Quote
var qErr error
wg := sync.WaitGroup{}
wg.Add(3) // CHANGED!
// NEW!
go func() {
defer wg.Done()
q, qErr = db.GetInspirationalQuotes()
}()
go func() {
defer wg.Done()
pRes, pErr = http.Get("https://example.blog/api/posts")
}()
go func() {
defer wg.Done()
tRes, tErr = http.Get("https://example.blog/api/tags")
}()
wg.Wait()
// Do more stuff...
}
Closing thoughts
This was fun to play with, and I learned a lot about Go's concurrency primitives in the process. I should mention that there are other Gophers who have played with creating general-purpose libraries for promises (here and here). The Go community's reaction towards these general purpose promise libraries hasn't been particularly positive, with my favorite unhelpful reaction being "Please, God… No!!"
My biggest criticism of general purpose promise libraries in Go is that they really don't reduce that much boilerplate code compared to WaitGroups. It's always fun to experiment though, and with any luck you may have learned something from this post.
If you enjoyed this article (or hated it!), please hit the like button and leave a comment below. Thanks!
Top comments (3)
pls don't color my grey Go functions😭😭😭
I had to look around and read some articles to understand this.
haha yeah, glad you learnt something