Go concurrency in the smoothiest way

1. Here is my problem

I have to admit here: concurrent programming is one of the biggest headaches I'm having as I learn Computer Science. But since I started studying Go, I realized I have to conquer this challenge to really understand the language. So, I'm writing this article to explain it to myself - or someone else who might be interested in this topic - in a way even a 5 year-old kid could understand (I might be exaggerating).

2. First things first

Every article about concurrent programming begins explaining the difference between concurrency and parallelism, but we don’t do that here, so forget about this. Let’s start, instead, with a naive Hello, World! program in Go:

func main() {
    var message string
    func() {
        message = "Hello, World!"
    }()
    fmt.Println("Output:", message)
}

Output: Hello, World!

In this code, I deliberately used an anonymous function to assign the string 'Hello, World!' to the message variable. Then, I printed the message to the screen. The key point here is that this program runs in a single goroutine, which means the code executes in one line of execution. Therefore, we can assume that each line of code executes sequentially, one after the other (ignoring low-level details).

Goroutines are the lines of execution of a Go application (something like threads but in a higher level of abstraction). Each Go program has a main goroutine that acts like the starting point and controls how long the program runs, but, if we want to have more goroutines besides the main one, we have to create them manually. Having more goroutines might allow us to explore the multiple cores of modern CPUs and improve the performance of our application, but since we add more lines of execution to our code strange things start happening:

func main() {  // main goroutine starts here
    var message string
    go func() {  // new goroutine starts here
        message = "Hello, World!"
    }()
    fmt.Println("Output:"message)
}

Output:

The only change made in the code above compared to the previous example is the addition of the go keyword before the anonymous function (the go keyword is always used before function calls to launch them as goroutines). This creates a new goroutine containing only the anonymous function, which executes concurrently with the main goroutine. However, as you observed, the 'Hello, World!' message is missing from the output. This occurs because the main goroutine doesn't wait for the new goroutine to complete its task (assigning a value to the message variable) and finishes execution first. This behavior becomes even more evident if we force the main routine to wait an additional second before the Println execution.:

func main() {
    var message string
    go func() {
        message = "Hello, World!"
    }()
    time.Sleep(time.Second) // wait one second before print
    fmt.Println("Output:", message)
}

Output: Hello, World!

But adding a time.Sleep to our main gorutine is not the right way of get the things done, instead, we can use channels, that are the default way of transporting messages between lines of execution in Go.

3. Channels

Our goal here is to ensure the main goroutine waits for the newly created goroutine to complete its task before executing the Println function. This will allow us to see the 'Hello, World!' message displayed on the screen without resorting to the jury-rigged time.Sleep approach. One way to achieve this is by creating a channel that facilitates communication between the two goroutines by transferring the string from one to the other.

func main() {
    channel := make(chan string)
    go func() {
        channel <- "Hello, World!"
        close(channel)
    }()
    fmt.Println("Output:", <-channel)
}

Output: Hello, World!

In this code we use function make to create a channel (the name of the channel here is channel, but it could be any name) that can send and receive strings.

channel := make(chan string)

Within the newly created goroutine, we send the 'Hello, World!' message through the channel (the arrow indicates the direction of data flow). We then close the channel, signifying that we've finished sending data. This informs the receiver that no further information will be transmitted, allowing it to stop waiting for additional messages.

channel <- "Hello, World!"
close(channel)

Now in the main goroutine, in the Println argument, we receive the string that have been sent from the new goroutine.

fmt.Println("Output:", <-channel)

This code functions correctly because the main goroutine is instructed to wait for data from the channel before proceeding with the Println execution. So note that channels have the power to pause the goroutine execution while waiting for send or receive data. In this way, the channel acts as a synchronization mechanism within the asynchronous context, ensuring the message is received in the right time.

3.1 Buffered channels

The Go channels by default don’t have any memory capacity, which means they only send and receive data, but don’t store them. Yet sometimes we want to hold the data in order to control the data flow among goroutines. To achieve this we can use buffers.

func main() {
    channel := make(chan string)

    channel <- "Output:"
    channel <- "Hello, World"

    fmt.Println(<-channel, <-channel)

}

fatal error: all goroutines are asleep - deadlock!

When we run the code above, we receive a deadlock error. This occurs because when the channel receives the "Output:" string, execution stops. Then the channel waits to send this data somewhere else (remember, channels lack internal storage and cannot hold data), but, since that are no goroutines in the moment waiting to receive it, the wait is in vain. As a result, the line that should receive the "Hello, World!" string never executes, and the application panics. To solve this, we simply need to add buffers to the channel, allowing it to temporarily store data before sending it.

func main() {
    channel := make(chan string, 2) // the second argument here is the capacity

    channel <- "Output:"
    channel <- "Hello, World"

    fmt.Println(<-channel, <-channel)

}

Output: Hello, World!

To create buffered channels, we simply specify the desired buffer size as the second argument to the make function. In this example, using the number 2 allows the channel to store two strings, preventing the program's normal execution flow from being blocked. Buffers provide a mechanism to control the maximum amount of data that can be queued, which is particularly useful in scenarios like web servers handling high volumes of requests.

3.2 Select statement

As mentioned earlier, channels can pause execution while waiting to send or receive data. In some cases, particularly when working with multiple channels, this behavior might not be desirable. You might not want one channel to block the execution related to another. To address this, Go provides the select statement, a construct that allows channels to work together without one blocking the other. Before diving into the select statement, let's examine an example of code without it.

func main() {
    oneSecond := make(chan string)
    fiveSeconds := make(chan string)

    go func() {
        for i := 0; i < 20; i++ {
            time.Sleep(time.Second)
            oneSecond <- "One second"
        }
    }()

    go func() {
        for i := 0; i < 20; i++ {
            time.Sleep(time.Second * 5)
            fiveSeconds <- "Five seconds"
        }
    }()

    for i := 0; i < 20; i++ {
        fmt.Println(<-oneSecond)
        fmt.Println(<-fiveSeconds)
    }
}

One second
Five seconds
One second
Five seconds
One second

The code above is intended to display "One second" every second and "Five seconds" every five seconds. However, currently, "One second" is displayed only every five seconds. This happens because receiving from a channel with no data blocks execution until data arrives. In this case, the channel for "Five seconds" pauses the program for five seconds, effectively halting the channel for "One second". Since we're using goroutines, we want to leverage concurrency instead of waiting for functions to execute sequentially. To address this, we'll introduce the select statement.

func main() {
    oneSecond := make(chan string)
    fiveSeconds := make(chan string)

    go func() {
        for i := 0; i < 20; i++ {
            time.Sleep(time.Second)
            oneSecond <- "One second"
        }
    }()

    go func() {
        for i := 0; i < 20; i++ {
            time.Sleep(time.Second * 5)
            fiveSeconds <- "Five seconds"
        }
    }()

    for i := 0; i < 20; i++ {
        select {
        case <-oneSecond:
            fmt.Println(<-oneSecond)
        case <-fiveSeconds:
            fmt.Println(<-fiveSeconds)
        }
    }

}

One second
One second
Five seconds
One second
One second
One second
Five seconds

By using the select statement inside the for loop, we can print the message in the exact moment the channel receives it, doing what is expected from a concurrent code.

4. Other Go concurrency tools

Goroutines, channels, and the select statement form the highest level of abstraction in Go for concurrency. These are the tools we're most encouraged to use daily. However, there may be situations where lower-level tools like mutexes and waitgroups become necessary. While discussing these patterns would deviate from this article's focus, I encourage everyone to explore these topics to gain a deeper understanding of concurrency and parallelism.