Dependency Injection Explained

This post originally appeared on my blog: https://markphelps.me/posts/dependency-injection-explained/

Have you heard the term “Dependency Injection” before but struggled to really grok what it means? How do you properly do Dependency Injection in your Go applications and more importantly why? What does using Dependency Injection enable you to do and why is preferred over other methods?

These are the questions I hope to answer with this post. Hopefully, after reading, you’ll feel more confident and be better equipped to answer the above questions yourself and educate your team on best practices for managing internal dependencies in your code.

Overview

Before we get started, however, we should probably define what Dependency Injection is and why it’s widely considered a best practice, especially in the Go community.

“Dependency Injection is a 25-dollar term for a 5-cent concept.”

— James Shore

note: There are tons of articles written over the past 20 years or so about Inversion of Control/Dependency Injection so I won’t go too in-depth in this post, but I do want to cover the basics.

The way I think of Dependency Injection (DI) is that you ‘pass in’ the resources (dependencies) that your code needs to ‘do its job’ instead of your code ‘reaching out’ for those resources itself.

An example in a basic Go application would be providing a type with a sql.DB instance so that it can interact with the database. Let’s create write some code.

First Attempt

Here’s what this might look like without using Dependency Injection:

// services/user.go
// UserService queries and mutates users in the database.
type UserService struct {
    db *sql.DB
}

// NewUserService 'constructs' a UserService that is ready to use.
func NewUserService() (*UserService, error) {
    db, err := sql.Open("mysql", "user:password@tcp(127.0.0.1:3306)/db")

    if err != nil {
        return nil, err
    }

    // TODO: how do we close the db connection when we are done?
    // defer db.Close()

    return &UserService{db}, nil
}

This is probably how most beginning developers would fulfill the requirement that a UserService needs to be able to maintain a connection to the database. Let’s go over why implementing it this way is a bad idea:

1. This UserService type is extremely hard to test since this code assumes you will always be using a MySQL instance with the given connection string. This could be mitigated somewhat using environment variables, however, we’ll discuss why this still isn’t ideal later on.
2. It’s best practice to always close the sql.DB connection when you are done with it. Here since our DB connection is created in the NewUserService, we have no easy way to call db.Close() other than adding a Close() method on the UserService itself.. which is kind of weird when you think about it. Why would a thing called UserService need to close? It should just contain the business logic to handle users in our system.
3. Each time we call sql.Open we are likely creating a new pool of connections (this is driver specific) as the sql.DB docs states: “The returned DB is safe for concurrent use by multiple goroutines and maintains its own pool of idle connections. Thus, the Open function should be called just once ”. This means that if we had another ‘Service’ type, it would also be opening its own connection(s) to the database and not be able to make use of the existing pooled idle connections.
4. It’s unclear from an external ‘API’ perspective that the UserService does anything with a database at all since our database initialization is ‘hidden’ within. This makes the code harder to read and understand at a glance.

Global State

Instead of opening a connection each time you instantiate a new Service type, you could create the sql.DB handle once and use it wherever you need it in your application like so:

// services/service.go
var db *sql.DB

// runs once at startup
func init() {
    var err error
    db, err = sql.Open("mysql", "user:password@tcp(127.0.0.1:3306)/db")
    if err != nil {
        // nothing we can do here so just bail
        log.Fatal(err)
    }
}

func NewUserService() (*UserService, error) {
    return &UserService{db}, nil
}

// showing that db can be used multiple times
func NewFooService() (*UserService, error) {
    return &FooService{db}, nil
}

This is called global state, and it’s a really bad idea. Better articles than mine have been written about why it’s a bad idea, so I’ll just link to a few:

• https://wiki.c2.com/?GlobalVariablesAreBad
• https://peter.bourgon.org/blog/2017/06/09/theory-of-modern-go.html
• https://medium.com/higher-order-functions/golang-its-a-global-problem-1a939461c16d

The main reasons you want to avoid introducing global state in your applications are that it introduces tight coupling between components, makes testing harder, and also can lead to hard to debug race conditions. All bad stuff.

Also, switching to use global state still doesn’t solve items #1, #2 or #4 above in our list of shortcomings with the previous approach.

Let’s Inject Some Dependencies

Hopefully, you’re convinced that the above two approaches are less than ideal and that there must be a better way to share dependencies in our code. This is where DI comes into play. Let’s walk through our updated code:

// services/user.go
// UserService queries and mutates users in the database.
type UserService struct {
    db *sql.DB
}

// NewUserService 'constructs' a UserService that is ready to use.
// It requires an initialized sql.DB instance.
func NewUserService(db *sql.DB) (*UserService, error) {
    return &UserService{DB}, nil
}

Here we have modified our NewUserService function to accept a *sql.DB instance as an argument, which we then can use to instantiate a new UserService.

What’s the big deal? We’ll we’ve now made our code infinitely more testable while also allowing us to share the db instance with other services. Let’s look at how the updated NewUserService function gets called, and then cover the testing bits:

// main.go
func main() {
    db, err := sql.Open("mysql", "user:password@tcp(127.0.0.1:3306)/db")

    if err != nil {
        // nothing we can do here so just bail
        log.Fatal(err)
    }

    // note: here we can cleanly close our db connection since this defer will run right before the program exits
    defer func() {
        if err := db.Close(); err != nil {
            log.Error(err)
        }
    }()

    // here we call our updated function passing in our db object
    u, err := services.NewUserService(db)
    if err != nil {
        log.Fatal(err)
    }

    // now use our UserService however we want
}

You can see we now do all of the ‘setup’ of opening the DB connection in our main function, which also allows us to call db.Close() before the program exits via a defer function. We then ‘inject’ our database dependency into our code that will be using it instead of our UserService type creating or ‘reaching out’ to get this dependency itself. Not only that, but this code is also much clearer in that if you understand the New idiom, you now know that UserService will be interacting with some database.

OK, but how does this make the UserService code easier to test?

Testing With Dependencies

Now that we are passing in the *sql.DB object itself, our UserService is no longer responsible for creating it. This means that when testing our UserService we can swap in any SQL database that we want. This is huge for testing since it means that we no longer need to use a ‘real’ MySQL instance in our tests!

note: For any real application I would still suggest you have integration tests that exercise the same type of database you use in production.

Instead of depending on a MySQL instance when running our tests locally or in CI we can instead swap in a connection to a self-contained (on the filesystem) SQLite database. This reduces the complexity and external dependencies required for local development and testing.

Here’s an example test setup:

// services/user_test.go
// global OK here since it's only used in tests
var db *sql.DB

// https://pkg.go.dev/testing#hdr-Main
func TestMain(m *testing.M) {
    var err error
    db, err = sql.Open("sqlite3", "file:test.db")
    if err != nil {
        t.Fatal(err)
    }
    os.Exit(m.Run())
}

func TestNewUserService(t *testing.T) {
    u, err := NewUserService(db)
    if err != nil {
        t.Fatal(err)
    }

    // continue testing UserService here but using SQLite DB
}

What About Mocking

Some of you may be saying to yourselves “That’s great an all, but what if I don’t want to use a real database in my tests at all?”. First I would say to you, well you really should use a real database.. however, I’ll admit there are times when using a mock dependency is preferred, especially if that dependency is on an external service (like one that you would interact with over the Internet).

In our example here, the dependency that we are injecting is an actual object, a pointer to an instance of sql.DB. Instead of defining our NewUserService function with this parameter set, we could instead abstract the *sql.DB into the behavior that we depend on rather than the actual thing itself. This is part of what the common Go saying “accept interfaces, return structs” is getting at.

Let’s wrap up with a quick example.

While implementing our UserService we realize that currently our application only needs to ‘look up’ a user given an email address. The implementation might look like this:

// services/user.go
type UserService struct {
    db *sql.DB
}

func (u *UserService) GetUserByEmail(ctx context.Context, email string) (User, error) {
    var u User

    rows, err := u.db.QueryRow("select id, name from users where email = ?", email).Scan(&u.ID, &u.Name)
    if err != nil {
        return u, err
    }

    return u, nil
}

Now we can see that our code only cares that our db has a method QueryRow. Since that’s the case, we can declare an interface that defines the behavior we need and then modify our UserService type to accept this new interface instead of an actual *sql.DB object.

Here’s the full code:

type querier interface {
    QueryRow(query string, args ...interface{}) *sql.Row
}

type UserService struct {
    q querier
}

func NewUserService(q querier) (*UserService, error) {
    return &UserService{q}, nil
}

func (u *UserService) GetUserByEmail(ctx context.Context, email string) (User, error) {
    var u User

    rows, err := u.q.QueryRow("select id, name from users where email = ?", email).Scan(&u.ID, &u.Name)
    if err != nil {
        return u, err
    }

    return u, nil
}

Since interfaces are implicit in Go, sql.DB already implements the querier interface!

Now in our tests, we can use one of the several mocking libraries such as golang/mock or stretchr/testify/mock to mock a type that implements our new querier interface, or we can just implement our own! We can come up with all different kinds of scenarios and error cases that our querier could return to make sure our UserService can gracefully handle whatever we throw at it.

Next Steps

While this post uses a contrived example using a database as our dependency and a single query, the same technique can be used when writing code that depends on external actors outside of our control like payment processors, email service providers, etc.

In my opinion, Dependency Injection is a necessary technique when developing extendable, well tested, and well architected Go applications. Setting up your application in this way does take some forethought, however, I believe that this upfront work will pay off in the end with better testability and cleaner code.

Here I showed how I personally do Dependency Injection in my applications the manual way, however, there do exist a few libraries that can help reduce the boilerplate code required and automate Dependency Injection in larger codebases. The two main ones that I’ve seen used are:

• https://github.com/google/wire
• https://github.com/uber-go/fx

I haven’t personally used either of them nor seen the need to frankly, but I would love to hear if you have and your thoughts on either!

Do you currently use DI in your Go apps? If so, do you prefer the ‘manual’ way or to use a library to do the heavy lifting for you? I’d love it if you reached out on Twitter and told me about your experiences!