Fakes & Mocks on Android: Well Partner, that depends.

Over the course of 2021 and 2022, software testing fashion has swayed away from mocks, towards fakes. So when should you use one or the other? As with most serious engineering questions, the answer is "it depends."

Yet, in recent times mocks have reached levels of hate not seen since Disco Demolition Night, when America gave up on disco and burned all of their records.

Even our cultural guru and spiritual leader, the fabled Jake Wharton (🙌), shares this world view:

A newer piece of Android documentation now stops short of wholly recommending fakes, but does only showcase how fakes work - not mocks.

An Appeal to the Classics

The canonical essay on test doubles is Martin Fowler's famous 2007 essay, Mocks aren't Stubs. If you haven't ever read it, close this tab, and go read that instead. You'll learn more, and I won't be offended. It offers now-standard definitions for the different types of test doubles, including fakes and mocks:

Fake objects actually have working implementations, but usually take some shortcut which makes them not suitable for production (an in memory database is a good example).

Mocks are what we are talking about here: objects pre-programmed with expectations which form a specification of the calls they are expected to receive.

One of my favorite parts of this article is where Martin explores the two schools of thought, purists and mockists - or the "Detroit" style vs. the "London" style. This is the most correct framing of the debate, as one between schools of thought.

In full disclosure, I am a purist of the Detroit school. I don't care what behaviors a component need exercise; I prefer to look at its results. Even so, I will continue arguing a place for mocks, ✊.

How Sociological Trends Work

Before we get into the merits, I'd like to point out that we are indeed talking about a sociological fad more so than a technical evolution. I'm not pointing this out to be contrarian or diminutive, rather so that we can move forward with appropriate and metered expectations.

The Gartner Hype Cycle describes how societies respond to new technologies. New tech comes in, people don't yet see or understand the rough edges, and so hopes are high. "Wow, this has benefits!" Later, the drawbacks are learned. The cycle ends with a tool that you can use, but "it depends" when.

"Mocks Require Mock-able Code"

One argument I've seen is that you have to write impaired source code to use mocks. Actually, I think the opposite is sort of the case.

In languages that close implementations by default (our beloved Kotlin) you have to create an interface to take advantage of a fake. So, unless you're only doing component-level testing, you'd need to create interfaces for every unit. You'll end up with a lot of silly little interfaces, all in the name of faking.

On the other hand, maybe you should just do component-level testing. Here, I define "Component" as a collection of units (classes, interfaces, data types), that serve some well-bounded purpose. For example, a repository dressed with its production dependencies might be a component. Does every little detail beneath the repository need an interface? No, probably not.

Components are likely the biggest granule that can be reasonably tested on your host's JVM, without needing real implementations on your target device. This is, in my view, the best combination of:

1. "Right-sized" abstractions with interfaces;
2. Bang-for-your-buck w.r.t. test coverage;
3. Most approximating of real world behavior, while still being able to run on your workstation.

But, how would it work with mocks? You don't need to do anything to use mocks, that's arguably part of their charm. They're a great choice for legacy code which has few walls of abstraction. Frameworks like Mockito and MockK will happily spoof final/closed functionalities - even returning null from non-null functions. This is thanks to the Magic of Reflection™. Are all of these good traits? I mean: probably not. However, it is still true that mocks are less impacting to your actual source code.

Reflection is Slow and Spooky

Bars, dude. Bars. This is true.

Even Java, upon introducing the capability to its own language remarked in its documentation:

Reflection is powerful, but should not be used indiscriminately. If it is possible to perform an operation without using reflection, then it is preferable to avoid using it.

Now, it is possible to create a fake that uses reflection (your fake implementation could internally use a dynamic proxy pattern, or something.) And, it is technically possible to create mocks that don't use reflection. You could write a custom, concrete implementation for every single test case - obviously, though, that would be just: wow.

Never-the-less, it is most often the case that mocks = reflection, fakes = compiled contracts. So, fakes do have the usual benefits of a reflection-free existence here: they run faster, and you can catch more bugs in your test implementation through the compiler instead of while running your test.

Fakes Have an Upfront Cost

Fakes may require more up-front investment. Let's suppose you have a happy little interface like this:

Now, you want to use it, say in some use case - maybe you're updating user preferences.

class UpdateUserPreferencesUseCase(
  val userRepository: UserRepository,
  val user: User,
) {
  fun disableAnimations() {
    // ... 
  }
}

And, let's further suppose that your code layout looks something like this:

When you go to write this UpdateUserPreferencesUseCaseTest, you come upon a decision point. Should you create a fake of the UserRepository or should you mock it? And where should that code live, in either case?

For small interfaces, the cost of writing a fake vs. a few mock statements is pretty similar. You can write a fake of two functions very quickly - in about as much time as it takes to write the two mocking clauses.

For larger interfaces, the cost is higher right now. It will take you longer to think through a reasonable fake implementation for 5, 10, 20 function contracts. It will be cheaper to write only the two mock statements your test needs to pass, right now.

So in either case, the fake will take at least as much energy as the mock for right now.

"Fakes Have Lower Maintenance Cost"

Now, the real argument we hear in favor of fakes is that they have lower long-term maintenance cost. But is that true? Long-term maintenance cost also depends on various factors.

Suppose you put your fake in :settings:impl, next to your UpdateUserPreferencesUseCaseTest. At this point, there's no long-term cost. The one fake will have the same maintenance cost as the one file using mocks.

But what happens if you have 5 or more uses of that UserRepository in tests across your code base? Well, the natural evolution would be to end up with 5 different fake implementations, each living nearby to the tests that need them, in different modules. If you end up here, fakes are in fact awful for long-term maintenance.

Of course, there's a better way. A better strategy would have been to put the fake implementation in some re-usable module, say :users:fakes:

If you do this, all of your tests that need a FakeUserRepository can use the one common implementation. As the number of tests that need this component grow, so will you see improved long-term maintenance costs, and amortized returns on your up-front investment to build it. That's essentially Zac's argument in this graph:

But, here's the thing. This has nothing to do with fakes. You can do this with mocks, too! You could create a mocks factory and put it in :users:mocks:

object UserRepositoryMocking {
  fun mock(): UserRepository = mockk {
    // etc.
    every { getUser(any()) } returns User("Joe Swanson")
  }
}

So, it's more how you re-use code that impacts long-term maintenance cost, not something intrinsic about fakes vs. mocks.

If you put your fake implementations next to individual tests, you'll end up having to create multiple fake implementations, which will be costly to maintain.

And vice-versa, if you don't put your mocking code in some central tool, you'll end up with a lot of repeated setup code that you'll have to implement in every test.

Wrapping up

Honestly, one of the things I dislike most about tech articles is when someone just leaves you hanging with "it depends", and thinks they've done something clever with that. Like bruv, we pay you proper quid to get this stuff sorted, innit?

Here are the specific best practices that I think you should bring into your code base.

1. For interfaces that will see high re-use, write a fake implementation. Place the fake into a re-usable test module so that future tests can leverage it, too.
2. For larger interfaces, legacy components without interfaces, or components with very few uses ("the long tail"), prefer mocks. This will have lower up-front and lower long-term maintenance cost.
3. For mocking behavior you use across many tests, create a re-usable factory utility to centralize boilerplate, akin to what you do for fakes.
4. When authoring new source code, honor the Single Responsibility Principle and Interface Segregation Principle, which will facilitate either of these approaches, as you go forward.

Happy testing 🎉