I don't trust unit tests with mocks

Dave Farley asks why TDD hasn't conquered the world:

Dave Farley

@davefarley77

Why hasn’t TDD taken over the world? Anyone have any ideas? 🤷‍♂️

17:15 PM - 21 May 2020

Besides the obvious, and correct, reasons like:

• People don't know how to do write tests.
• People don't have enough time to practice testing.
• Most of the code we write is untestable.

I find the following reason especially true for myself:

Unit tests with heavy use of mocks are terrible and I don't trust them!

Some example

TDD advocates unit tests should be written before the implementation. It is also considered that units refer to single methods. In modern C# Web applications these methods typically refer to service methods. Typical service is stateless and various methods do not share anything but being part of the same class. The only reason we have service in C# is to use constructor based dependency injection in order to simplify dependencies. An example:

sealed class UserService : IUserService
{
    private readonly IUserRepository userRepository;

    public UserService(IUserRepository userRepository)
    {
        this.userRepository = userRepository;
    }

    public void ChangeUsername(string userId, string newName)
    {
        if (string.IsNullOrEmpty(newName))
            throw new ArgumentException(nameof(newName));

        User user = userRepository.GetUserById(userId);
        user.Name = newName;
        userRepository.Save();
    }
}

By using constructor injection our public interface is simpler.

interface IUserService
{
    void ChangeUsername(string userId, string newName);
}

I would argue that this kind of code is much easier to write and reason about. However, testing is deceptively easier. And in that deception the problem lies.

TDD...is impossible?

Forget the above implementation and just focus on the method we need to test:

void ChangeUsername(string userId, string newName);

How should we approach implementing this method using TDD? Once we test the obvious validation which just tests that the method throws, we need to test the happy path. Thus we start by creating our system under test (SUT):

void NewName__IsSaved()
{
  // Arrange
  string userId = "FAKE-ID";
  string newName = "new-name";
  var sut = new UserService(Mock.Of<IUserRepository>());

  // Act
  sut.ChangeUsername(userId, newName);

  // Assert
  // ???
}

In the above sample, mocking is done using Moq, but other libraries could be used as well.

What matters is that it is extremely hard for a developer to correctly continue implementing this test. First of all, how do we verify correct behavior? How do we write that code intuitively? What does it mean for this code to succeed.

void NewName__IsSaved()
{
  // Arrange
  string userId = "FAKE-ID";
  string newName = "new-name";
+ var user = new User();
+ var userRepoMock = new Mock<IUserRepository>();
+ userRepoMock
+   .Setup(r => r.GetUserById(userId))
+   .Returns(user);
+ var sut = new UserService(userRepoMock);

   // Act
   sut.ChangeUsername(userId, newName);

  // Assert
+ user.Name.Should().Be(newName);
+ userRepoMock.Verify(r => r.Save(), Times.Once);
}

Added lines are marked with +.

Ouch

The first issue is the explosion in complexity of our arrange phase. The second problem is that it is not obvious what to do withour prior knowledge of the implementation. This point cannot be stressed enough - for TDD to work it should be easy to write tests beforehand just by reasoning. And the above test cannot be written before the implementation - thus TDD won't help you.

And that code is typical in most applications.

The code is testable, code coverage is high, some regressions are prevented...but personally, this style of coding and testing is ugly, nonintuitive, heavy, and not in the spirit of TDD.

In fact, the test suite comprised of tests like that one often trigger memes where unit tests pass, but the system doesn't work.

Is there a way out of here?

Sure, use TDD to unit test pure methods with black box testing where the output is easy to test and comprehend just by looking at the input and method name.

If you need to open the method to correctly mock stuff then you'll face issues.

Some points to consider:

• For complex features use BDD testing on a higher level instead of focusing on individual methods like this.
• Mocking doesn't necessarily mean using a low-level mocking library - an in-memory DB is a valid mock. If you are testing feature interaction, the database implementation is literally mocked away by using an in-memory representation. How DB works is not the point of those tests.
• Consider only mocking code you don't own - thus interior part of your module/assembly shouldn't be mocked away.
• Learn your tools - testing should be done and some proficiency of tools is required before knowing what and when to test.
• Learn the difference between state testing and property testing.
• Learn the difference between mockist and non-mockist schools of TDD. If you are learning about TDD, the not-so-subtle difference of these schools of thought is usually hidden from the text. This is extremely dangerous when paired with naive examples like testing ICalculator.Add method which is a pure method and is extremely easy to TDD. Unlike the above example typical in most line of business applications.

TDD is an extremely powerful technique, but it is not a silver bullet. Practice and learn where it can be applied with ease for a big gain, and where it is just a hindrance and waste of time.