It’s no secret that automated testing can be immensely helpful in the creation of better software. Perhaps less widely appreciated, however, are the features that ASP.NET Core offers to support testing. Here we discuss some of them, and see how enjoyable they make testing ASP.NET Core applications compared to old ASP.NET MVC, for instance. But first let’s review some of the different types of test we might wish to automate.
Automated testing and how it applies to ASP.NET Core
Testing is an integral part of the development process, and automated testing becomes crucial in the long run. You can always run your ASP.NET website, open a browser, and click everywhere to test your features. That's legit, but it is harder to test individual units of code that way. Another downside is the lack of automation. When you first start with a small app containing a few pages, a few endpoints, or a few features, it may be fast to manually run those tests. However, as your app grows, it takes longer, and the likelihood of making a mistake increases. Granted, you need real users to test out your applications, but you may want those tests to focus on the UX, the features content, or on some experimental features that you are building, rather than on bug reports that automated tests could have caught early on.
There are multiple types of test, and developers are very creative at finding new ways to test things. Three broad categories of automated test, however, are:
• Unit tests
• Integration tests
• Functional tests
Unit tests focus on individual units, like testing the outcome of a method. Unit tests should be fast and should not rely on infrastructure such as a database. These are the kind of test that you want the most, because they run fast. Each one should test a precise code path. They should also help you design your application better because you use your code in the tests, so you are becoming your first customer (or first consumer), leading you to find some design flaws and make your classes better. (If you don’t like how you are using your system in your tests, that’s a good indicator that nobody else will.)
Integration tests focus on interactions between components. Integration tests often require some infrastructure, such as a database, to interact with, which can make them slower to run. You want integration tests, but as a rule of thumb (to which there are sometimes exceptions) you want fewer of them than unit tests.
Functional tests focus on application-wide behaviors, like what happens when a user clicks on a specific button, navigates to a specific page, posts a form, or sends a PUT request to some Web API endpoint. Functional tests focus on testing the whole application from the user’s perspective, from a functional point of view. Usually, functional tests should be run in-memory, using an in-memory database or other resource. This helps to speed things up, but you could run end-to-end (e2e) tests on real infrastructure as well in order to test your application and your deployment.
There are other types of automated test, and some sub-genres as we could call them. For example, we could do load testing, performance testing, end-to-end testing, regression testing, contract testing, penetration testing, and more. You can automate tests for almost anything that you want to validate, but some kind of tests are harder to automate or more fragile than others, like UI tests. That said, if you can automate a test in a reasonable timeframe: do it! In the long run it should pay off.
One more thing: don’t blindly rely on metrics like code coverage. Those metrics make for cute badges in your GitHub project’s readme.md file, but can lead you off track writing useless tests. Sure, code coverage is a great metric when used correctly, but remember that one good test can be better than a bad test suite covering 100% of your codebase.
Writing good tests is not easy and only comes with practice.
Note
Keep your test suite healthy by periodically adding missing test cases and removing obsolete or useless tests. Think in terms of use-case coverage, not about how many lines of code are covered by your tests.
Testing made easy through ASP.NET Core
The ASP.NET Core team has made our life easier by designing ASP.NET Core for testability, and in general testing is way easier than before the ASP.NET Core era. Internally, they use xUnit to test .NET 5 (and .NET Core for it) and EF Core. And xUnit happens to be my favorite testing framework; what a happy coincidence!
Creating a xUnit test project
To create a new xUnit test project, you can run the dotnet new xunit command, and the CLI does the job for you by creating a project containing a UnitTest1 class. That command does the same than creating a new xUnit project from Visual Studio like this:
Figure 1: Create a xUnit project
Basic features of xUnit
In xUnit, the [Fact] attribute is the way to create unique test cases, while the [Theory] attribute is the way to create data-driven test cases.
Any method with no parameter can become a test method by decorating it with a [Fact] attribute, like this:
public class FactTest
{
[Fact]
public void Should_be_equal()
{
var expectedValue = 2;
var actualValue = 2;
Assert.Equal(expectedValue, actualValue);
}
}
From the Visual Studio Test Explorer, that fact looks like:
Figure 2: Tests result
Then, for more complex test cases, xUnit offers three options to define the data that a [Theory] should use: [InlineData], [MemberData], and [ClassData]. You are not limited to only one; you can use as many as it suits you to feed a theory with data. You must make sure that the number of values matches the number of parameters defined in the test method.
[InlineData] is the most suitable for constant, literal values, like this:
public class InlineDataTest
{
[Theory]
[InlineData(1, 1)]
[InlineData(2, 2)]
[InlineData(5, 5)]
public void Should_be_equal(int value1, int value2)
{
Assert.Equal(value1, value2);
}
}
That yields three test cases in the test explorer, where each can pass or fail individually.
Figure 3: Tests result
[MemberData] and [ClassData] can be used to simplify the declaration of the test method, to reuse the data in multiple test methods, or to encapsulate the data away from the test class. Here are a few examples of [MemberData] usage:
public class MemberDataTest
{
public static IEnumerable Data => new[]
{
new object[] { 1, 2, false },
new object[] { 2, 2, true },
new object[] { 3, 3, true },
};
public static TheoryData<int, int, bool> TypedData => new TheoryData<int, int, bool>
{
{ 3, 2, false },
{ 2, 3, false },
{ 5, 5, true },
};
[Theory]
[MemberData(nameof(Data))]
[MemberData(nameof(TypedData))]
[MemberData(nameof(ExternalData.GetData), 10, MemberType = typeof(ExternalData))]
[MemberData(nameof(ExternalData.TypedData), MemberType = typeof(ExternalData))]
public void Should_be_equal(int value1, int value2, bool shouldBeEqual)
{
if (shouldBeEqual)
{
Assert.Equal(value1, value2);
}
else
{
Assert.NotEqual(value1, value2);
}
}
public class ExternalData
{
public static IEnumerable<object[]> GetData(int start) => new[]
{
new object[] { start, start, true },
new object[] { start, start + 1, false },
new object[] { start + 1, start + 1, true },
};
public static TheoryData<int, int, bool> TypedData => new TheoryData<int, int, bool>
{
{ 20, 30, false },
{ 40, 50, false },
{ 50, 50, true },
};
}
}
That test case should yield 12 results. If we break that down, the code start by loading three sets of data from the IEnumerable Data property by decorating the test method with the [MemberData(nameof(Data))] attribute.
Then to make it clearer, we replace the IEnumerable by a TheoryData<…> class, making it more readable, which is my preferred way of defining member data. We feed those three sets of data to the test method by decorating it with the [MemberData(nameof(TypedData))] attribute.
Then, three more sets of data are passed to the test method. Those originate from a method on an external type taking 10 as an argument (the start parameter). We specify the MemberType where the method is located, so xUnit knows where to look, which is represented by the [MemberData(nameof(ExternalData.GetData), 10, MemberType = typeof(ExternalData))] attribute.
Finally, we are doing the same for the ExternalData.TypedData property, which is represented by the [MemberData(nameof(ExternalData.TypedData), MemberType = typeof(ExternalData))] attribute.
When running the tests, those [MemberData] attributes yield the following result in the test explorer:
Figure 4: Tests result
These are only a few examples of what we can do with the [MemberData] attribute.
Now to the [ClassData] attribute, that one gets its data from a class implementing IEnumerable or by inheriting from TheoryData<…>. Here is an example:
public class ClassDataTest
{
[Theory]
[ClassData(typeof(TheoryDataClass))]
[ClassData(typeof(TheoryTypedDataClass))]
public void Should_be_equal(int value1, int value2, bool shouldBeEqual)
{
if (shouldBeEqual)
{
Assert.Equal(value1, value2);
}
else
{
Assert.NotEqual(value1, value2);
}
}
public class TheoryDataClass : IEnumerable<object[]>
{
public IEnumerator<object[]> GetEnumerator()
{
yield return new object[] { 1, 2, false };
yield return new object[] { 2, 2, true };
yield return new object[] { 3, 3, true };
}
IEnumerator IEnumerable.GetEnumerator() => GetEnumerator();
}
public class TheoryTypedDataClass : TheoryData<int, int, bool>
{
public TheoryTypedDataClass()
{
Add(102, 104, false);
}
}
}
These are very similar to [MemberData], but instead of pointing to a member, we point to a type. Here is the result in the test explorer:
Figure 5: Tests result
Now that [Fact] and [Theory] are out of the way, xUnit offers test fixtures to allow developers to inject dependencies into a test class constructor, as parameters. Fixtures allow those dependencies to be reused by all of the test methods of a test class by implementing the IClassFixture interface. That is very helpful for costly dependencies like accessing a database: created once, reused many times.
You can also share a fixture (a dependency) between multiple test classes by using the ICollectionFixture, [Collection], and [CollectionDefinition] instead. We won’t get too deep intothe details here, but you will know where to look if you ever need something similar.
Finally, if you’ve worked with other testing frameworks, you might have encountered Setup and Teardown methods. In xUnit, there are no particular Attribute or mechanisms to handle setup and teardown code. Instead, xUnit uses existing OOP concepts:
- To set up your tests, use the class constructor.
- To teardown (clean up) your tests, implement IDisposable, and dispose of your resources there. That’s it! xUnit is very simple, yet powerful, which is the main reason why I adopted it as my main testing framework several years ago. Now let’s see how to organize those tests. Organizing tests There are many ways of organizing test projects inside a solution. One that I like is to create a unit test project for each project in the solution, one or more integration tests project(s), and a single functional tests project. Since most of the time we have more unit tests, and the unit tests are directly related to single units of code, it makes sense to organize them into a one-on-one relationship. Then we should have fewer integration and functional tests, so a single project should be enough to keep our code organized. Note Some people may recommend creating a single unit tests project per solution instead of one per project. This approach could save discovery time. I think that for most solutions, it is only a matter of preference. If you need performance and find a better way for yours, by all means, use that approach instead! That said, I find that one unit test project per assembly is more portable and easier to navigate. Most of the time, at the solution level, I create my application and its related libraries into an src directory, and I create my test projects into a test directory, like this:
Figure 6: The Automated Testing solution explorer, displaying how the projects are organized
In my Automated Testing solution I don't have any integration test, so I haven't created an integration test project. I could have named one IntegrationTests or MyApp.IntegrationTests depending on my approach.
One more detail that I’ve found helps get tests and code get aligned perfectly is to create unit tests in the same namespace as the subject under test. To make it easier in Visual Studio, you can change the default namespace used by Visual Studio when creating a new class in your test project by adding [Project under test namespace] to a PropertyGroup of the test project file (*.csproj), like that:
net5.0
false
MyApp
Then I name my test classes [class under test]Test.cs and create them in the same directory as in the original project, like this:
Figure 7: The Automated Testing solution explorer, displaying how tests are organized
Finding tests is trivial when you follow that simple rule. Sometimes, it is not possible to do that for integration tests or functional tests; in those cases, use your specifications to help you create clear naming conventions that make sense for your tests. Remember, we are testing use cases.
Finally, for each class, I nest one test class per method that inherits from the nested class's parent class, then I create my test cases inside of it using [Fact] and [Theory] attributes. This help organizes tests efficiently by method, ending with a test hierarchy like this:
namespace MyApp.Controllers
{
public class ValuesControllerTest
{
public class Get : ValuesControllerTest
{
[Fact]
public void Should_return_the_expected_strings()
{
// Arrange
var sut = new ValuesController();
// Act
var result = sut.Get();
// Assert
Assert.Collection(result.Value,
x => Assert.Equal("value1", x),
x => Assert.Equal("value2", x)
);
}
}
}
}
That technique allows you to set up tests step by step. For example, you can create top-level private mocks; then, for each method, you can modify the setup or create other private test elements, then you can do the same per test case, inside of the test method. Don’t go too hard on reusability however; it can make tests hard to follow for an external eye, like a reviewer or another developer that needs to play there. Unit tests should remain clear, small, and easy to read. The [Fact] attribute define the method as a test case, more on that later.
How is it easier?
Microsoft built .NET Core (now .NET 5) from the ground up, so they fixed and improved so many things that I cannot enumerate them all here, including testability. Not everything is perfect, but it is way better than it ever was.
Let’s start by talking about the Program and the Startup classes. Those two classes are the place to define how the application boots and its composition. Based on that model, the ASP.NET Core team created a test server class that allows you to run your application in memory.
They also added WebApplicationFactory in .NET Core 2.1 to make integration and functional testing even easier than before. With that class, you can boot up your ASP.NET Core application in-memory and query it with the supplied HttpClient. All of that in only a few lines of code. There are extension points to configure it, like replace implementations by mocks, stubs, or any other test-specific elements that you may require. TEntry should be your project under test Startup or Program class.
I created a few test cases in the Automated Testing project that exposes this functionality:
namespace FunctionalTests.Controllers
{
public class ValuesControllerTest : IClassFixture>
{
private readonly HttpClient _httpClient;
public ValuesControllerTest(WebApplicationFactory webApplicationFactory)
{
_httpClient = webApplicationFactory.CreateClient();
}
Here we are injecting a WebApplicationFactory into the constructor by implementing the IClassFixture interface. We could use the factory to configure the test server, but since it was not required here, we can only keep a reference on the HttpClient that is configured to connect to the in-memory test server running the application.
public class Get : ValuesControllerTest
{
public Get(WebApplicationFactory webApplicationFactory) : base(webApplicationFactory) { }
[Fact]
public async Task Should_respond_a_status_200_OK()
{
// Act
var result = await _httpClient.GetAsync("/api/values");
// Assert
Assert.Equal(HttpStatusCode.OK, result.StatusCode);
}
In the test case above, we are using the test HttpClient to query the http://localhost/api/values URI, accessible through the in-memory server. Then we test the status code of the HTTP response to make sure it was a success (200 OK).
[Fact]
public async Task Should_respond_the_expected_strings()
{
// Act
var result = await _httpClient.GetAsync("/api/values");
// Assert
var contentText = await result.Content.ReadAsStringAsync();
var content = JsonSerializer.Deserialize<string[]>(contentText);
Assert.Collection(content,
x => Assert.Equal("value1", x),
x => Assert.Equal("value2", x)
);
}
}
}
}
This last test does the same but deserializes the body's content as a string[] to ensure the values are the same as what we were expecting. If you worked with an HttpClient before, this should be very familiar to you.
When running those tests, an in-memory web server starts; then, HTTP requests are sent to that server, testing the full application. In that case, the tests are trivial, but you can create more complex test cases as well.
You can run .NET 5 tests within Visual Studio, or using the CLI by running the dotnet test command. In VS Code, you can use the CLI or find an extension to help you out with test runs.
Conclusion
Automated testing is an indispensable element of modern software development practice, and an effective test regime must encompass a wide variety of test types. Here we’ve looked at three of the fundamental test categories: unit tests, integration tests, and functional tests. We also had a quick look at the xUnit testing framework, which provides a highly effective mechanism for both implementing and organizing tests. Finally we’ve seen how ASP.NET Core, by allowing us to mount and run our ASP.NET Core application in memory, makes it easier than ever to test our web applications.
Purchase your copy on Amazon.com
Top comments (0)