How to Make JSON String Comparisons in Unit Tests Less Brittle

We previously looked at how we can extract longer texts used as expected values into separate files and load them as and when needed in our unit tests. As an example, we looked at a string representing the body of a welcome email for an (imaginary) online shop.

In this article, we’ll look at another use case where we can use this technique. We’ll also look at how we can make the checks a bit more robust too.

JSON Data and Unit Tests

Transferring data is a necessary part of building a useful system. Before being sent, it’s often converted into a common format understood by all subsystems involved. Due to its simplicity and human readability, JSON is a popular choice. We’ll use it in these examples, but the concept should also be applicable to other text-based data formats too, e.g. XML.

If we wanted to check that a method sends a particular piece of data, one option would be to compare its output against an expected value. When writing our test, we have two options:

1. Declare the expected JSON directly in test code.

2. Add it to a separate file and read from it while the test is running.

Option (1) can make the data difficult to read from the programmer’s perspective unless the transformed object is simple and has only a few properties. Line spacing and formatting aside, JSON properties need to be surrounded with double quotes; this conflicts with how strings are represented in C#. To work around this, we can:

• Escape JSON double quotes by prefixing them with \.

• Wrap JSON data containing double quotes inside double quotes in a C# verbatim string literal, i.e. a string starting with @.

• Use raw string literals, available in C# 11/.NET 7 (or greater). With this option, we can express a multi-line value that includes double quotes inside a block that starts and ends with a series of double quotes.

To avoid this problem, we can declare the expected value in a separate file; this is Option (2) in the list of options previous presented. However, if we formatted the data to make it readable, we’ll have one more issue we need to resolve. Let’s say we have the following class.

public class MyObject
{
    public string Property1 { get; set; }
    public string Property2 { get; set; }
}

It’s used in the following test. In this example, we instantiate a MyObject directly, but an instance could also have been created from a service we want to test.

[Test]
public void SerializedJsonIsAsExpected()
{
    // Arrange

    var myObject = new MyObject
    {
        Property1 = "Value1",
        Property2 = "Value2"
    };

    // Act

    var json = JsonConvert.SerializeObject(myObject);

    // Assert

    var expected = ReadEmbeddedResource("ExpectedJson.json");
    Assert.That(json, Is.EqualTo(expected));
}

private static string ReadEmbeddedResource(string resourceName)
{
    var assembly = Assembly.GetExecutingAssembly();
    using var stream =
        assembly.GetManifestResourceStream(resourceName);
    using var reader = new StreamReader(stream);
    var result = reader.ReadToEnd();
    return result;
}

ExpectedJson.json contains the following text.

{
      "Property1": "Value1",
      "Property2": "Value2"
}

When run, the test fails with the message:

Expected string length 53 but was 43. Strings differ at index 1.
Expected: "{\r\n\t"Property1": "Value1",\r\n\t"Property2": "Value2"\r\n}"
But was:  "{"Property1":"Value1","Property2":"Value2"}"
------------^

Formatting and String Comparisons

We formatted ExpectedJson.json to make it easier to both read and edit if necessary (this becomes more important with real-world data which is likely to be much more complex). However, the failing test shows that doing so causes problems when comparing string values. To resolve this, we can minify both sets of JSON data before the assertion. In the following code, we’ve added a Minify method where we strip out newline terminators, tabs, and spaces.

[Test]
public void SerializedJsonIsAsExpected()
{
    // Arrange

    var myObject = new MyObject
    {
        Property1 = "Value1",
        Property2 = "Value2"
    };

    // Act

    var json = JsonConvert.SerializeObject(myObject);

    // Assert

    var expected = ReadEmbeddedResource("ExpectedJson.json");
    var minifiedJson = Minify(json);
    var minifiedExpected = Minify(expected);
    Assert.That(minifiedJson, Is.EqualTo(minifiedExpected));
}

private static string Minify(string json)
{
    var minified = json.Replace("\r", "")
        .Replace("\n", "")
        .Replace("\t", "")
        .Replace(" ", "");

    return minified;
}

private static string ReadEmbeddedResource(string resourceName)
{
    var assembly = Assembly.GetExecutingAssembly();
    using var stream =
       assembly.GetManifestResourceStream(resourceName);
    using var reader = new StreamReader(stream);
    var result = reader.ReadToEnd();
    return result;
}

This implementation of Minify should be sufficient for most cases. However, we can deserialize and reserialize the data for a more consistent and robust approach.

private static string Minify(string json)
{
    var minified = JsonConvert.SerializeObject(
        JsonConvert.DeserializeObject(json));

    return minified;
}

Summary

Comparing JSON in unit tests can be trickier than expected. Tests can become harder to read and fail due to formatting. But you can work around these issues by following a few tips.

JSON property names must be surrounded by double quotes, conflicting with how string values are typically represented in C#. However, you can either escape the double quotes or use a more specialised string representation. Another option is to extract the JSON into a separate file, which also makes it more readable.

Formatting can also affect your test results. New line and indentation characters will make string values differ, even when your object and data values are otherwise identical. However, you can minify your JSON before making any comparisons. Doing this to both your expected and test values will remove any formatting differences and let you compare the actual data.

Thanks for reading!

This article is from my newsletter. If you found it useful, please consider subscribing. You’ll get more articles like this delivered straight to your inbox (once per week), plus bonus developer tips too!