KWAN

Posted on Dec 13 • Edited on Dec 17

Swift Testing: A Comprehensive Overview

#testing #swift

Let’s deep dive into Swift testing, get to know the strengths of XCTest and the exciting new SwiftTesting API introduced at WWDC24. While XCTest remains a staple, it struggles with concurrency and customization. Enter SwiftTesting, a modern approach that uses powerful macros to streamline code and enhance test organization. Discover how these advancements are transforming testing, making it more efficient and developer-friendly in the evolving Swift ecosystem.

The Most Used Framework

XCTest is the go-to framework for unit testing in the Apple ecosystem. Originally written by Apple in 1998 in Objective-C, it was integrated into Xcode in 2013, providing tools for writing and making assertions. It still works like a charm despite Swift’s evolution and the introduction of incredible new APIs. However, the XCTest framework isn’t quite what we needed once upon a time.

Challenges with the Competition

Despite the numerous improvements to XCTest for testing concurrency more easily, Swift Testing now offers a more efficient solution, giving us better tools while running tests in parallel by default.

Since the introduction of the new concurrency API, developers have been experimenting with various ways of writing unit tests using this framework. Sometimes, this results in boilerplate code or even code that doesn’t make much sense.

Here’s a fair example of it:

Even if we call the function using the await keyword, the test will fail because, in theory, we have to wait for the task to complete. Hence, we need to call the .value property, which stands for the result of a task after it is completed.

Customization and Other Issues

There are other problems, such as customization. Currently, it is hard to configure an environment for the tests. There is no effective way to organize the tests, and performance issues are common since the framework does not provide robust support for parallelization and async jobs. Debugging test failures has always been time-consuming and challenging.

Swift Testing

Apple has introduced an innovative open-source API that is ergonomic, modern, expressive, and highly customizable. This new API extensively uses macros to minimize boilerplate code and enable complex operations with less effort.

With the introduction of the @test macro, developers can pass traits, allowing for the specification of runtime conditions.

The API leverages macros to define the test structure, utilizing @suite and @test for the organization. These macros, combined with the customizable traits, offer a fully tailored testing experience. Additionally, the #expect macro ensures that expected conditions are validated and any failures are reported effectively.

Parameterized tests

For scenarios where a test must be executed with different inputs, parameterized tests provide a convenient solution. These tests allow the definition of a collection of input values that the test can iterate over, ensuring each case is evaluated.

Adding tags

You can add custom tags to categorize, identify, or filter specific subsets of tests.

Configuring custom behavior

By using traits, we can define custom behaviors such as enabling or disabling tests, configuring timeouts, running tests serially or in parallel, and adding annotations to associate tests with bugs. Additionally, we can create custom traits, perform device-specific testing, or limit tests to specific OS versions for more targeted test coverage.

Migration to Swift Testing

Apple designed Swift Testing to work alongside XCTest, allowing for gradual adoption. Rather than needing to fully replace XCTest, developers can slowly migrate their tests by writing new ones with Swift Testing while maintaining their old XCTest tests.

Grouping Tests

Previously, we needed to use a class that was inherited from XCTestCase to group tests. Now, we can use a struct, which is recognized as a suite. Also, for a function to be considered a test function, it no longer needs to start with “test”. Apple has effectively leveraged macros to achieve that.

Setup and Teardown

Running code before or after each test is now simpler. Since our suite is a struct, we can use the init and deinit methods to handle setup and teardown.

Simplifying Assertions

Assertions in Swift testing have been redesigned to feel more natural, allowing developers to use plain Swift code instead of relying on specialized functions. This eliminates the need to constantly look up testing functions, making the process more intuitive.

Additionally, test result reporting has been greatly improved. Instead of vague or unclear feedback, the new system provides powerful diffing results, making it much easier to understand why a test is failing.

This table demonstrates how we can express our expectations in the new API based on XCTest.

Here’s how the result is displayed after a test failure. We can expand the failure diff, which is a game-changer for our productivity.