Discovering UI Performance testing with XCTest. Navigation performance.

One of the most important components of app performance is the performance of navigation transitions between different screens. Applying new changes to a screen without proper testing can significantly degrade this aspect of an app’s performance. In this article, I will show how to build performance tests to address this problem. This article has some references to my previous article about scrolling performance and I recommend to read it first.

Setup

I prepared the demo project. This is a simple app that shows you a list of plants, you can select a plant and open the details screen. For all measurements I used MacBook M2 Pro with Ventura 13.0 and iPhone 13 Pro with iOS 16.0, XCode version is 14.2.

Here is the test I used for testing. It tests navigation performance when opening the details screen from the main screen:

import XCTest

final class NavigationPerformanceTests: XCTestCase {
    let app = XCUIApplication()

    func testNavigationTransition() {
        app.launch()
        let measureOptions = XCTMeasureOptions()
        measureOptions.invocationOptions = [.manuallyStop]

        measure(metrics: [XCTOSSignpostMetric.navigationTransitionMetric], options: measureOptions) {
            app.staticTexts["Monstera"].tap()
            app.navigationBars.buttons.element(boundBy: 0).tap()
            stopMeasuring()
        }
    }
}

As I found out in the previous article there are limited metrics available on a simulator compared with a real device. To understand if this method is applicable I will test it both on a simulator and a real device.

Testing

Firstly we need to set up baselines for our test. I described the process in detail here. After baseline setup we are ready to go. To simulate delay I used this:

/* x is number of microseconds, for example usleep(10000)
blocks current thread for 10000 microseconds
or 0.01 second and usleep(1000000) blocks for 1 second */

usleep(x)

I did not notice any difference between different launches in contrast to measuring scroll performance so there is only one try for each delay value. Following hitch-related metrics are not relevant in this case and during testing value for them is always zero:

• Hitch Time Ratio (NavigationTransition): 0.000 ms per s
• Hitches Total Duration (NavigationTransition): 0.000 ms
• Number of Hitches (NavigationTransition): 0.000 hitches

I ignored these values during testing. For delay values I tried to find the pivot point when tests start to fail. Following table demonstrates test results:

Results show the following:

1. Duration is the most sensitive metric during measuring navigation performance.
2. Simulator results are very similar to real device results.
3. Tests start to fail both for a simulator and a real device with a delay equal to or more than 50 ms
4. Frame Count and Frame Rate are not predictable in the described case and hardly usable

Conclusion

The described method is an efficient tool to detect navigation performance issues in early development stages. It has some constraints such as using the same device model for test recording and testing, but even with a simulator it can detect delays that are not noticeable for a human. In my opinion, this can be a great way of controlling crucial flows performance in your app.

Useful links

Writing and running performance tests