As a software tester, I've worked on numerous projects regarding mobile applications, websites, etc., but never have I imagined myself testing a tvOS application for Apple TV. When I got the message that I will be testing an app for it, I was really thrilled and couldn't wait to test it, but at the same time, a bit nervous because I didn't have any experience of testing smart TV apps whatsoever. After researching for a bit, I've realised that it's nothing more than an iOS-based system with similar frameworks, concepts and technologies only converted into a little plasticky black box for TVs. Therefore, here are some tips that I've learned while testing the application in terms of comparing it to mobile apps.
TIP 1: "Size matters"
The first and main thing you'll notice (if you've never tested apps for TVs) is that in general, TVs are large-sized 16:9 screens. Accordingly, layout of the elements should scale properly on the interface and they should be clearly readable and seen without any cut outs and blurriness (especially text and picture sharpness). Also, because of the layout (and because apps are made in that way), elements (in general) will be displayed in horizontal orientation and thus, testing becomes a bit different. Since there are many sizes of TVs on the market, there's a difference between displaying the amount of elements on a 32" and 55" TVs, so advice to you is to check with appropriate person if elements should be dynamically resized and shown on any kind of TV sizes.
On the other hand, apps made for mobile devices (which are ofc, much smaller) have different aspect ratios and since the layout is automatically different, elements will be displayed in vertical orientation.
Side note: I've also tested layout on a monitor that's 21:9 aspect ratio, and believe me, it's night and day difference from 16:9. Everything was stretched in size and it was clearly visible that text and pictures became more blurrier (you never know if someone will open your app with Apple TV box connected to a 49" super ultrawide monitor and see those ugly black boxes on the sides).
TIP 2: Prepare to collect a lot of pictures (not screenshots)
While testing the app, I've managed to find many differentiations in terms of shown layout design and elements compared to the actual design. Naturally, I was going to screenshot it directly with Apple TV remote, only to see that the screenshot is black. I didn't want to waste time so I used my phone to take a picture and use it as a reference in a bug report. After that, I googled quickly on how to take screenshots only to find out (yeah, you guessed it) Apple doesn't allow it.
So if you encounter many issues while testing application, the most painless method is by taking pictures with your phone and either Airdrop it to your Mac (if you use it) or send via email. Many of the methods I've googled were too complicated and time-consuming to set up, and pictures that were taken with phone had a good quality for developers to notice differences.
TIP 3: "Did you try turning it on and off?"
One morning I was installing newest build and logged into the application. After a couple of minutes of scrolling and redirecting to different screens, I've noticed a strange loading screen that didn't disappear for a couple of minutes. There was no error message or any popup that would indicate which problem occurred. I've lost half an hour of troubleshooting only to see that I've lost internet connection in my house. So, after getting internet connection back and checking the application, screen loaded as intended. Therefore, you should always test these types of cases because you don't know what type of internet end user has which, in the end, really brings down user experience by a mile.
Conclusion
TV manufacturers, IT companies and other interested parties make a lot of investments into smart TV apps nowadays and it's either "make or break" situation. End users tend to not use apps that are poorly optimised because experience nowadays disappears in a couple of seconds. Make sure that you create a proper amount of test cases that have certain complexities to it which, in the end, will deliver the best possible end user experience and make your testing actually fun, thriving and exciting.
Feel free to share your experience with testing tvOS apps and smart TV apps in general and if you have also experienced similar problems.
Top comments (0)