DEV Community


Automatic cross-platform testing: part 4: Windows

drhyde profile image David Cantrell Updated on ・3 min read

Previously on this channel

Testing on Windows

I am very much not a Windows person, I've not touched it in over 15 years and not specifically targeted it as a developer for over 20. Back then, Windows NT 4 was where it was at. But nevertheless, even though I'm not specifically targeting Windows, there are people who want to run my code on Windows, so just to make my life easier I'm testing what I can on Windows. This time, when I looked around for a hosted Windows CI system Appveyor was the first reasonable-looking result.

As you will now be familiar with, having done it with Travis and Cirrus-CI, the first thing to do is sign in using your Github account and grant them some permissions. Then, as with Travis, you'll need to tell it which repositories you want it to pay attention to.


This will require another round of permission granting first. It gives you a choice of authentication methods, I've gone with the recommended "Github App" method. Once you've done that, add a config file and Appveyor will start building. Or trying to build anyway.

The build failed, with a helpful diagnostic

Thats a very nice helpful error message, I wish the others were like that, it would save me aggravation and save them money answering my support requests, to which they have invariably replied "your configuration file makes no sense because X Y and Z". A quick look at the documentation and I have a working build ...

Yay, my tests failed!

... which fails correctly. I expected it to fail, because Windows even with WSL doesn't properly support symlinks. There is apparently a mklink command that works instead of ln -s, but twiddling my code to actually make it work on Windows is beyond the scope of this article :-)

For a real-world example of how having Appveyor's Windows testing available let me fix a real bug before releasing my code, see this commit where I started testing on Windows (and it failed) and this sequence of commits where I first made the tests spew out some useful diagnostics on failure, and then flailed around a bit to make them pass. All, of course, while still automatically testing on Unix-a-likes to make sure I didn't break them in the process.


I hope you've found this series of articles useful. You should now be able to add automatic testing for your code on the four most commonly used platforms.

There are some others I'd like to test on but haven't yet found suitable hosts. I'd like to test on other hardware platforms, not just amd64 - I want to test on 32-bit as well as on platforms like ARM.

And I'd like to also test on VMS. I don't really have enough VMS users for it to be the slightest bit useful, but it would be an interesting challenge.

Discussion (0)

Editor guide