WordPress Integration Testing Overview and Issues
Starting almost a year ago I started experimenting with the “official” WordPress plugin testing tools while working on our Ratify plugin. I’m not a glutton for punishment but I do like a good challenge and, for the most part, I wasn’t let down.
The first half of this year we had another, ripe opportunity to work with the testing framework. We were building a custom publishing solution based on WordPress. As a custom solution, we had a few plugins to write that would allow the authors and editors to (more) easily manage their content in addition to tracking your every breath while on their web site. As a team of almost 10 developers at one point, it was crucial that we all knew how to write and run tests.
A big part of my job is developer training. I decided to create a practice plugin that we could use for training developers how to use the framework and write good tests. The plugin would allow you to assign a “countries” taxonomy to any standard or custom post type. We wrote a general description of the plugin and then a series of tests we thought would help us do this as TDD. The view is always better from on high.
Coding Environment and Tools
We use vscode with the following extensions (more, actually, but these are pertinent to this project):
- EditorConfig for vscode
- Intelephense (PHP intellisense for Visual Studio Code)
- PHP CodeSniffer for Visual Studio Code
Although I am a life-long BBEdit user (and occasional contributor), there are certain features in vscode that make doing this kind of development a joy (in-editor PHPCS feedback, code completion via Intellisense, multicursor editing in particular). Do note: I’m writing this post using MarsEdit but only because it’s already configured to post to my blog directly.
WP-CLI and Laravel Homestead
We use Laravel Homestead for nearly everything we do here at Secret Source. Yes, we know, Docker is the future, but we like Homestead. Homestead may require a little tweaking to use with WordPress but normally it just works.
We use wp-cli to generate the testing framework / harness (wp scaffold plugin-tests). It works well enough but this is where we start to struggle and I’m going to go into detail here because Josh Pollock was wondering what we were struggling with.
Issues with the Testing Framework
In general, the testing framework works and I can’t imagine how much effort I would have to put into it to recreate it myself so, kudos to the WordPress team for just producing this. That said, as systems get more complex, it becomes even more important to have a testing framework in your tool belt so, IMHO, more effort should be put into making the framework as easy to use as possible.
Installation and Setup Seems Difficult
The wp-cli scaffolding consists of a few files you’ll need to get the full benefit of the testing framework. It does not include the actual tools or binaries you’ll need. For example, in order to use the framework you need phpunit, and not just any version, but, in our experience, anything higher than 6.5 won’t work. Subversion is also required, so, sudo apt install subversion. I have to believe that these two steps could be encapsulated in a composer.json file but they aren’t, so two additional manual steps are required to get your environment configured.
Initializing the Database Issues
When you install the test scaffolding, it includes a bash script (bin/install-wp-tests.sh) that tries to set up the whole environment, including the database that is used when running the tests. The script, however, could be improved. On more than one occasion I’ve found myself having to manually fiddle with MySQL (deleting tables) and delete all references to WordPress in /tmp/wordpress in order to get the script to run properly. Also, at the end of the script there is a scary MySQL message about including the password on the command line. It can be ignored but there are solutions to minimize this kind of misdirection.
I would start by isolating dependencies as much as possible and including a little more sanity checking and such. Since this is written in bash, I could do it myself, but I have a feeling it should probably be some sort of PHP script, maybe even installable via composer?
There is no Way to Test Plugin Installation
As far as I can tell… the framework doesn’t allow you to test the installation hooks (installation seems to be bypassed during bootstrapping). As (bad) luck would have it, our practice plugin has a very specific requirement that it not proceed under certain circumstances, but we’re unable to automate this test.
Issues with (Vscode and) PHPCS
I’m not a big fan of vscode but it does have some very compelling features including a PHP CodeSniffer extension that helps you write properly formatted code and code, in theory, is less complex. This is a feature I really want as I need all the help I can get!
There are really two issues:
- The WordPress plugin test scaffolding comes with a different set of code sniffs (not sure what they are called) than the WordPress core, which is reasonable, but if you don’t know how to phpcs in general, this is going to take a fair amount of investigation to figure out, as was our case.
- The vscode phpcs extension has a per project installation option (if you don’t mind installing phpcs via npm) but getting vscode to find the binary can be tricky and then configuring phpcs to look at the right sniffs can be even harder. We managed to get it set up and working, mostly, but if you’re looking to learn how, I suggest you look at the wprig.io github repo for an example of how to do it.
Difficulty Defining Tests
Writing good tests for TDD is an art, plain and simple. If you ever find someone who is genuinely good at it, latch on to her and don’t let go. Learn as much as you can from her, and if you’re her supervisor, give her the freedom (and time) she needs to practice her art.
The very first test we wrote for our practice project turned out to be untestable using the WordPress testing framework. For the curious, we wanted to the plugin to fail to install gracefully if a taxonomy named “countries” already existed. Due to how the test harness is invoked, the plugin skips the normal installation processes, which is when we were planning on testing for the existence of said taxonomy. I won’t detail the time it took us to figure this out but I will say it was not insignificant and there was little documentation around this subject. We just read the bootstrapping code.
The second test makes some assumptions about what the environment will contain, what data will be available in the environment. This is fine but it means that we have to do a fair amount of mocking or even creating of data prior to running the test to be able to test things. This not a huge problem so long as the tests are relatively short. It seems to me that there ought to be a more direct, and faster way of testing this aspect but I haven’t found one yet.
The Future
Thanks to other, motivated WordPressers the future still looks kind of bright for WordPress and testing. My plan is to get this kinks resolved in the next couple of weeks. I will be publishing our sample plugin eventually so follow me on Twitter if you’re interested in seeing the final result.
The post Testing Our Limits, and Patience, in WordPress appeared first on Chicago IT Systems, S.L..
Top comments (0)