DEV Community

loading...

Versions and Testing

Jesse Phillips
Senior Quality Assurance (SDET) ¶ Avid hobby D programmer ¶ Telling people what to do because I am right.
・3 min read

Another version post, but I'm going to focus on some challenges for testing.

Pattern one I see: tags specify the version. These tags can show up on any branch and they can even encode which environment CD installs to. This is nice when you only want to test at predefined times. If you have all testing automated this is not a bad choice.

I work in a less than ideal world and I'm promoting QA getting their hands on changes sooner. But there is a catch. QA is usually expected to go through and document their test efforts noting version and environment. Similarly I want to know the features tested exists in a particular version. So let me explain what I have tried.

Git Hash

Every commit produces a unique hash to representing the current state of the repo. This is the ultimate in version identification, however there is no encoded info about its relation of one hash and another.

This makes the hash important, but not good as a version number.

Tags

Git allows for a named reference to a commit and calls it a tag. It can basically be anything but choosing 'v1.8.6' allows for creating a version tree.

There is only a versioned build when a tag is added. The CI installs each commit push to an environment and runs tests, this isn't a big problem, but if the environment is then used in some manual verification the dev pushes an update and states the issue was fixed.

Ideally this would be reviewed in short order. However it may not get looked at until after development pushes several additional changes. Now do these builds include the original item to test? If you check version it will be the same as the others.

Partial Tag

Git has a command which uses the tag to describe the current state.

$ git describe

v1.8-6-g2414721

This provides the latest tag and the number of commits since that tag. If we tag with just the first two numbers and combine it with the count we get the nice 3 digit version.

Unique?

No. This will duplicate versions as you branch, however for a single branch with protection it will be unique.

It also causes two builds with different versions if you don't push the tag with the commit.

Semantic Meaning

This breaks semantic versioning, well sort of. You still have control over when the major and minor tags are added. You could add the new tag to a branch, but that would mean you know which feature is coming in first.

The other side is if I'm testing a build and approve it for production a build off a new tag means I'll need to have at least some additional testing or review.

Retrieving Source Code

This is actually much harder because tooling isn't built to query history like this. Trying to count the number of commits since a tag is hard at best, especially with merges.

I generally use the build pipeline for recent builds. Otherwise keeping the hash available is valuable for this.

Semantic Versioning

Revisiting this topic we can take advantage of some additional semantic meaning.

Rather than placing the build count on the patch version, it can be added to the pre-release.

v1.8.6-after.14+g2414721

Now we have a version which is incremental, controlled patch versions, and git hash meta data.

I have chosen to mark the pre-release as 'after' to state this build is newer even though semantic parsing will claim it is older. It would be nice to have this concept accepted into semantic scheme.

Similarly if the tag is

v2.0.0-alpha

The commit count can be added to the alpha count

v2.0.0-alpha.14+g2414721

Conclusion

Using the git describe for building in the version allows for changing versions for every commit.

Semantic versioning provides additional meta information to include additional details about the git commit.

Discussion (0)