DEV Community

Cover image for Nice tests to run in CI before deploying any website
Andy Li
Andy Li

Posted on

Nice tests to run in CI before deploying any website

Since I'm the only dev behind my startup, Giffon 🎁, I've to take care of all the technical things from development to deployment. Though I've been following best practices and be careful about edge cases, still I don't trust the code I wrote very much.

I added a number of tests in CI to check my website before deployment, and they have caught my mistakes many times. The cool thing is, many of those tests are generic enough to be applicable to any website, so you should consider using them too!

1. Markup validation

The first easy low-hanging fruit is HTML (and CSS/SVG) validation. Most HTML templates do not enforce valid HTML, so it's quite easy to output documents with mismatched tags, which can be a pain to spot! Giffon uses ReactDOMServer, which prevents tag mismatches, but the validation still caught me many times about putting block-level elements (e.g. <ul>) in <p>, and also missing alt attribute in <img>.

Giffon's CI uses the html5validator command line tool against a development server. It's basically calling the commands as follow:

html5validator --html http://localhost:3000/every/page
html5validator --css www/css/*.css
html5validator --svg --errors-only www/images/*.svg
Enter fullscreen mode Exit fullscreen mode

2. Detecting browser console error messages

The second generic error detection is to check the browser console. It reports JS run-time errors as well as broken image references and some invalid HTTP respond headers too.

Giffon's way to do it is to use Selenium web driver, navigates to each page and calls driver.get_log("browser"), asserts that there is no log.

3. Detecting horizontal scrollbars

Most websites don't use horizontal scrollbars. It's annoying (especially on mobile) when one appears due to layout bugs.

Again, Giffon's CI makes use of Selenium. The CI navigates to each page and detects the present of horizontal scrollbars by asserting the following code returns zero:

driver.execute_script(
    "document.scrollingElement.scrollLeft = 1;" +
    "return document.scrollingElement.scrollLeft;"
)
Enter fullscreen mode Exit fullscreen mode

4. Spellchecking

Who doesn't make typos? It may not be critical, but having typos in your website does give a bad impression to visitors.

Giffon's CI uses spellchecker npm library. Giffon has its UI text strings isolated when I implement multi-language support, so it's easy for me to iterate over all the strings and call SpellChecker.checkSpelling(str).

If you don't have your strings isolated, you may save the html output as a file and use the good old aspell cli to list all misspelled words as follows:

cat path/to/my.html | aspell --mode=html list
Enter fullscreen mode Exit fullscreen mode

Any others?

Do you run similar tests in you CI pipelines? Let me know if you have any good ones to share!

Top comments (14)

Collapse
 
remibruguier profile image
Rémi BRUGUIER

Those are great. About the console messages: do you use something in your CI to remove all the console.log you may have used during development and forgot to get rid of? I personally use UglifyJS to remove the pesky console logs...

Collapse
 
andyli profile image
Andy Li

I treat any detected console message as an error in CI, so there wouldn't be any console.log() left. Personally I don't care left-over logs that much since they wouldn't be visible to normal users... Just don't log with swear words... ;)

Collapse
 
patrickt010 profile image
Patrick Thompson

Great list Andy. A few things that we test in CI as well are:

This helps our team move fast, and avoid breaking things. 🍻

Collapse
 
therealkevinard profile image
Kevin Ard

Love the horizontal scroll test! At least in part because those things are a pet peeve of mine lol.

If you have a large, uncertain inventory (eg: you manage several hundred client marketing sites or something like that), there's a good pre-script using sitemap-generator to crawl the site before test. I like to run the most basic sanity tests (response codes, valid markup, things like that - and now, horizontal scroll 😅) against every page, but you can't always know every page.

  1. Crawl sitemap with node
  2. Probably want to update that since you have it.
  3. Parse the fresh xml map into json
  4. Iterate the json with your every-page suite.
Collapse
 
andyli profile image
Andy Li

Using a sitemap is a good tip! Thanks for sharing :)

Collapse
 
abahgat profile image
Alessandro Bahgat

Great post Andy!

I also found it very valuable to have screenshot testing and visual review included as a step of my CI pipeline.
I documented my setup in this post if you would like to check it out.

Collapse
 
andyli profile image
Andy Li

Nice article! I heard about screenshot regression testing a few times in the past but haven't tried that yet. I guess I should!

Collapse
 
sergsoares_15 profile image
Sergio Soares

Great article, useful things to CI for WebPage.

Something that i want to use too is add Google Ligthouse at CI using github.com/GoogleChromeLabs/lighth....

We can't leave a commit reduce our product quality =)

Collapse
 
andyli profile image
Andy Li

That's an interesting idea. Thanks for sharing :)

Collapse
 
chrisshort profile image
Chris Short

Are there any technical details on implementing the horizontal scroll bar? That would be super handy.

Collapse
 
andyli profile image
Andy Li

Not sure if it's necessary for everyone, but I have a 5-second wait until I treat the present of horizontal scrollbars as errors. I think it's due to browser loading (of CSS and images etc.).

You can take a look at the exact code I use. It's written in Haxe, but it should be easy to understand since it's very similar to TypeScript/Java.

Collapse
 
jinksi profile image
Eric Jinks

This is amazing advice 👏
I’ll be keeping this bookmarked for future reference. Thanks!

Collapse
 
charlyjazz profile image
Carlos Azuaje

Amazing...

Collapse
 
tyrrrz profile image
Oleksii Holub

Great list!