DEV Community

Discussion on: 🚀 Uilicious Snippet - Capture, Replicate and Share website bugs 🐞 (without hard-coding stuff!)

Collapse
 
picocreator profile image
Eugene Cheah

We are not related to either entities 😉

Our testing scripts interface directly with the webdriver protocol. And was written from scratch in JS to implement all its various functionalities using webdriver.

One of our main goals was to make it easy enough to use that anyone, including non programmers, and especially those who managed to learn excel, could use for 90% of test case, simulating user behaviour as much as possible. While keeping all programming functionalities available for those hard to test parts.

CodeceptJS - was unfortunately heavily design around using promise chains, a concept which is hard to teach many programmers properly, let alone non programmers. However it does serve as one of the initial inspiration for our test language.

Cypress - focuses on chrome based testing, being embedded inside the browser itself. On the downside, while it means lack of other browser testing such as IE11, it has one of the fastest! test speeds without any of the webdriver overheads.

Hopefully that answers it clearer 😁

Collapse
 
entrptaher profile image
Md Abu Taher

Cool!

  • How long did it take to make it from scratch? or, Did you make from scratch?
  • What problems did you face?

Any story to share?

:D

Thread Thread
 
picocreator profile image
Eugene Cheah

About 2+ years ago - the initial prototype of the language took about 6 months to make from scratch, a very large part of it done by my co-founder @shiling .

To be fair, webdriver protocol in itself is already a miracle. Which is what we depend on.

We then made our own JS test library that gets injected into each browser, to scan the website, which then communicates back to our server to when performing specific steps (like button presses)

It, however had one very big catch, it only worked only on chrome, and on about 50% of the website frameworks out there. Unless you were one of those who followed the HTML guidelines perfectly with aria labels and all.

(It also came along with the very terrible v1 of our backend infrastructure. Where we were burning cloud credits by giving every users dedicated servers - but that's another story... haha)

Fast forward 2 year. And as more time was spent into refining the engine, we managed to cover all major UI framework and design patterns within our system. So as much as there are a 100+ ways to style a button, we would detect and test accordingly.

At the same time, coverage been extended to Firefox, Safari, Edge, and IE 11.

And we have long since ran out of cloud credits, and restructure our entire system as a shared infrastructure (except for those deploying on-premise)


Problems faced? Oh boy, many. The earliest was getting our test engine to deal well with human language and ambiguity, especially how it decides which login button to click on if there are 5 of it on screen.

Alot of work is put into making our system be able to understand user testing patterns, and acting accordingly. Without an AI. More is covered by my co-founder here : dev.to/uilicious/taking-the-horror...

From then on - lots of infrastructure issues..

Especially with our underlying technology selenium/webdriver. It's hilarious and sad, how in some selenium testing lessons, they teach users to run one test 4 times, and if any 1 pass. Consider it as a pass. And that's due to the huge failure rate of the system by default. Which is honestly not acceptable. So lots of effort is then put into configuring selenium to be stable enough for reliable use.

It also doesn't help when all the browser vendors keep breaking their webdriver specs. Amusingly, IE11 now is increasingly getting more love from our team, for never changing haha.


Oh, and to find out after you thought you got all 100 ways of testing a login page working... to find a website that does something completely different 😂

Thread Thread
 
entrptaher profile image
Md Abu Taher • Edited

O_O This was a great story, it should've been shared on actual post.

I certainly faced almost all problems and had to tackle in different ways you have mentioned while working on various web automation projects. Even till now I haven't finished solving all those public problems stated on stackoverflow.

Thread Thread
 
picocreator profile image
Eugene Cheah

Haha, thanks, it certainly will be fleshed out into its own post one day. Especially once we dive into the details.

Its great to see others tackling this problem as well, as I seen many have given up on automation out of frustration sadly. We have yet to solve most of our problem like those you face as well after all, part of the reason why we consider "IE11, Safari, and edge" beta right now.