This article will describe a common approach that could be very helpful for UI test automation of processes that have some degree of associated uncertainty. This approach was tested practically during test automation for real application with good results. But in general, if it is possible to automate such uncertain process by non-UI means it is highly recommended to do so, due to a variety of issues like execution performance, UI automation tools stability, overall level of complexity associated with UI test automation, etc.
For the matter of this article, we'll be automating part of an imaginary online service that allows assigning different kinds of exams (please remember that this application is imaginary and some of its parts aren't covered intentionally as they aren't related to the described topic). Our goal here is to automate the process of taking an exam providing random answers to all questions that are used in the exam.
The process of taking an exam is organized in the next way - each examinee should receive an email with a unique link to the exam. This link will navigate to the page with the description of the exam and "Start exam" button. After "Start exam" button is pressed user redirected to a page with questions. Each exam may consist of one or multiple pages with questions. Each page may have a different set of buttons for navigation:
Single page exam - "Save and continue later" and "Submit answers"
First page of multi-page exams - "Save and continue later" and "Navigate to next page"
Middle page of multi-page exams - "Navigate to previous page", "Save and continue later" and "Navigate to next page"
Last page of multi-page exams - "Navigate to previous page", "Save and continue later" and "Submit answers"
Each page will contain one or multiple questions. Questions may have different answers types which will be represented on UI side as:
- group of radio buttons – only one answer can be provided
- group of checkboxes – multiple answers can be provided
- text input/text area – inputs for free text. Order of questions is random for each examinee, and as a result reading list of questions, that expected to be present in exam, ahead of time won't help us with automation except for the fact that we'll be able to confirm that all expected questions are present in a current exam. Since time constraints are not important for us at this moment, we won't be considering them in this article.
Before we continue I would like to add few notes here:
- all examples in this article are written in C# 8.0, but I won't be using some of its features, like default implementation in interfaces, which, for sure, can be helpful in this scenario, but will make examples incompatible with other languages, that may be lacking such feature.
- this article won’t provide any application-specific logic, so please treat all comments like
/*specific implementation here*/or code likethrow new NotImplementedException("specific implementation here");just like a placeholder for such specific code, as an implementation of such members is not part of this article.
OK, now it is time to write some code.
Since answers are smallest functional parts, let’s implement them first. We need to create common interface for all answers.
public interface ICommonAnswerComponent
{
void SetAnswer(string answer);
}
Text input/text area answers do not have any options to choose from, so such answers will have own interface
public interface ITextAnswerComponent: ICommonAnswerComponent
{
string AnswerText { get; }
}
Answers represented with radio buttons/checkboxes within one question should be treated as one AnswerComponent and since it is expected that there may be more than one option to choose from their interface will look like
public interface ISelectableAnswerComponent: ICommonAnswerComponent
{
List<string> Answers { get; }
List<string> SelectedAnswers { get; }
}
From this point it will be possible to implement 3 concrete classes for answers. We will utilize custom wrapper for LoadableComponent. If you would like to get more familiar with usage of LoadableComponent - please refer to this article
Here is an example implementation of a such wrapper
public abstract class Loadable<T>: LoadableComponent<T> where T: Loadable<T>
{
protected readonly ISearchContext searchContext;
protected readonly TimeSpan timeout;
private IWebDriver driver {
get/*current instance of IWebDriver should be returned here*/;}
protected override void ExecuteLoad()
{
var wait = new WebDriverWait(this.driver, this.timeout);
wait.IgnoreExceptionTypes(
new[] { typeof(NoSuchElementException),
typeof(StaleElementReferenceException),
typeof(NullReferenceException) });
wait.Until(driver => this.EvaluateLoadedStatus());
}
protected Loadable([Optional]ISearchContext searchContext, [Optional] TimeSpan? timeout)
{
this.searchContext = searchContext ?? this.driver;
this.timeout = timeout ?? TimeSpan.FromSeconds(5);
Load();
}
}
With Loadable in place it would be great to enforce each answer component’s constructor to set component’s search context and load timeout.
public abstract class GenericAnswerComponent : Loadable<GenericAnswerComponent>
{
protected GenericAnswerComponent(ISearchContext searchContext, TimeSpan? timeout) :
base(searchContext, timeout) { }
}
Now it is time to implement concrete classes for answers
public class TextAnswerComponent: GenericAnswerComponent, ITextAnswerComponent
{
public void SetAnswer(string answer) { /*specific implementation here*/ }
public string AnswerText { get /*specific implementation here*/; }
protected override bool EvaluateLoadedStatus() =>
throw new NotImplementedException("specific implementation here");
public TextAnswerComponent(ISearchContext searchContext, TimeSpan? timeout) :
base(searchContext, timeout) { }
}
public class SingeOptionAnswerComponent : GenericAnswerComponent, ISelectableAnswerComponent
{
public void SetAnswer(string answer) { /*specific implementation here*/ }
public List<string> Answers { get /*specific implementation here*/; }
public List<string> SelectedAnswers { get /*specific implementation here*/; }
protected override bool EvaluateLoadedStatus() =>
throw new NotImplementedException("specific implementation here");
public SingeOptionAnswerComponent(ISearchContext searchContext, TimeSpan? timeout) :
base(searchContext, timeout) {}
}
public class MultipleOptionAnswerComponent : GenericAnswerComponent, ISelectableAnswerComponent
{
public void SetAnswers(List<string> answers) => answers.ForEach(this.SetAnswer);
public void SetAnswer(string answer) { /*specific implementation here*/}
public List<string> Answers { get /*specific implementation here*/; }
public List<string> SelectedAnswers { get /*/specific implementation here*/; }
protected override bool EvaluateLoadedStatus() =>
throw new NotImplementedException("specific implementation here");
public MultipleOptionAnswerComponent(ISearchContext searchContext, TimeSpan? timeout) :
base(searchContext, timeout) { }
}
To implement class for QuestionComponent, we need to create a mechanism that will allow us to figure out what exact answer type each question have. For this matter, we will utilize the fact that if component which was derived from Loadabale will not be loaded within specific time exception will be thrown. The most simple and straightforward approach here is to try to create each type of answer and one that will not throw an exception will be the correct one and can be returned. While this is the most universal approach that will cover any case, it has one major drawback – performance. Some optimizations could be done here, but such improvements have dependencies on DOM of the specific application and are outside of the scope of this article. But we may do another trick, by collecting some stats, we can improve average performance by trying to instantiate first those types that can be found in the application more often than others - time to create AnswerFactory
public static class AnswerFactory
{
public static ICommonAnswerComponent InstantiateAnswerComponent(IWebElement answerContainer)
{
ICommonAnswerComponent result = null;
foreach (var answerInstantiatorFunction in answerInstantiatorsFunctions)
{
try
{
result = answerInstantiatorFunction(answerContainer, answerComponentLoadTimeout);
break;
}
catch (LoadableComponentException e)
{
continue;
}
}
if (result == null)
{
throw new Exception("Unable to continue. Question doesn't have any known answer types.");
}
return result;
}
private static TimeSpan answerComponentLoadTimeout => TimeSpan.FromMilliseconds(500);
//Order of records defined statistically in a way that answer type that appears more frequently is
//closer to the beginning of the list. This can be changed by wrapping instantiator function into
//any data type that will add weight functionality. Here is example implementation using tuple –
//(int Weight, Func<ICommonAnswerComponent> InstantiatorFunction)
//and then elements can be ordered as
//answerIntantiatorsFunctions.OrderBy(e => e.Weight)
private static List<Func<IWebElement, ICommonAnswerComponent>> answerInstantiatorsFunctions =>
new List<Func<IWebElement, ICommonAnswerComponent>>
{
(answerContainer) => new SingeOptionAnswerComponent(answerContainer, answerComponentLoadTimeout),
(answerContainer) => new TextAnswerComponent(answerContainer, answerComponentLoadTimeout),
(answerContainer) => new MultipleOptionAnswerComponent(answerContainer, answerComponentLoadTimeout)
};
}
Now we are fully prepared to create class for question component
public class QuestionComponent : Loadable<QuestionComponent>
{
private By questionLocator => By.XPath("specific XPath here");
private IWebElement questionElement => searchContext.FindElement(questionLocator);
public string Question => questionElement.Text;
private By answerLocator => By.XPath("specific XPath here");
private IWebElement answerElement => searchContext.FindElement(answerLocator);
public ICommonAnswerComponent AnswerComponent =>
AnswerFactory.InstantiateAnswerComponent(answerElement);
protected override bool EvaluateLoadedStatus() =>
throw new NotImplementedException("application specific logic should be implemented here");
public QuestionComponent(ISearchContext searchContext, [Optional] TimeSpan? timeout) :
base(searchContext, timeout) { }
}
Well, answers and questions are implemented and now it’s time for pages. Each page should have questions so as usual, we’ll start with an interface for a page
public interface IExamPage
{
List<QuestionComponent> Questions { get; }
}
Our scheme of navigation allows users to navigate forward and back, save answers without submitting them, and submit answers for the final check. Here are interfaces that will describe all these interactions
public interface INextPageNavigation
{
IExamPage NextPage();
}
public interface IPreviousPageNavigation
{
IExamPage PreviousPage();
}
public interface ISubmitablePage
{
//In a real-life scenario specific page should be returned here. We’ll ignore it.
void Submit();
}
public interface ISaveablePage
{
//In a real-life scenario specific page should be returned here. We’ll ignore it.
void Save();
}
And now it is time for interfaces for each specific exam page
- Single page exam
public interface ISinglePageExam: IExamPage, ISaveablePage, ISubmitablePage {}
- First page of exam
public interface IFirstExamPage: IExamPage, ISaveablePage, INextPageNavigation {}
- Middle page of multi-page exam
public interface IMiddleExamPage: IExamPage, IPreviousPageNavigation, ISaveablePage, INextPageNavigation {}
- Last page of exam
public interface ILastExamPage: IExamPage, IPreviousPageNavigation, ISaveablePage, ISubmitablePage {}
Since all exam pages are implementing IExamPage interface, which determines how to access list of questions on each page, there is no need to have it implemented in each specific class. At the same time, we cannot be sure that each exam page will have the same DOM, so we will start with common ancestor whose task is to implement IExamPage and at the same time enforce each specific exam class to describe locator for questions records in DOM of this particular page. With this in mind, we will create an abstract class - GenericExamPage. It should be derived from Loadable - as we want to be sure that each exam page will be loaded before we will be able to proceed with taking the exam.
public abstract class GenericExamPage : Loadable<GenericExamPage>, IExamPage
{
protected abstract By questionsLocator {get;}
private IList<IWebElement> questionsContainers => searchContext.FindElements(questionsLocator);
public List<QuestionComponent> Questions => questionsContainers
.Select(e => new QuestionComponent(e))
.ToList();
protected GenericExamPage(TimeSpan? timeout) : base(timeout: timeout) { }
}
Now everything is set for concrete classes that will be representing specific exam pages. Since we cannot be sure about how specific DOM would be built on each page, Save() method from ISaveablePage interface will be implemented by each class:
- Single page exam
public class SinglePageExamPage : GenericExamPage, ISinglePageExam
{
protected override By questionsLocator { get/*specific locator here*/; }
protected override bool EvaluateLoadedStatus() =>
throw new NotImplementedException("specific logic here");
public void Save() =>
throw new NotImplementedException("specific logic here");
public void Submit() =>
throw new NotImplementedException("specific logic here");
public SinglePageExamPage(TimeSpan? timeout) : base(timeout) { }
}
- First exam page
public class FirstExamPage : GenericExamPage, IFirstExamPage
{
protected override By questionsLocator { get/*specific locator here*/; }
protected override bool EvaluateLoadedStatus() =>
throw new NotImplementedException("specific logic here");
public void Save() =>
throw new NotImplementedException("specific logic here");
public IExamPage NextPage() =>
throw new NotImplementedException("specific logic here");
public FirstExamPage(TimeSpan? timeout) : base(timeout) { }
}
- Middle exam page
public class MiddleExamPage : GenericExamPage, IMiddleExamPage
{
protected override By questionsLocator { get/*specific locator here*/; }
protected override bool EvaluateLoadedStatus() =>
throw new NotImplementedException("specific logic here");
public IExamPage PreviousPage() =>
throw new NotImplementedException("specific logic here");
public void Save() =>
throw new NotImplementedException("specific logic here");
public IExamPage NextPage() =>
throw new NotImplementedException("specific logic here");
public MiddleExamPage(TimeSpan? timeout) : base(timeout) { }
}
- Last exam page
public class LastExamPage : GenericExamPage, ILastExamPage
{
protected override By questionsLocator { get/*specific locator here*/; }
protected override bool EvaluateLoadedStatus() =>
throw new NotImplementedException("specific logic here");
public IExamPage PreviousPage() =>
throw new NotImplementedException("specific logic here");
public void Save() =>
throw new NotImplementedException("specific logic here");
public void Submit() =>
throw new NotImplementedException("specific logic here");
public LastExamPage(TimeSpan? timeout) : base(timeout) { }
}
OK, since all possible exam pages types are implemented now, we need to implement a mechanism that will return correct page type for the currently loaded page. We will use the very same approach as with answers – try to load page, if load exception will be raised, we’ll try to load the next one. And same as with answers, this is the most “bulletproof” approach, but it is also the slowest one. Real-life applications may have some properties based on which we can figure out correct type of specific page much faster, but in this example, performance concerns will be put aside, and we will go with generic solution. Same as before it makes sense to measure the probability of appearance of each specific page and sort instantiation functions accordingly.
public static class ExamPageFactory
{
public static IExamPage InstantiateExamPage()
{
IExamPage result = null;
foreach (var examPageInstantiatorFunction in examPageInstantiatorFunctions)
{
try
{
result = examPageInstantiatorFunction();
break;
}
catch (LoadableComponentException e)
{
continue;
}
}
if (result == null)
{
throw new Exception("Unable to continue. None of exam pages was instantiated.");
}
return result;
}
private static TimeSpan pageLoadTimeout => TimeSpan.FromSeconds(10);
//Order of records defined statistically in a way that page that appears more frequently is closer to
//the beginning of the list. This can be changed by wrapping instantiator function into any data type
//that will add weight functionality. Example implementation using tuple –
//(int Weight, Func<IExamPage> InstantiatorFunction)
//and then elements can be ordered as
//examPageInstantiatorFunctions.OrderBy(e => e.Weight)
private static List<Func<IExamPage>> examPageInstantiatorFunctions =>
new List<Func<IExamPage>>
{
() => new MiddleExamPage(pageLoadTimeout),
() => new FirstExamPage(pageLoadTimeout),
() => new LastExamPage(pageLoadTimeout),
() => new SinglePageExamPage(pageLoadTimeout)
};
}
With the ability to create pages, questions, and their answers, dynamically, it is time to put it all together and create some code that will take an exam without knowing its exact structure, prove random answers to questions and return information what answers were provided to what questions
public class Test
{
private IWebDriver driver { get /*return current IWebDriver instance*/; }
public Dictionary<string, string> TakeExamUsingRandomAnswers(string examUrl)
{
var result = new Dictionary<string, string>();
driver.Url = examUrl;
_ = new StartPage().StartExam();
var examPage = ExamPageFactory.InstantiateExamPage();
while (true)
{
var questions = examPage.Questions;
foreach (var question in questions)
{
var answers = getAnswerOptions(question.AnswerComponent);
var answerIndex = new Random().Next(0, answers.Count);
var finalAnswer = answers[answerIndex];
question.AnswerComponent.SetAnswer(finalAnswer);
result.Add(question.Question, finalAnswer);
}
if (examPage is ILastExamPage)
{
break;
}
else
{
examPage = ((INextPageNavigation)examPage).NextPage();
}
}
(examPage as ILastExamPage).Submit();
return result;
}
private List<string> getAnswerOptions(ICommonAnswerComponent answerComponent) =>
(answerComponent) switch
{
_ when answerComponent is ISelectableAnswerComponent =>
(answerComponent as ISelectableAnswerComponent).Answers,
_ when answerComponent is ITextAnswerComponent =>
new List<string> { "random answer 1", "random answer 2",
"random answer 3", "random answer 4" },
_ => throw new NotSupportedException(
$"Parameter of type {answerComponent.GetType().Name} is not supported.")
};
}
And this is how we can implement UI automation of processes with some degree of uncertainty. It is quite clear that this approach cannot be used everywhere, to some extent given scenario was deterministic and for sure there will be plenty of other situations where levels of uncertainty may be much higher. But in general, approach to similar problems still will be pretty much the same – find and describe all entities that can be determined, accurately compose them and create a correct workflow that will allow you to utilize previous steps to solve the main problem.
Top comments (0)