Improve your web UI test automation framework

UI web test automation

So you have your Selenium or other UI based test automation framework. Good for you! You have a tool that simulates user behavior and it uses the same interface that your product end client will use. Moreover, you have something that will check regression so manual testers in your project could do more important things (hint: Exploratory testing).

But…

After a few runs, some of your automated checkers got flaky. UI change forced you to do some updates on hundreds of your test scripts. And of course, your test case execution is slow.

UI test automation is not some unicorn that resolves all testing issues. But it is important to have them in your project. In this article, I would like to present ways of improving your UI test automation and general automation in your project.

UI test script usual suspects

Before we talk about improvements, let’s check what is the cause of issues with UI test automation.

1. UI automation is slow and not reliable – you have to wait for the browser to pop up and then for elements to show up. The fact is that the manual testers have to go through the same process, but their mission is to gather new information about the product (testing new features or finding bugs) your automation tool will never do the same work (even with randomized data).
   
   Reliability issues mainly come from Browsers (and sometimes your code implementation) issues.
   
   
2. UI automation tools are hard to maintain
   1. A simple change in UI can break a lot of your recorded or scripted test cases.
   2. Scalability is painful
      
      
3. Test are not readable or usable – People from your team cannot get the required information from your automation, or developers do not use them when creating new app build.

So… How to fix this?

Let’s start with UI automation being slow – you can do little to none to speed up starting a browser or waiting for the element to be visible. But there are other issues that also slow your test case automation.

• Don’t automate everything with UI automation. There is this famous test automation pyramid and it is famous for the certain reason, It’s a guideline for proper test automation on your project. If you have the ability to write your tests in the lower level of this pyramid than do it or let your dev friends do it and you will be only responsible for checking if they are written correctly. Checking stuff in low level does not only fix slowness issue but also makes automation more reliable.
  
  Cool way to deal with automation is to check one part of some complex task with UI automation and the rest of it with Unit/Integration tests. With this method, you gain speed by checking for regression and also you will cover GUI checking.

• Don’t use sleep()/pause() or any other static waiting methods. There is always a way to implement a smart dynamic wait. So for what we are always waiting for? For the elements. When are we waiting for them? Usually after the click action. So the easiest way to have dynamic wait is to create custom click method. With the implicit wait or waiting for an element to have the certain condition before clicking.
  
  If the issue with timeouts appears in few test cases, you can easily just increase your driver default timeout.

• Run your test cases in the already opened browser. Check how long is for the browser to be created and then closed. Like I said before, every second count and creating a new browser instance could take few seconds, so overall equation for only creating a new driver each test is:
  
  Few seconds to create a browser x test cases quantity x runs quantity

How to make a test script more readable?

• BDD or not to BDD? That is the question. Everybody can read test written in BDD, developers, managers, clients, so the easiest answer is to use some BDD framework. The answer is: Yes and No. This always depends on your team needs. If you have Management/Client that want to know what is going into the code so they could read it without any technical knowledge, sure try BDD. But if you don’t need that or you are in a team with software/development experience, try other ways to make your test more readable (or add them to your BDD framework).
  • Let your test case classes names represent your test case (by an id or a name)
  • Let your class test method represent the summary of your test case
  • Let your code methods represent user interaction with your product (e.g. clickSaveButton())

Technically you would end up with:

public class TC647{
  
  @Test
  Public void checkFirstSearchResult(){
    
    visit(“https://www.google.com”);
    fillSearchField(“something”);
    clickSearchButton(“elementLocator”);
    clickFirstSearchLink();
    
    checkFirstResultText(“textToVerify”);
  }
}

The whole test script is more readable and all technical stuff is encapsulated in methods. We don’t have to add additional tools to our test framework. But the solution is always up to you.

Ok, but one thing is a readable code and the second thing is reporting. Here again, everything depends on what you need. There are a lot of reporting tools, but you also can create something yourself. You don’t have use cucumber every time as a reporting tool because you used it “n” times before.

Create your reporting depend on the team and your needs. When your team just has to know what has failed with some stack trace with no fancy HTML report. Use console only as an output of your tests.

And what about Maintenance and scale?

1. First of all, if you have recordable test cases – ditch them. Initially, It’s easy to create this kind of tests, but good luck with maintaining 100+ recorded test cases.
2. Design your test cases with having in the mind that your test case will be run 1000+ times. Make your runs optimal regarding time, every second count!
3. After you grow up to have only scripted test cases the next step of maturity is creating your own framework structure. Let your test cases be in one place, your web driver settings in the other, your runner somewhere else. Thanks to that you have more integrity.
4. You don’t have to run all of your test cases every time – group them in test suites and run suits depending on your needs.
5. Next step is to write code according to some common guidelines. So here you will get a list of solutions used by many regarding the approach towards creating frameworks, they actually work so I highly recommend using most of them.
   • Page object – the most known pattern in UI automation and technically the easiest to implement. The Page object is a model that represents your website page. All pages in your tested products are in isolated classes, thanks to that if one element changes on a web page, you only change your implementation in one place.
   • Strategy – Only software design pattern that I used so far in my frameworks. It is needed when you use more than one implementation of a certain method ( e.g. different navigation or login methods for web and mobile version).
     
   • Data-Driven – If you have one test case that requires a lot of data variety, you know that you would need data-driven approach. Don’t repeat your test cases with little quirks to them, write one test case with data variations.
   • Create tests/framework that could run parallel – This way you would easily save time, but be aware of test cases dependency.
   • DRY (Don’t repeat yourself)
     • DRY with test cases – use Data-Driven approach, code look neat and you don’t have to write the same test cases (with little quirks to it) over and over.
     • DRY with writing test steps – if you have same steps for some test cases – create the method with these steps and use this method everywhere when it is needed.
     • DRY with testing settings – You only have to set up a browser or other test setting once. Don’t repeat this process for each test case.
   • Use others… knowledge. Let me explain. Most of you work in agile teams. And if your team has an agile mindset that means one thing. Everybody in your team is responsible for the quality. So use their knowledge, skills or expertise to achieve your goals. The issue that could take you hours to overcome, with help of others could take minutes.

Conclusion:

1. Create your framework for certain business needs and not for general usage.
2. Don’t rely only on UI test automation.
3. Use patterns.
4. Remember that your scripted test cases will be executed thousand times.
5. Make your scripts readable and accessible for others.
6. Have fun