What is the best UI automation tool to use?

It’s a common question I see asked: What is the best UI automation tool to use? Similar to questions such as:

• What’s better selenium or cypress?
• Is Java better than Python
• Should I use codeless tools?

So I quickly wanted to share with you, based on my experience of working with many different UI automation tools, which UI automation tool I know is best…

None of them!

What! Are you saying that they are all rubbish and we should stop using them entirely? No. I’m saying it’s madness to try and name a single tool as the best when every project we are involved is different.

We can spend days debating browser support, marketing material, personal success stories, or whether it can fix your selector problems but in the end, none of that matters if you’re ignoring what is going on around you. Your team, your product and your project should dictate your choice in tools, rather than what some tech guru showed off at a conference recently, or whatever latest and greatest tool is being shared on social media.

Make your own decision based off of the research you do of your current working context.

What criteria should you use to select an automation tool?

Picking a tool based on its popularity or how often it gets mentioned in clickbait articles puts you at risk of failing (sometimes you’ll get lucky, but it was luck not good advice that got you there). So what information should we seek to help inform us in our tool selection? Here are a few areas that we like to highlight in Automation in Testing that you should consider before picking tools:

Team

Automation should always be owned by the team rather than an individual or a specific role. A team invested in automation will contribute towards automation getting written, building a product that is automation-friendly and using automation in a way that genuinely helps the team. So to get that initial buy-in, you want to pick a tool that works for everyone. This means understanding aspects of your team such as:

• What languages the team are comfortable with? - Using a tool that supports a language everyone works with is going to get better buy-in. It’s no good building something in Ruby if you work at a .NET based business.
• What experience do they have with automation? - If the team are new to automation, consider tools that offer low investment and high reward to help highlight it’s value.
• How is your team organised? - Do you work closely with Devs, do you have a culture that supports one another, are you working in an Agile or Waterfall based team? This will impact what you might need to learn and how to share the work with others.

Product

When things go wrong with automation, we can be highly critical of the tools we’re using. I’ve heard the “I’m using Cypress because Selenium was too flakey” argument too many times to mention. However, I am willing to bet that a fair amount of those failures aren’t because of the tool, but because of the product, it’s automating. Understanding what your product can and cannot do and how it can and cannot be automated is key when choosing a tool:

• What technologies does your product use? - Some technologies are built in a way that makes them easier to automate. Just ask a UI developer who has to unit test a React component compared to a JQuery script!
• What type of product is it? - The domain of your product is going to matter a lot. Consider mobile and the many ways you can build a mobile app. Native, hybrid, web or responsive. Whilst they may have some crossover there are also specific challenges to consider. Understanding those challenges will help hone your choice.
• How is it built? - Is the product a huge black box that can be accessed by its user interface only or can it be broken down and automated in isolated chunks?

Project

Once I worked on an automation framework that was a work of art. Beautiful reporting dashboards, strong stable checks that used fancy waits, clear layout of code that clearly showed what their purpose was, etc. It was amazing. Only one problem. The project was only two weeks long, and the work I did was never used again after that. What a waste of time!

When choosing tools and implementation it’s important to be aware of project details such as:

• Project deadlines - When is it going live? Do you have time to automate anything?
• Project budget - How much money do we have to spend on tools and training?
• Project planning - Is there space to experiment with tools? Or is the team?

Context is key

These are just a few items to look out for, but I feel it helps highlight that each project and product is different from the other. Specific tools will fit more comfortably within specific contexts. But understanding that fit relies on observing and questioning our contexts to see what’s right for us. It’s not based on picking a tool that has appeared in a bunch of top 15 lists.

So the next time you see someone asking what the best UI automation tool is to use, don’t just throw out the name of a tool you like, but don’t just say ‘It depends’. Instead, ask them what can they tell you about their context.