Key Parameters while choosing an Automation Testing Tool

Hey all,

Hi Techies! How are you doing? Everything alright? How is the software industry treating you?

I hope it is keeping you all at the tip of your toes, as always. At least here in India it is! The industry is evolving at a blazing pace. There are start-ups propping up in every nook and corner. There are huge rounds of funding for those which show promise & all my buddies working in such start-ups are earning big bucks. Older more established companies are either withering away or are reinventing themselves to stay relevant & competitive.

Software Development  has moved on from the ‘Steam Engine phase’ to the ‘Bullet Train phase’. Cloud Technologies like AWS, Azure are enabling companies to set shop quicker,cheaper,lighter & to scale their systems more easily. Release times are getting shorter and shorter.To aid these faster release cycles there is a new set of Software Ninjas called Devops Engineers who are bringing a paradigm shift to the way the industry was working uptil now. There is a cultural and ideological change happening and conventional boundaries of Dev, QA and Support are slowing fading away for the better.

Coming straight to the subject of this post…

There is a huge necessity of software automation for various tasks & processes in today’s industry.Each company needs to have a robust automation framework in place. This is to match up to the speed of software development and test out the iterative builds. A quick reliable feedback loop on the code quality thus ensures moving to a ‘Continuous Delivery’ model where builds are deployment ready. It is close to impossible to test the quality of every single release with manual testing alone.I’m sure you’ll agree with me on this. Hence automation is the pressing need of the hour.

Automation requirements are massive in the industry – from UI automation for Desktop Applications,Browsers and Mobiles, to API level automation for backend architecture, to machine level automation for various tasks. There are a vast many automation tools out there in the market which can be used for different automation requirements. In my 5 years of experience in the Software Industry, I’ve come across many instances wherein I’ve had to select a tool for a type of automation & I’ve always been spoilt for choices. I had very little time to decide on the tool & design my automation framework. To add to this, an engineer is always expected to deliver fast tangible results. Simply put, you have the opportunity to make it or break it.

It becomes very important that you make a quick but well balanced decision while selecting your tool. Else sooner or later you will be in a situation that you have to scrap the tool you selected for more reasons than one. The tools all look similar in the beginning, but on careful reading and research, you will be able to identify the variations in each tool & the limitations of each one.

In this article I would like to share few key parameters to keep in mind while selecting an Automation Tool for your software quality assurance needs :-

1.Ease of setup- setup must be pretty straightforward and all dependencies must be handled internally during installation of the tool.

2.Ease of implementation-the APIs provided by the tool must be simple to understand. There must be good documentation for beginners.

3.Execution time of the scripts- each command in the script must be executed fast and overall time needed to execute a test-suite must be in acceptable limits.

4.Reliability- the tool must be reliable and must not give different results for the same test. Execution times & results must be consistent when run on a day to day basis. The tool must also work as per the documentation. Any deviation from that is a sign of unreliability.

5.Big developer/user community-Very important parameter. The bigger the community of users, the faster you will be able to resolve any roadblocks.

6.Support for issues/upgrades-There must be frequent updates to the tool with bug fixes. This shows that there is an active development team working to make the tool more and more robust. See the release notes too if possible and get an idea of the issues that have been fixed and how quickly they were fixed.

7.Integration with Continuous Integration- The tool must have hooks to be integrated with CI tools like Jenkins,Bamboo,Travis etc. Without that a user must then create his own hooks to plug the automation scripts with these CI tools which again requires time and effort.

8.Ease of maintenance-The tool must have a good overall framework which can handle maintenance related work. Eg: An update version of the tool must not break existing workflows. Google for such issues and if the occurrences of such issues are frequent, then that is a warning signal.

9.Open Source- I would prefer to select a tool that is open-source. Period.

10.Choice of Language- Lastly, select a tool which supports a programming language you are familiar with so that script development can be faster for you.

So there it is! These are the key parameters which came to my mind while selecting automation tools for my workflows. Hope it helps!

As you venture out to select your automation tool, it is best to create a table with all these parameters and evaluate all of your prospective tools against these parameters carefully. Believe me this definitely helps your decision making process and you will come out with a well balanced, level headed decision.

Until next time… From the Silicon Valley of India… Love to all 🙂