Test automation should not feel like a nasty video game that feels like it has been created only to smash your spirit and the last vestiges of your team's willpower. Sadly, it often feels like this is the case.
Being at the coalface of software testing, and particularly test automation, we see first hand where the industry is getting it wrong when implementing test automation. Interestingly, for something that should be so intrinsic to modern, digital application delivery programs, most people who fail with test automation the same wrong things.
I think this is because the software testing industry as a whole has not been forced to innovate and, as a result, been forced to adapt to change. So when the "experts" moved from manual testing to automated software testing, they simply
Insanity is doing the same thing over and over again and expecting different results.
Why do I know this secret?
First, it's not a secret. It is an unfortunate consequence of a lack of effective and practical knowledge-sharing about how to move from manual testing to automated software testing. We do this from some of the world's largest companies and, frankly, we do it well; we deliver results, so I do know what I'm talking about.You want proof? Well we recently finished the first round of automation for a customer in Europe where we cut their testing time for a major release down from 4 weeks to 36 hours! What could such results do for you?
Why are you automating your tests?
When discussing test automation with a customer we often rephrase this quesiton to, "do you NEED to automate your software testing?" Let's assume that you have made this decision for the right reasons, you would most likely identify with this finding of the World Quality Report:
If you identify with this significant group of IT decision makers then you are most likely in pursuit of a solution that allows you or application development effort to be more nimble and responsive to the needs of your business. Ideally, automated software testing should enable an application delivery process that helps the company achieve not just its IT-specific goals, but more importantly, it's commercial / business goals.
Check out our Ultimate Guide to Test Automation for IT decision-makers if you want to understand how to map your goals to an efficient automated software testing program that actually delivers results. The Ultimate Guide also includes a great comparison of the various automation engines and popular software testing tools.
However, it is never enough just to have your heart in the right place. Nice guys do win, but not because they're nice! The answer to "why are you automating your tests" may have been your original goal when your team began on its test automation journey. This goal, however, may have changed based on situational factors. Ask yourself, are your tests failing because the goalposts have changed, ie. am I trying to achieve something that I didn't design my test automation program to facilitate?
If your answer is no, then you will likely identify with the benefits of automated software testing that others in your shoes are trying to achieve.
Want to see how our testing tools & services can cut your testing time from weeks to hours?
The expected benefits of automated software testing...
...Are fairly consistent across most industries. The World Quality Report collects them into 6 distinct buckets and we find that our customers generally aim for all 6 of these benefits when partnering with us for test automation services:
We like graphs and numbers so we came up with our own comparison of the benefits of test automation over manual software testing. We've tried this on some of our customers and they agree that the "Green Zone" encapsulates all of the six benefits above:
The Green Zone also attempts to convey that reaping the benefits of test automation is not just about tools or infrastructure or processes or people / service providers. It's about ALL of these things! This is a whole new topic and if you're interested, check out my LinkedIn article where I discuss the confluence of these factors in relation to using test automation to battle the spiralling cost of software testing.
Identification of these benefits is important, because the most common failing of test automation programs is usually a victim of trying to achieve these outcomes. If your anticipated benefits are different and your test automation is not delivering, then talk to us for personalised help.
The essential components of automated software testing
Remember, any of these components in and of themselves are almost worthless. They must work together to achieve the final outcome that is important to you.
None is more important than the other and each element must be carefully calibrated to enable collective successes. The most common mistake that test automation programs make is to assume that automation will reduce their reliance on people. Test automation does not reduce your reliance on people. It increases your reliance on good team members, but it should reduce the overall size of your testing team.
The reliance on smart team members is brought about due to a couple of significant factors:
- The robots that will perform your test automation need smart people to tell them what they do in the first place; and
- The time savings through automation should be reemployed to helping you innovate further. For this you need very smart people.
The interplay between the 3 P's is best explained in our guide to 10x Your Testing Effectiveness for Digital Applications. Download the 10x guide for the latest best practices for software testing. Explaining all that here will turn this blog post into a thesis (and I'll also need a pulpit!).
Need some personalised help about how you can fix your test automation or get it off the ground?
So which component are you getting wrong?
Probably a combination of the three P's. But for a clue, I looked at the following graph from the World Quality Report and it left me highly perplexed:
If things like test data automation, robotics automation and test environment virtualisation are "emerging automation techniques" then our Qsome Technology Platform should have scooped every innovation award in the software quality space for the last few years!
The other USB plug that stood out in that graph (for the right reasons!) was "test design automation." At Qsome when we are automating tests for our customers, this activity incorporates a broad range of activities including, test design, test scope selection and scripting.
However, the one area that we find makes the biggest difference for customers who came to us after failing with their own test automation efforts is in test design. Why? Good question! The most common flaw in test automation efforts is the the practice of writing automated scripts for manual test cases.
What's wrong with that you ask? Another good question. There are two problems with this approach:
- Manual test cases usually cover well defined functions within your applications. When they cover inter-functional user journeys, the logic of the test becomes cumbersome and unwieldy, causing the automated test to fail at multiple points.
- Manual test cases are designed to be run once. Maybe 10 times. They are simply not engineered to be run hundreds, or even thousands, of times, causing them to fail often when automated as is.
If you are undertaking quality for a large application then you may literally be shooting yourself in the foot, to the tune of:
How do you fix your test automation program?
First ask yourself, are you automating the right tests? If you are satisfied that you have that right, then look closely at your test design processes, specifically the person undertaking automated test design.I'm not suggesting you fire them, but inquire as to whether they have the know-how necessary to convert manual test cases into automated user journeys that can be executed hundreds or thousands of times by robots. If your answer is no, then you might need to find someone who can help you this crucial function.
Test automation is not just about speed. It requires intelligence, adaptability and much gumption. When you have those three characteristics in order then you can focus on getting these 10 things right!
If you need help in cutting testing time and finding more bugs before your customers find them or simply to fix your test automation program, speak to us understand how we might be able to help you. Right now, we're offering a free strategy session to help set you on the right path to achieve your goals.
Or contact us on +61 8 8312 1287 or solutions[at]qsometech.com
0 comments :
Post a Comment