25 March 2021

5 unexpected reasons why outsourced test automation fails

Failure isn't fatal, but failure to change might be.
John Wooden
And therein lies the crux of the problem with many outsourced test automation services providers. The ability and willingness to acknowledge, analyse and learn from mistakes, both your own and those of others, is a very noble and productive human characteristic, but sadly and uncommon one.

Far too often we receive mayday calls from prospects who have invested a lot of money, but more importantly time, into selecting and working with a outsourced test automation services provider, only to be rewarded with a never-ending tug of war that inflicts much damage to their professional reputations. At the root of these frustrating and poor outcomes is a test automation service provider who was either out of their depth, unwilling to own up to failure and probably did not fully understand the customer's desired future state.

In the case of outsourced test automation services, most customers are trying to prevent losses of the magnitude reported by Computer Weekly magazine from the UK:

One analyst, who follows the outsourced software testing space closely, believes:
Cost savings are a major driver and also the need to do testing in a more professional manner.
Dominique Raviart, analyst at Nelson Hall

While he is definitely on the mark, it is obvious to us that the first desired outcome often drives a disproportionate share of decision-making when selecting outsourced test automation serviced providers. This only serves to diminish the importance of and attention to actually conducting "testing in a more professional manner." In fact, as far back as 2007, Gartner predicted:

Although the statistic above related to the BPO industry but our experience tells us that this would also hold true for outsourced test automation engagements. Gartner research director Alexa Bona put that statistic down to high staff attrition rates suffered by many outsourced service providers. If you think about it, there are no free lunches, right? If you pay peanuts, you get...

While some test automation services outsourcing engagements fail because of technical reasons, it is our analysis and that of other storied industry veterans, that most fail because of misaligned priorities, inadequate discovery and the IT services industry's propensity to try and fit square blocks in round holes.

At Qsome, we ask a lot of questions up front before agreeing to work with a customer. This process of understanding one another is an important step in building confidence and assessing whether the relationship will be one of mutual wins. We have seen that the following five questions are often not asked or not understood during the "courtship" phase between a customer and the test automation services provider. Its almost never as simple as just these questions alone, but they are a great starting point if you are trying to figure out why your test automation services provider just isn't delivering.

The 5 often forgotten reasons outsourced test automation engagements fail are a set of questions that you should ask yourself before your outsource your next engagement. Answering these questions for yourself even if the outsourcing service provider does not ask you these questions will help give any engagement a greater chance of success.

1. What are you trying to achieve?

As an outsourced software testing service provider, this helps us quickly prioritise wins from a potential engagement, as the customer sees them. To the earlier link between the predominance of cost saving motives in failed engagements, answers to this question also help us understand if the customer is looking for fast and effective test automation services that really kick goals, or just an improvement to their cost control target for KPI assessment.

By no means is attempting to achieve cost savings an unworthy goal. However, if the customer is only interested in cost savings then there is little on which to form a lasting and mutually fruitful relationship.

If your software testing outsourcing partner has not asked the overarching goals for engaging, then you should explain these goals and tie progress evaluation to the achievement of these goals. Mutual and detailed understanding of the desired future state is the bedrock on which your software quality will flourish or flounder.

The future state should define quantifiable and measurable metrics covering software quality, internal and external resourcing, timeliness and investment targets.

Want to see how our testing tools & services can cut your testing time from weeks to hours?

2. What are your expectations of us?

The desired future state you described above is only going to be achieved if the combined labour of the customer and the outsourcing service provider produces the desired results. Just because you outsource something, does not mean you are absolved of all responsibility and accountability.

We ask this question to understand two things:
  • How hands-on the customer wants to be throughout the project; and
  • How the customer views their involvement in the project changing over time.
The answers to this question also help to clear many of the unknowns and misconceptions that are very common in outsourcing engagements and lead to increasing distance between both parties. You can minimise, if not eliminate, the "crossed wires" and "we thought you were taking care of that" moments by clearly understanding the roles each partner will play throughout the engagement.

This clarity can be a big boon in engagements where customers want us to develop a test practice for them and leave them to run it. It goes without communication that overall outcomes are significantly improved through this greater degree of communication.

3. How important is this goal in the big scheme of things?

This answer really tells a story about customers' priorities. Are your non-financial priorities at least on par with your cost saving goals? Are you really serious about finding bugs before they get into production or are you more concerned about shifting risk?

Prioritising these goals will help you make a more informed decision about your preferred outsourced testing provider. Some outsourcing partners will be happy to simply help you achieve your cost targets, whereas others may decide that they would rather concentrate on projects where cost reduction is one element of a good outcome.

It is a horses for courses world, so make sure you communicate your goals in prioritised order so that you can attract the outsourced test automation partner that is best aligned to what you are really trying to achieve.

How do you ACTUALLY know if your software testing is working? You must have a dashboard to easily track key testing metrics.

4. How will we talk to each other?

Phone, email and Whatsapp are all relevant and useful modes of communication. Many stalwarts of this industry also produce very fancy looking PDF or Excel reports prior to every progress meeting. But, how do you really assess project progress?

To remove this information asymmetry we developed a real-time dashboard as part of Qsome's set of functional test automation tools and load testing tools. This dashboard is not only the central source of truth for project progress, because our test engineers see exactly what our customers see, but it also helps our customers make highly objective decisions about if and when they release to production. This removal of subjectivity between both parties and within both parties is a critical trust-building exercise.

By eliminating any doubts about the veracity of the information being discussed, we enable deeper, more frequent and more productive discussions about how we can achieve the customer's desired future state even faster.

Ask yourself, how could objective and accurate real-time data exchange between you and your outsourced test automation partners improve the results that are delivered by your team for the organisation? Think about it.

5. Have we understood you properly?

The clincher! All the thinking and explaining and strategising you have done to date are of no value if you and your outsourced test automation partner are not on the same page.

A simple technique that we ask our team to practice during sales meetings and progress review meetings is to explain what they think they have understood in their own words back to the customer. Nothing promotes clarity and consistency like repetition.
Any idea, plan, or purpose may be placed in the mind through repetition of thought.
Napoleon Hill
If this question does one thing, it is that it quickly brings unrealistic expectations to the fore. By repeating your goals and expectations aloud back to you, your potential outsourcing partners will be able to quickly decide if they are really attracted to you or if they just want the money. If it is the former, then you may just be on to a good thing.

If you picked up a strong whiff of communication being the central theme behind these questions, then you are on point. Good communication can be an abstract concept to those who miss the forest for the trees. In fact, you know communication is non-existent if someone believes they have communicated well with their outsourced test automation partner because the "legal agreement is watertight!"

Let's be honest, if any commercial relationship devolves into both parties quoting from the contract, you may as well close the door on achieving any goals. While this is an unfortunate fact of life, you can minimise the chances of suffering this fate if you and your test automation outsourcing service provider understand your answers to these key questions.

If you need help in cutting testing time and finding more bugs before your customers find them, speak to us understand how we might be able to help you. Right now, we're offering a free strategy session to help set you on the right path to achieve your goals.
Or call us on +61 3 7001 1430 or +44 20 3411 4974 if you're in the UK, or email solutions[at]qsometech.com


Post a Comment