How to select the best outsourced software testing services company

The selection of the best and most compatible provider of outsourced software testing services could be the difference between your applications delivering real business value or ending up on the scrapheap of those poorly executed and costly great ideas that damaged the reputations of everyone involved. The positive or negative results of this decision will be magnified because of the added complexity of the relationship between your teams and those of your outsourced software testing services partner.

I strongly recommend a thorough perusal of best practice guide to software testing to enhance your understanding of what you should reasonably expect from your outsourced testing services partner. Once this preparatory work has been done, there are six crunch questions that you must ask prospective outsourcing partners before making your decision.

First, a word of caution

Price and affordability has its place when selecting a test automation service provider, but the underlying value that is delivered should be equally, if not more important. When assessing value, it's always worth remembering:

By no means am I advocating that your organisation pay through the nose for software testing tools and services that you do not need. Nor should you even consider contract with service providers that you genuinely believe have a sub-standard offer.

However, you should stop yourself now and enquire within about your primary motive for outsourcing your software testing program. If it is for reasons of pure cost minimisation, then you may have to incorporate specific provisions in your agreement with your outsourcing partner to ensure that the non-financial outcomes of a good software testing program are delivered throughout the engagement. If you don't, then the statistic above would imply that your outsourcing project is already doomed to fail.

Why do organisations outsource software testing?

A recent KPMG survey of senior IT executives found that testing for mobile applications was a strong driver behind looking for outsourced application testing services:

Having been in this space for some time now, we have heard a lot of reasons the other 73% of customers request our outsourced software testing or test practice development services. We categorise these into three main buckets:

• Symptoms of poor execution
• Inadequate software testing tools
• The by-product of human ambition

By far the most prevalent bucket is the one defined by symptoms of poorly executed software testing program. There are many examples of such symptoms and we colloquially refer to them as sh*t hitting the fan moments. You may have heard, or even experienced some of these in your career:

Our software testing takes too long and our competitors are releasing their products much faster than us.

IT Director of a mid-sized company

Others are more concerned about the rising cost of supporting bug-infested applications, particularly in the digital age of inter-dependent applications environments:

All of our test cases pass, but the support tickets keep piling up every time we release to production.

CIO of a financial services company

There is a group of IT professionals that is trying to make their technology teams more responsive to business needs and commercial realities, only to suffer increasing amounts of reputation damage with each failed project:

We have great ideas but our execution, particularly when it comes to quality, is letting us down in a big way.

Senior Applications Manager of a mining and resources company

There is another group of IT decision makers that are driving or are key members of expansive digital transformation programs, but they simply have not had the time to keep abreast of the latest developments in software testing tools. In fact, we realised that this problem was more common than we had assumed after coming across this statistic:

If you fall into this bucket then I recommend you read our Ultimate Guide to Test Automation for IT Decision-Makers. While it will give you some important tips about how implement test automation, it will also provide a comparison of enterprise-grade software testing tools.

The final bucket is also a very real commercial reality. Most (not all) good software testers are also qualified software engineers. They started their professional lives with dreams of writing code and creating amazing software. Somewhere along the way, they fell into a software testing role that provided them almost all of their wants, other than that job satisfaction that comes with being a badass developer.

Finding good software testers is a hard task. Retaining them for a decent length of time is even more difficult. The churn that comes with frequent turnover in your team will cripple any otherwise effective application delivery program if you are not able to give your testers an engaging work environment where they feel valued.

There are many more reasons we could add to this list. The point is, if you can identify with any of the above or variations on these common issues then you are absolutely correct to seek the expert assistance of an outsourced software testing services company. While there is no one "best" outsourced software testing company, there will be one or two that are best suited to your project. Ask the following questions to find your best fit.

Does the provider use testing tools that suit your needs?

Modern software testing tools can do a lot more, a lot faster than those tools that have been preferred by large enterprises for the last decade. If your organisation has already gone digital or is undergoing a digital transformation, then you need a fit-for-purpose software testing tool that centralises the software testing activities that will need to be executed with each code change. Where it was commonly accepted that test design, test management, test automation and test execution would be done by clumsily throwing together a bunch of different testing tools, the new breed of testing tools allow users to perform all these tasks in place.

At Qsome, we always assert that the best automated software testing tools include the following features, without exception:

All good software testing tools include test management, planning and execution. Only the best software testing tools also include automated test execution, automated scope selection and comprehensive debugging tools and analytics too.

Never accept a testing tool just because it is presented as the "industry-standard" or most popular. Not all tools are created equal and you will should take the time to understand which testing tool is right for your needs. Our guide to 10x Your Testing Effectiveness offers a list of verifiable factors that will help you understand what type of testing tool your organisation actually needs.

Before you complete this process, don't forget about load testing, especially for digital!

Want to see how our test automation tools use analytics to cut testing time to hours or minutes?

Schedule Demo Now

How will you objectively assess progress during the project?

We live in a world where all relevant information should be at your fingertips. The minutes of your weekly progress meetings should contain agenda items, not a bunch of statistics that attempt to show you progress. Your software testing service partner must provide you real-time information about the status of your tests. Information in "real-time" is key here, because without this immediacy you will still be playing catch up.

The data points that you need visibility of during a project can be varied like those given below, but should also be able to be customised to your needs:

• Test script design and creation, including editable data sets
• Visibility of tests passed, failed and unexecuted
• Defect reports, particularly those that are overdue
• Verifiable results of test execution, including video logs, screenshots and technical results

Does the provider commit to measurable and achievable project outcomes?

Insist that your software testing service provider agree to a fixed-fee and fixed-scope engagement. Such engagements compel both parties to work to reasonable timelines, concrete deliverables and measurable results. For the occasional project where a time and materials model is required, be certain to ensure that the value you desired from test outsourcing agreement are still being delivered.

In addition to the project-specific outcomes that you require from an outsourced software testing engagement, there are three key non-negotiable outcomes that your software testing services provider should deliver:

• Reduce testing time from days / weeks to hours, without compromising application quality; and
• Increased depth and coverage of tests to minimise the number of bugs that leak into production; and
• An objective, analytics-based methodology to verify if an application is ready for release.

Can the service provider deal with failure?

Let's be honest, things will go wrong in every project. So will your test outsourcing partner cover up their failures or will they communicate with you about the best path forward? Will they be honest with you in a timely fashion that allows you to pivot your development and delivery plans?

Assuming that you outsourcing your software testing for more than just cost management reasons, you will want to see thought leadership from your outsourcing partner. You will want them to experiment with new ideas that will add future value to your partnership. However, you must be ready to manage less than optimal results, because we all know that failure is a necessary outcome of innovation. Your partnership will flourish if both parties can manage sub-optimal results in a responsible and constructive manner.

The ability to deal with failure is a very difficult characteristic to assess about a third party, but it may just be the most important of the six decision points on which you will make your decision. Churchill's line about failure has proven very instructive for when we have encountered it in our projects:

Success is not final, failure is not fatal: it is the courage to continue that counts.

Winston Churchill

A customer of ours from the United Kingdom is a very innovative company that placed a great deal of faith in our technology, but most importantly in our ideas. After a few weeks of work it became clear to our project leads that the path we had adopted was not going to deliver on our promises to the customer. Our plan to tackle the project wasn't wrong, but our execution was needed a tweak.

When communicating this to the customer, we explained the reasons behind our actions and conclusions. Not only was a viable and effective remedy presented, but we also communicated how this would affect the project's timelines. Most importantly, we delivered on this new plan. A few months after this experience the same customer signed a second agreement to assist them with the remainder of their applications ecosystem.

Most organisations, even the ones outsourcing the work, do not handle failure well. The loss of time, money and more importantly, face, is an unpleasant exercise. Despite this the fruits of encouraging thought leadership and innovation can outweigh short term losses if managed well.

Our customers have reduced bugs in production by over 70%? What could such results do for you?

Get Services Delivery Guide

Can the outsourced testing provider demonstrate a testing framework that is better than yours?

So you've got your testing tool, engagement model and end-goals sorted. But then what? What is the outsourced software testing partner going to do that is a) different from your current approach, and b) how do they know that there approach will deliver the results you seek?

The single biggest factor that kills enthusiasm and future funding for test automation is a regression suite that merely cycles through a bunch of screens without actually testing for bugs. This often happens when you or your outsourced testing provider relies on generic testing framework, with no regard for how that may fit with your applications, appropriate automated test design and the value of model-based testing.

It is of vital importance to test the solution provider's technical expertise and their ability to marry technical skill with the core requirements of the user journeys that your applications enable. Check out our guide to 10x Your Testing Effectiveness to understand the technical best-practices that your software testing services partner should adhere to.

Can the provider back their claims with customer references?

This may seem obvious, but you will be surprised by how often reference checks are not performed. Even if you are 100% satisfied by what your provider of choice has demonstrated to you, it doesn't hurt to seek the views of others who have travelled the outsourced testing journey.

When speaking to referees, focus on the key topics of communications, timeliness, strengths and weaknesses in delivery. Then use this knowledge during your outsourced project to improve communication with your chosen partner and ensure that the intended benefits of the outsourcing project are delivered.

If you need help in cutting software testing time and finding more bugs before your application's users find them, speak to us understand how we might be able to help you. Right now, we're offering a free strategy session to help set you on the right path to achieve your goals.

Book Free Strategy Session

Get Services Delivery Guide

Or contact us on +61 8 8312 1287 or solutions[at]qsometech.com