A software testing recipe for a $5 million bug & ruined reputations

In a classic case of how not to release a digital application, the Economic Times reported a real case from the Bank of Maharashtra where US$5 million was swindled by digital thieves.

How is this possible? Well, the recipe for such disasters is quite simple and actually very common. This unenviable outcome was possible because the bank's digital application testing program relied on antiquated software testing techniques that borrowed from all of the following software testing misdemeanours:

• A completely manual process with inadequate coverage
• Releases made on hunches, rather than objective data
• Test plans that only test the path of least resistance, with little or no negative testing

Such software testing practices are commonplace, particularly in organisations who are only starting out on their digital transformation. Indeed the report itself states that the Bank of Maharashtra's service provider was implementing the same solution at three other banks in the country!

Why is manual testing not enough when testing digital applications?

As an aside, we argue that manual testing must always be augmented with a thorough, automated regression testing suite, even for "non-digital" applications such as ERPs.

However, digital applications usually have two characteristics that increases complexity and renders manual-only testing inadequate:

1. They provide an external interface to the organisation's core business processes. This means that external users have a lot of real-time power at their fingertips, without necessarily understanding the full implications of wielding this power by using the application.
2. A myriad of mobile devices adds unending risk and complexity. Developers well understand the compatibility problem between IE, Firefox and Chrome in a browser-only environment. Add to this independent devices with differing browsers, operating systems and processing engines and you begin to understand the digital Pandora's Box or risks and potential problems.

In such complex environments, no human tester or team of testers is accurate and consistent enough to comprehensively test every permutation and combination of user inputs, prior to every release. Rely on manual testing alone and you allow room for disastrous to fester in your applications.

What is required to effectively test digital applications?

In our guide to 10x the effectiveness of digital testing, we suggest a testing program that effectively combines people, communication and technology. There are no shortcuts to this. All three elements are necessary to turn your digital testing program from a cost-centre into a value producing activity.

A central theme in companies that have effective and efficient digital testing programs is automation. Not only automated regression testing across browsers and devices, but also automation in test scope selection and automation in process prioritisation when the entire application does not need to be tested.

Often, working out what your organisation needs to bring its testing practice into the digital age requires an impartial quality audit. It never hurts to get back to basics, right?

Request Free Advice Now

10x Testing Effectiveness

Or contact us on +61 8 8312 1287 or solutions[at]qsometech.com