Give your testers a break! Discover how our test automation solution can help your business.

We believe automation should deliver not only better applications, but also be cost-effective and get your new products and features to market quicker. Why not pick our brains about how we do this for our customers?


What We Do


We are a specialist software testing company backed by our proprietary technology.

We complement your current quality program to create a paradigm shift in your application quality.


  • Consulting Advice from Dev to Delivery

    Consulting advice that covers testing strategy development, expert validation and application architecture reviews Learn More


  • Digital Test Practice Development

    Helping you create a scalable and responsive program to help you improve application quality and release speed Learn More


  • Outsourced Software Testing As A Service

    Bespoke test automation of end-to-end processes that traverse browsers, devices & applications Learn More


  • Testing Tool: Qsome Technology Platform

    Our world-class test planning, management & execution tool built for the digital age with a focus on digital test automation Learn More

Solutions For Today's Software Quality Challenges


  • Today's businesses & consumers interact with your app across multiple devices, so why should your testing be restricted to only a few?

    A robust testing program caters for functional, performance & security testing across all relevant devices. Does your testing program do this?

    Learn More


  • The entire premise of continuous delivery is speed and accuracy in execution. Both of these elements cannot be achieved using traditional testing tools and techniques.

    The Qsome Technology Platform is built to improve quality at speed. The combination of technology and bespoke services allows you to achieve your continuous delivery goals.

    Learn More


  • Today's users take mere seconds to judge an app's user experience. You should give them every reason to rate your app highly.

    Additionally, the speed at which app updates need to be released requires a serious quality program that inlcudes automated testing.

    Learn More


  • Performance optimisation is a dynamic exercise that requires multiple iterations. Its importance is magnified in a digital context where users expect, rather than desire, responsiveness.

    The Qsome Technology Platform allows users to execute load tests using functional test scripts. No extra investment is needed.

    Learn More


  • We have developed proprietary algorithms that enable more relevant test management & enhanced coverage & oversight of the most at-risk processes.

    Our custom-developed dashboard gives your team a conscise and updated view of the riskiest processes & the outcome of their recent test results.

    Learn More


  • Making sense of data is one of today's greatest challenges and potentially a very lucrative opportunities.

    Our ability to conduct intensive data-driven testing at speed will help verify that your Hive SQL queries are behaving as intended.

    Learn More




A Sample of Enterprise Applications Where We Deliver Results



Make no mistake, we're not a body shop. We provide our customers test management, automation and execution. All our testing services are backed by our leading-edge Qsome Technology Platform.

We create bespoke application testing programs to execute manual and automated regression tests. Despite the complexity of these applications, our customers are successful because our tests encompass greater test depth and coverage, at speed.

Some of Our Customers


  • A Global Supply Chain Giant

    Designed & implemented an end-to-end, automated software testing process & systems


  • A Leading Auto Manufacturer

    We helped design & implement a test management & execution strategy for upgrades of the SAP Business Suite


  • A Financial Services Company

    Implemented an industry best-practice testing strategy for a new customer-facing technology product prior to roll-out


  • A Government Digital Transformation Project

    Parallel testing for a new suite of citizens' self-help apps, including geospatial systems


  • A Global Entertainment Industry Giant

    Functional & performance QA for a mobile app serving content to millions of users


  • A Peak Information Technology Industry Body

    Automated functional testing of double-sided ecommerce marketplace for members

Ready to witness the magic?



Contact Us




solutions[at]qsometech.com

Global: +61 8 8312 1287

Australia: 1300 28 44 92

India: +91 9845 00 86 96

United Kingdom: +44 20 3769 2460

Australia: 120 Collins Street, Melbourne, VIC 3000

India: 201, Green Glen Layout, Bellandur, Bangalore - 560103



20 July 2017

4 powerful software testing tool features for model-based testing

Model-based software testing can be the panacea to your late and poor quality application delivery woes. However, every panacea needs an enabler and the right (cloud) software testing tools are as important as any other factor if you want to use model-based software testing in your team.

In fact, through experience, blood, sweat and tears toil, we have been able to narrow down the exact characteristics of a good software testing tool that enables model-based testing, to just 4 distinct features!

But first...

What is model-based software testing?

I was going to give you Wikipedia's definition, but why not take the word of a company that has made a business out of giving us extremely complex, but well tested software:
Model-based testing is the automatic generation of software test procedures, using models of system requirements and behavior. Although this type of testing requires significantly more up-front effort in building the model, it offers substantial advantages over traditional software testing methods.
Microsoft Developer Network
We will get to advantages of model-based testing a few paragraphs later. At this point it is important to understand that model-based testing attempts utilises user journeys within an application to design and execute the tests, whereas traditional software testing concentrated on testing discreet features or modules, without caring much for how the user may choose to interact with the application.

Why do we need model-based testing?

Model-based software testing is not only our preferred methodology for designing, executing and maintaining fast and robust application tests, but it is also one that has been refined over time to try and solve the age-old conundrum between speed and quality in software testing. This was quantified by the World Quality Report 2016-17:
The "speed" component of our conundrum is two-sided. Speed is demanded by the end-user in the form of application responsiveness, and it is demanded by the project team, who want to release the application as quickly as possible. With reference to the latter, the velocity of the shift towards Agile, DevOps and Continuous Delivery development approaches has only made it harder for software testing teams:
Frankly I don't understand why the other 59% of respondents decided not to agree with the statement above. After all, common sense dictates that faster releases will also necessitate more tests. Despite this anomaly, this observation completes the speed versus quality conundrum that confronts all application development and delivery professionals:

Want to see how our test automation tools use analytics to cut testing time to hours or minutes?

What specific problem does model-based testing solve?

Through interactions with our customers, we have condensed speed versus quality conundrum down to 5 specific symptoms. You will see below that we have also added a 6 symptom, because it is what every software delivery professional worries about, but for some reason the issue of reputation damage from poor software roll-outs remains the big elephant in the room.

We have an internal philosophy of calling a spade a shovel. This helps us ensure that no elephants get into the room in the first place! So, the 6 problems that model-based testing can solve include:
  • Unclear & incomplete requirements: with fast-paced Agile environments and the race to quickly adapt to user demands, documentation and clarity always suffer. This usually results in poorly tested applications being released into production.
  • Poor test design: this excuse is the general refuge of testers who cannot keep pace with rapid development and deployment cycles. A new paradigm of communication and documentation is required to create better, more robust and flexible tests.
  • Lack of representative data: software testing without data is fast and easy, but it does not result in high quality applications. Data sets often get stored and lost in CSV files, compromising the quality and robustness of tests.
  • Limited or ineffective test automation: automation is not easy, but it is a must-have element of any modern testing program. When combined with the right software testing tools, infrastructure and great test design, it can produce magic.
  • Slow & locked infrastructure: the days of waiting for test servers or licenses to become free should be over. These constraints impede productivity, quality, innovation and, ultimately, the willingness to break boundaries.
  • Damaged reputations: most professionals try their hardest and apply their minds to advance their careers. Traditional methods of software testing have repeatedly resulting in failure, ruining reputations of otherwise solid pros. There are few professional worries in life worse than being unable to sleep for fear of when and how a release might fail in production.

The perceived problem with automation is the result of unsuitable test automation tools and ineffective approaches to test automation. It has resulted in testers having to suffer the poor performance of a myriad of test automation tools, cobbled together with proverbial duct tape:
There is a better way and it is available now.

How does the right software testing tool help with model-based software testing?

Qsome co-founder and my colleague, Ayush Trivedi, created this model to explain test automation. However, it's also apt for explaining model-based software testing. At its core, model-based testing requires the successful confluence of people, processes and technology.
The right technology, ie. software testing tools and test infrastructure, provides the oomph that model-based testing practitioners need to execute successfully. Because we use model-based software testing when providing software testing services for our customers, we have been able to build the Qsome Technology Platform to support us and all other practitioners of model-based software testing.

Specifically, the following 4 features of our software testing tools help to move and synchronise the wheels of an effective model-based testing program:
  • Dynamic user journey mapping: you can document, upload, edit and maintain user journeys right inside the testing tool, irrespective of how many times stakeholders change their minds.
  • Central knowledge repository: all test artefacts and results are stored in one place, which becomes the single source of truth. Nothing is misplaced by being stored on local drives and the people who need access can have it from any browser, on any device.
  • Testing tools & infrastructure on-demand: by using parallel testing technology on an infinitely scalable platform that is maintenance-free, your test team can focus on running high quality, robust tests, instead of worrying about how to keep the test server running.
  • Cross-browser & cross-device automation: your users use your application across all browsers and all devices. Your testing tool must be able to help you get ahead by allowing the advantage of automating tests across browsers and devices without extra effort.

Request a demo now if you want to see first hand how the Qsome Technology Platform could help you successfully implement model-based software testing. Right now, we're also offering a free strategy session to help set you on the right path to achieve your goals.
Or contact us on +61 8 8312 1287 or solutions[at]qsometech.com
Publisher: Meena Kumar - 16:29

12 July 2017

Proven malware cyber attack on CMS templates makes you vulnerable

Malware is taking advantage of an easy path to your machines. By tampering the standard Content Management System (CMS) templates used for web application development, an age old technique called phishing is still actively used to gain entry and compromise applications built with Wordpress, Drupal, Magento and other open source platforms.

Almost millions of systems have been compromised till date.

The web is full of such attack vectors and this trend shows no signs of abating in the near future as patched problems are replaced by newer and more ingenious attack routes. While there is no silver bullet or cure-all panacea, the potential to fall victim to such malware attacks can be minimised.

A few months ago I learned a very interesting and new method to inject malware into Joomla templates. I've since discovered that this attack vector is not limited to just Joomla templates, but such attacks can happen using many other standard CMS templates as well. This technique allows the attacker to gain access to the compromised servers and install malicious files that can then spread and wreak further havoc throughout the application.

Cyber attackers who use this method make use of simple PHP code injected into the CMS Templates. This code then allows them to call files remotely to help the malware spread to the compromised server. The vulnerability is such that the cyber criminal will have the ability to take full control over victim machines.

I have tried this on test systems that we have spun in-house and I can confirm that this works!

The fix

A malware infection through this method can be avoided if all stakeholders play their part. Developers must:
  1. Minimize changes to the CMS's core building blocks.
  2. Use checksum hash to validate the version.
  3. Only download free code and modules from trusted sources.
  4. Vigilantly check downloaded or externally-sourced code for cyber security vulnerabilities or backdoors (contact us if you need help with this)
System administrators should:
  1. Use Agile methodology for development & maintenance so issues can be identified and resolved quickly.
  2. Conduct thorough automated regression testing to ensure business rules are appropriately applied throughout the application
  3. White list allowed IP ranges to access the server and application backend
  4. Regularly run malware checks (contact us if you need help with this)
  5. Plan and stick to a disciplined and structured data backup regime to recover from an attack
  6. Conduct regular penetration tests before all major releases
Users of the CMS system can also help to assure the security of the Magento / Drupal / Joomla / Wordpress application by:
  1. Never clicking or open suspicious links and reporting such links to their teams when found.
  2. Ensuring that they only access the application backend on trusted networks.
  3. Installing and maintaining updated versions of reputable anti-virus and anti-malware applications on their computers.
Open source CMS's and platforms like Wordpress, Joomla, Drupal and Magento are incredibly complex, flexible and powerful applications. However, their most sought-after advantages can also become their Achilles Heel.

If you need help in finding cyber security vulnerabilities in your application or with effective and constant security monitoring of your CMS, speak to us understand how we will be able to help you. Right now, we're offering a free strategy session to help set you on the right path to achieve your goals.
Or contact us on +61 8 8312 1287 or solutions[at]qsometech.com
Publisher: Suresh - 13:17

11 July 2017

Why we love test automation (and you should too)

Test automation technology is a couple of decades old. In this period, we have seen several waves of technologies sweep through the industry. These waves have transformed many industries. Information driven innovations continue to disrupt businesses and continue to surround us all around.

Test automation has, however, not become mainstream. A lot of testing still is done manually. I wonder why?

One common statement I hear is that we should automate repetitive tests. True. But is that all? I contend that test automation, if done smartly, can help us run much more comprehensive tests. We can generate combinations of conditions and leverage automation to run these over the test system tirelessly. Try running these tests manually and you will know how laborious it becomes. The result, when done manually, we skip many conditions, thereby compromising the effectiveness of tests.

Product development is becoming iterative with Agile and Lean start-up models becoming mainstream. These would simply lose almost all advantage if not complemented by a strong test automation focus.

Manual tests become very person dependent. The expertise of the tester determines how well a test case is run and whether the results are meaningful. Some testers do a thorough job of reading the test cases, understanding the core idea and enhancing the tests as they run. Many others, just repeat the documented steps and finish the job assigned to them.

Is there a better way?

I have seen companies that assigned every test case in their catalog to at least 3 testers. Till date, I have not understood the idea behind doing so. A test once executed, should be adequate – unless off course, you hope that at least one of the testers will do a thorough job, or you believe that the test case is not well designed.

Companies spend a lot of time and resources writing ‘idiot-proof’ test cases. Writing and updating these test cases can be extremely time consuming. I wonder therefore, why write tests in English (or for that matter any other language)? Why not write the tests in java (or python, or cucumber) and make them run over computers?

Manual testing needs to be done, well, when people are available – during normal business hours. Automated tests, on the other hand can run over night, over weekends - 24/7. Cloud infrastructure and parallel computing allows us to run the tests fast. Tests that would take humans weeks to complete, are run by machines within hours!

So, if you do not have much time to test, opt for automation. If you do have enough time to test, even then let machines do the job – you focus on your next big idea.

If you need help in solving these challenges, or in cutting software testing time and finding more bugs before your application's users find them, speak to us understand how we will be able to help you. Right now, we're offering a free strategy session to help set you on the right path to achieve your goals.
Or contact us on +61 8 8312 1287 or solutions[at]qsometech.com
Publisher: Asim Virani - 11:32

6 July 2017

A proven software testing company can solve 5 painful challenges for you

Proven software testing companies can overcome 5 painful challenges
Many a software testing company will claim to do everything and anything under the sun for you. But only proven software testing experts will be able to prove to you how they can solve your most painful software quality challenges, like speed of testing, speed to market, support costs, process optimisation and others.

The challenges you are facing in your software development and delivery pipeline are unlikely to be unique. These issues usually stem from a mishmash of approaches and methodologies that are ill-suited to modern applications and their development practices. Even though software development techniques and industry participants have changed dynamically to suit the needs of users and enterprises, the software testing world has, for the most part, remained rooted to ineffective ideologies and practices.

The fix for these ails is not easy, but it is achievable. As a specialist software testing company we created our own testing tools and infrastructure platform and methodologies to help our customers get out of the rut of failed IT projects. The problems we solve and those I'm going to tell you about here are commonly repeated to us by our prospects before they become our customers. The same problems are also described in research conducted by reputable global organisations.

Hang on, what do you mean by "proven software testing company?"

This is an easy one because a proven or specialist software testing company combines three elements to deliver value and solve customers' pain points, specifically those that I will describe below:
3 components of a successful software testing & test automation services
The best software testing services companies bring a unified approach that comprehensively addresses each of the three factors above. Other service providers who use software testing services as a "bolt-on" up-sell strategy usually end up creating situations that do irreparable damage to otherwise competent professionals' reputations or disasters like this:
The practice of asking your developers or your development service company to test the application they are building is like asking Volkswagon to test the emissions of their cars. You may snicker quietly, but it astounds me how prevalent this practice still is:
If you just had a eureka moment or are compelled to let a rueful "d'oh", take advantage of our free software quality strategy sessions where one of our experts can help you devise a plan to get you back on track. In times like this we often fall back on these words of wisdom from a legendary British war-time leader:
Success is not final, failure is not fatal: it is the courage to continue that counts.
Winston Churchill

Painful Challenge #1: My requirements keep on changing

Dynamically shifting requirements and scope creep have been the eternal bugbear of every development team that has ever formed on this planet. It has resulted in messy legal brawls and media outbursts that have only served the purposes of highly paid lawyers and media barons.

A recent study has noted that this is a significant problem for senior IT executives responsible for software testing too:
Frankly, it surprises me that this number is so high. Isn't it the role of software testing to conform to dynamic user journeys and ensure that a high quality product is released, particularly with the increased adoption of Agile and BDD? We certainly think so and anyone in our team who complains about this quickly understands that the care factor for such complaints is south of zero.

I think such complaints from testing teams is a creative take on two similar complaints:
  • There is too much work and I can't be bothered with any more; and
  • I got into testing because I thought it would be easy, and this is not easy!
Attitude factors aside, the requirements shifting issue in software testing can also be attributed to a history of using fragmented systems and processes, cobbled together from competing vendors and kept in place by nothing stronger than masking tape. Such systems do not allow the creation and enhancement of purpose-built testing frameworks that are flexible enough to adapt to change. This symptom likely explains this observation, albeit that it was made within the context of test automation:
Integrating multiple tools just for test automation doesn't make sense
Proven software testing companies will clearly explain to you about their process for adapting to changing requirements and how the technology platform they use allows this to happen. Without a solid process for doing this, software testing specialists such as ourselves would be dead in the water. Effective processes that enable adaptation are also a reason that product companies trust us to do undertake their entire gamut of software quality work, from unit tests to end-to-end functional test automation.

Painful Challenge #2: I can't find and keep testers

This was one of the top 5 challenges in software testing faced by senior IT executives, but I find the percentage who complained about this, incredibly low. From my experience of talking to prospects and customers alike, this number should be well above 50%:
My friend and Qsome co-founder Meena Kumar wrote this about finding and retaining high quality software testers:
Finding good software testers is a hard task. Retaining them for a decent length of time is even more difficult. The churn that comes with frequent turnover in your team will cripple any otherwise effective application delivery program if you are not able to give your testers an engaging work environment where they feel valued.
Meena Kumar
Most good software testers have a software engineering background. This usually implies that they had dreams of becoming a software developer when they started their tertiary education. Along the way life happened (and probably a few stock market crashes, too) and they turned their talents to software testing instead. No amount of success in the testing field will extinguish the desire to be known as a badass developer for many such testers.

The other element to the software tester retention problem is one of monotony, especially in manual testing-only environments. How long can you expect a human to religiously ply through the same regression test scripts without falling victim to the twin evils of boredom and concentration lapses?

Proven software testing companies solve this dilemma in a number of ways. By taking a consulting-centric approach and making our software testers part of value generating methodologies, we can not only attract the best brains in the business, but also retain them longer than most of our competitors.

This subtle difference in how we are able to engage and motivate our teams is one that is often missed by our competitors who use software testing as a bolted-on up-sell approach to the customer.

Want to see how our test automation tools use analytics to cut testing time to hours or minutes?

Painful Challenge #3: We (want to) release fast & testing can't keep up

This perception is a combined effect of the first two pain-points and the fact that most IT executives either don't believe in test automation or have been pouring money into test automation that does not work. Both extremes have led to a stagnation in the perceived efficiency of software testers and a growth in the school of thought that believes test automation is an expensive mirage.

One of the reasons fuelling these two perceptions is this:
There is a lot of noise in the software testing tools space and much of it is being made by "influencers" who take cover under buzzwords and fads to promote underwhelming testing tools. Unfortunately, this creates mayhem in the minds of IT decision makers who end up blaming their software testing teams:
Proven software testing companies well understand the power of test automation. More importantly, we design our testing methodologies to ensure that every test that should be automated, is automated. This allows our customers to execute regression testing early and often and enables software testers to devote greater time to tasks such as exploratory testing, which serve to reveal the fringe situations where a user journey may break down.

Completing the feedback loop so that automated and manual regression testing suites are updated to include testing for new-found defects or knowledge is vital to building a cohesive and efficient software testing practice. We disregard this last step in the process at our and our customers' peril.

Painful Challenge #4: High maintenance, unstable & complex test environments

This is an all-encompassing problem and one that is not easy to fix:
Most medium to large organisations are tied into multi-year licensing agreements with legacy software testing tool vendors. These vendors have not been able to enhance their testing tools to keep pace with modern development methodologies and delivery requirements. Because of such lock-in contracts, customers are unable to take advantage of new on-demand testing tools and testing infrastructure which is purpose-built to enable faster application delivery with greater coverage across browsers and devices.

I also observed with keen interest the increasing reliance on free and open-source "testing tools":
Open source has many advantages: it's free, there is usually a large body of knowledge available online and users don't have to start from scratch. It also has some disadvantages that are conveniently forgotten: it is not usually a complete product or system, configuration can be very time-consuming without prior experience and you have to support it yourself.

These drawbacks alone are enough to rob you of any cost savings you were hoping to make by getting a "free" testing tool. The most common example of this syndrome is Selenium. We use Selenium as the automation engine for web app test automation in the Qsome Technology Platform. It is great for that purpose and suits us and our customers well. Download our Ultimate Guide to Test Automation for CXOs to peruse the comparison charts of different enterprise-grade testing tools as well as automation engines for web and mobile testing.

The problem is that many in the industry consider Selenium to be the panacea for all their test automation woes. What such people are missing is that Selenium is not a software testing tool. It's a set of APIs to control a browser. The essential elements of any good testing tool, eg. test management, test infrastructure, test analysis, etc, still need to be built or cobbled together from other vendors.

Such practices only serve to confuse customers and create inefficiencies in software testing processes. Proven software testing companies understand this and will explain why their preferred software testing tool is more suited to achieving the desired goals.

It's for this reason that we only provide our software testing services if the customer agrees to use our functional testing and load testing tools.

Understand how an on-demand software testing tool and testing infrastructure could help you release faster & higher quality applications.

Painful Challenge #5: The Uber effect on users' expectations of application quality

It has been proven. Today's application users, particularly employees and vendors, are demanding higher quality applications:
I call this the "Uber-isation of enterprise IT". Most of us have used Uber instead of traditional taxis and have been spoiled by the near seamless user experience that Uber treats us to when booking our "personal driver." The bad news for senior IT executives is that their colleagues are demanding similarly seamless experiences in the applications that drive their daily workflow.

Satisfying this expectation is nigh on impossible if you can't find and retain testers, have the wrong testing tools, don't have the right testing frameworks or have cobbled together your testing environment using free code from the internet.

Proven testing companies not only have turn-key solutions to such problems, but we are also able to convert our learnings from one project into more effective and efficient service delivery of our other customers' projects too. Replicating this cross-pollination of ideas will likely be difficult for your teams.

Call me and I'll be happy to tell you how we used the knowledge learned from automating tests for time-delayed or asynchronous user journeys from one customer to help with delivery of our Salesforce testing services for other customers.

If you need help in solving these challenges, or in cutting software testing time and finding more bugs before your application's users find them, speak to us understand how we will be able to help you. Right now, we're offering a free strategy session to help set you on the right path to achieve your goals.
Or contact us on +61 8 8312 1287 or solutions[at]qsometech.com
Publisher: Ayush Trivedi - 15:12

4 July 2017

How to select the best outsourced software testing services company

Best outsourced software testing company
The selection of the best and most compatible provider of outsourced software testing services could be the difference between your applications delivering real business value or ending up on the scrapheap of those poorly executed and costly great ideas that damaged the reputations of everyone involved. The positive or negative results of this decision will be magnified because of the added complexity of the relationship between your teams and those of your outsourced software testing services partner.

I strongly recommend a thorough perusal of best practice guide to software testing to enhance your understanding of what you should reasonably expect from your outsourced testing services partner. Once this preparatory work has been done, there are six crunch questions that you must ask prospective outsourcing partners before making your decision.

First, a word of caution

Price and affordability has its place when selecting a test automation service provider, but the underlying value that is delivered should be equally, if not more important. When assessing value, it's always worth remembering:
80% of companies that outsource software testing to cut costs alone will fail
By no means am I advocating that your organisation pay through the nose for software testing tools and services that you do not need. Nor should you even consider contract with service providers that you genuinely believe have a sub-standard offer.

However, you should stop yourself now and enquire within about your primary motive for outsourcing your software testing program. If it is for reasons of pure cost minimisation, then you may have to incorporate specific provisions in your agreement with your outsourcing partner to ensure that the non-financial outcomes of a good software testing program are delivered throughout the engagement. If you don't, then the statistic above would imply that your outsourcing project is already doomed to fail.

Why do organisations outsource software testing?

A recent KPMG survey of senior IT executives found that testing for mobile applications was a strong driver behind looking for outsourced application testing services:
Mobile app testing is a strong driver for outsourced software testing
Having been in this space for some time now, we have heard a lot of reasons the other 73% of customers request our outsourced software testing or test practice development services. We categorise these into three main buckets: By far the most prevalent bucket is the one defined by symptoms of poorly executed software testing program. There are many examples of such symptoms and we colloquially refer to them as sh*t hitting the fan moments. You may have heard, or even experienced some of these in your career:
Our software testing takes too long and our competitors are releasing their products much faster than us.
IT Director of a mid-sized company
Others are more concerned about the rising cost of supporting bug-infested applications, particularly in the digital age of inter-dependent applications environments:
All of our test cases pass, but the support tickets keep piling up every time we release to production.
CIO of a financial services company
There is a group of IT professionals that is trying to make their technology teams more responsive to business needs and commercial realities, only to suffer increasing amounts of reputation damage with each failed project:
We have great ideas but our execution, particularly when it comes to quality, is letting us down in a big way.
Senior Applications Manager of a mining and resources company
There is another group of IT decision makers that are driving or are key members of expansive digital transformation programs, but they simply have not had the time to keep abreast of the latest developments in software testing tools. In fact, we realised that this problem was more common than we had assumed after coming across this statistic:
IT decision makers feel that they don't know about the latest software testing tools
If you fall into this bucket then I recommend you read our Ultimate Guide to Test Automation for IT Decision-Makers. While it will give you some important tips about how implement test automation, it will also provide a comparison of enterprise-grade software testing tools.

The final bucket is also a very real commercial reality. Most (not all) good software testers are also qualified software engineers. They started their professional lives with dreams of writing code and creating amazing software. Somewhere along the way, they fell into a software testing role that provided them almost all of their wants, other than that job satisfaction that comes with being a badass developer.

Finding good software testers is a hard task. Retaining them for a decent length of time is even more difficult. The churn that comes with frequent turnover in your team will cripple any otherwise effective application delivery program if you are not able to give your testers an engaging work environment where they feel valued.

There are many more reasons we could add to this list. The point is, if you can identify with any of the above or variations on these common issues then you are absolutely correct to seek the expert assistance of an outsourced software testing services company. While there is no one "best" outsourced software testing company, there will be one or two that are best suited to your project. Ask the following questions to find your best fit.

Does the provider use testing tools that suit your needs?

Modern software testing tools can do a lot more, a lot faster than those tools that have been preferred by large enterprises for the last decade. If your organisation has already gone digital or is undergoing a digital transformation, then you need a fit-for-purpose software testing tool that centralises the software testing activities that will need to be executed with each code change. Where it was commonly accepted that test design, test management, test automation and test execution would be done by clumsily throwing together a bunch of different testing tools, the new breed of testing tools allow users to perform all these tasks in place.

At Qsome, we always assert that the best automated software testing tools include the following features, without exception:
All good software testing tools include test management, planning and execution. Only the best software testing tools also include automated test execution, automated scope selection and comprehensive debugging tools and analytics too.
Never accept a testing tool just because it is presented as the "industry-standard" or most popular. Not all tools are created equal and you will should take the time to understand which testing tool is right for your needs. Our guide to 10x Your Testing Effectiveness offers a list of verifiable factors that will help you understand what type of testing tool your organisation actually needs.

Before you complete this process, don't forget about load testing, especially for digital!

Want to see how our test automation tools use analytics to cut testing time to hours or minutes?

How will you objectively assess progress during the project?

We live in a world where all relevant information should be at your fingertips. The minutes of your weekly progress meetings should contain agenda items, not a bunch of statistics that attempt to show you progress. Your software testing service partner must provide you real-time information about the status of your tests. Information in "real-time" is key here, because without this immediacy you will still be playing catch up.

The data points that you need visibility of during a project can be varied like those given below, but should also be able to be customised to your needs:
  • Test script design and creation, including editable data sets
  • Visibility of tests passed, failed and unexecuted
  • Defect reports, particularly those that are overdue
  • Verifiable results of test execution, including video logs, screenshots and technical results

Does the provider commit to measurable and achievable project outcomes?

Insist that your software testing service provider agree to a fixed-fee and fixed-scope engagement. Such engagements compel both parties to work to reasonable timelines, concrete deliverables and measurable results. For the occasional project where a time and materials model is required, be certain to ensure that the value you desired from test outsourcing agreement are still being delivered.

In addition to the project-specific outcomes that you require from an outsourced software testing engagement, there are three key non-negotiable outcomes that your software testing services provider should deliver:
  • Reduce testing time from days / weeks to hours, without compromising application quality; and
  • Increased depth and coverage of tests to minimise the number of bugs that leak into production; and
  • An objective, analytics-based methodology to verify if an application is ready for release.

Can the service provider deal with failure?

Let's be honest, things will go wrong in every project. So will your test outsourcing partner cover up their failures or will they communicate with you about the best path forward? Will they be honest with you in a timely fashion that allows you to pivot your development and delivery plans?

Assuming that you outsourcing your software testing for more than just cost management reasons, you will want to see thought leadership from your outsourcing partner. You will want them to experiment with new ideas that will add future value to your partnership. However, you must be ready to manage less than optimal results, because we all know that failure is a necessary outcome of innovation. Your partnership will flourish if both parties can manage sub-optimal results in a responsible and constructive manner.

The ability to deal with failure is a very difficult characteristic to assess about a third party, but it may just be the most important of the six decision points on which you will make your decision. Churchill's line about failure has proven very instructive for when we have encountered it in our projects:
Success is not final, failure is not fatal: it is the courage to continue that counts.
Winston Churchill
A customer of ours from the United Kingdom is a very innovative company that placed a great deal of faith in our technology, but most importantly in our ideas. After a few weeks of work it became clear to our project leads that the path we had adopted was not going to deliver on our promises to the customer. Our plan to tackle the project wasn't wrong, but our execution was needed a tweak.

When communicating this to the customer, we explained the reasons behind our actions and conclusions. Not only was a viable and effective remedy presented, but we also communicated how this would affect the project's timelines. Most importantly, we delivered on this new plan. A few months after this experience the same customer signed a second agreement to assist them with the remainder of their applications ecosystem.

Most organisations, even the ones outsourcing the work, do not handle failure well. The loss of time, money and more importantly, face, is an unpleasant exercise. Despite this the fruits of encouraging thought leadership and innovation can outweigh short term losses if managed well.

Our customers have reduced bugs in production by over 70%? What could such results do for you?

Can the outsourced testing provider demonstrate a testing framework that is better than yours?

So you've got your testing tool, engagement model and end-goals sorted. But then what? What is the outsourced software testing partner going to do that is a) different from your current approach, and b) how do they know that there approach will deliver the results you seek?

The single biggest factor that kills enthusiasm and future funding for test automation is a regression suite that merely cycles through a bunch of screens without actually testing for bugs. This often happens when you or your outsourced testing provider relies on generic testing framework, with no regard for how that may fit with your applications, appropriate automated test design and the value of model-based testing.

It is of vital importance to test the solution provider's technical expertise and their ability to marry technical skill with the core requirements of the user journeys that your applications enable. Check out our guide to 10x Your Testing Effectiveness to understand the technical best-practices that your software testing services partner should adhere to.

Can the provider back their claims with customer references?

This may seem obvious, but you will be surprised by how often reference checks are not performed. Even if you are 100% satisfied by what your provider of choice has demonstrated to you, it doesn't hurt to seek the views of others who have travelled the outsourced testing journey.

When speaking to referees, focus on the key topics of communications, timeliness, strengths and weaknesses in delivery. Then use this knowledge during your outsourced project to improve communication with your chosen partner and ensure that the intended benefits of the outsourcing project are delivered.

If you need help in cutting software testing time and finding more bugs before your application's users find them, speak to us understand how we might be able to help you. Right now, we're offering a free strategy session to help set you on the right path to achieve your goals.
Or contact us on +61 8 8312 1287 or solutions[at]qsometech.com
Publisher: Meena Kumar - 13:49

3 July 2017

1 powerful idea that made our software testing services awesome

1 powerful idea to improve software testing services

Our customers turn to our software testing services to help them transform their software testing programs from cost centres to value generators. If we followed the same principles that the industry has been using for decades, we would never deliver the lofty outcomes that our customers seek. Instead we turned our practices upside down and created some overarching principles that are non-negotiable in every project. One such principle that helps us deliver this value consistently and across projects is the concept of team-owned tests.

Don't worry if you read that and thought, "WTF!" That is the most common reaction from test design and test automation candidates who attend our interviews. It is our version of XP's collective ownership principle.

What is the problem we’re solving?

Many symptoms contribute to the problem of lengthy, onerous and ineffective software testing cycles that seem more like a burden or an annoying cousin to be locked in the spare room, than the good looking new boyfriend who is to be shown off to friends. In fact, I think the waterfall concept gets unfairly castigated for giving software testing the bad rep it currently seems unable to escape from. It's a combination of a number factors that are having a devastating impact on an economy-wide level:
The annual cost of poor software testing services to the US economy

The activities that have got the software testing profession and function to this point stem from hardware testing. Developing, testing and maintaining hardware is intrinsically a more static exercise compared to testing software. More so now that cloud, agile, devops, etc are hellbent on redefining the concept of speed and efficiency in the software development pipeline.

The very nature of modern software design and development renders traditional software testing methodologies somewhat akin to what propeller-driven planes are to the airline industry. User journeys in software development change often in response to evolving user demands, adding to software testing costs and pressure. The technologies and methodologies (think DevOps and CI) used to build these newfangled applications add an extra layer of complexity to a tester’s task and the costs of testing an application, to the extent that:
Shift to agile & DevOps creates more software testing & increases costs

Also, and very unlike software development, software testing is undertaken by many smaller teams where the test and test script is often designed, scripted and executed by different people. Conversely, in most development teams a single developer is responsible for creating a function or even a module within an application. Therefore, traditional human models about “ownership” and “only I have to know what’s going on under the hood” are about as relevant as a keyboard is to Siri or Alexa.

Essentially, the problem we are solving with the concept of team-owned tests is that traditional methods of executing software testing only result in wasted time, lots of bugs in production and shredded reputations in this day and age. The traditional methods of software testing do not allow easy modification, maintenance or, often, even execution.

If your software testing (particularly automated regression testing) takes an age and still leaks bugs to production, it is probably suffering from this problem.

The difficulties in changing mindset & culture are huge

You will agree that asking a proud and talented professional to unlearn a very natural human trait such as “this test is mine” is a very difficult task. In fact, the problem of culture was referenced in the latest World Quality Report via quote from this very astute professional
The number one challenge is finding the skilled people.
IT Director, Pharmaceuticals Company in Ireland

To overcome this cultural issue that permeates the entire software testing industry the world over, we had to create some very specific structures within our teams to enable the implementation of team-owned software testing. So we now test the understanding this principle as early as the first interview. Some people understand the concept before the explanation has left our lips. Most take a little bit of prodding and require examples to grasp it. About 20% of our interview candidates find within the first few minutes of the technical interview that they are better off looking for jobs in a different company.

You will find that your software testing teams probably display a similar distribution. However, you will find a similar problem to the one GE encountered when implementing Jack Welch’s philosophy of bringing the middle band up to the “stars” band. This is not just a change of personal habit, but a cultural shift that must be espoused by every team member, else it makes little difference in transforming software testing from a cost centre to a value creator. Download our best practice guide for digital application testing to understand how your team can also transform software testing into a value creator for your organisation.

We have found that peer reviews of test design, test scripting (where automation is concerned) and test maintenance activities help to propagate this new mindset. Peer reviews are not the magic bullet to overcoming this challenge, but every law needs a sheriff to enforce it, and our take on extreme development (for software testing) is delivering the desired shift.

Want to see how our software testing tools & services can cut your testing time from weeks to hours?

But, what is team-owned software testing in practice?

Great question. To help our folks deduce if they are implementing this principle in their work, we recommend that they ask themselves one big question:
Can the person next to you understand your test / script / idea without you explaining it to them?
Qsome principle #1 for team-owned tests
One of the ways we build this self-assessment into our test creation pipeline is by splitting the test design and test scripting processes for automated software testing. If the person scripting the test cannot understand the designed test then that question is automatically answered in the negative.

This question also assumes that the person coming after you has no knowledge of the system under test. For obvious reasons, this assumption is not feasible for totally out-of-the-box applications under test, but we find that it remains relevant for most applications that we test for our customers.

The achievement of this goal in software development often results in us users triumphantly labelling an application “intuitive.” Similarly at Qsome, we aim to build intuitive manual and automated software tests that can not only transcend our internal team boundaries, but also the silos in which most of our customers' teams operate.

Then what?

As a company, we are never short of questions. So, after the Big Bertha of a question that gets asked up front, we add a left and a right jab to really start probing:
Have you placed only logic and the right logic in your scripts?
What are you having for dinner tonight?
This is not a joke. The first question is unquestionably a very serious and pertinent one, given that we believe in thorough and robust software testing that actually uncovers bugs instead of acting like a revolving door that allows anything to be released to production.

At Qsome, we are VERY big on test automation. We automate early and often, to the extent that we our default thinking is that the testing of most digital applications can be significantly automated, until a posse of insurmountable obstacles reveal themselves from the shadows. To make this philosophy work, we prefer to use decision trees to support our data-driven testing framework. Our test automation engineers know that the stress must be placed on the data set, rather than the automation script. This is not an easy task because it is generally not the way that their minds have been wired to work throughout their education.

The odd thing about dinner is that you can only have it if you have the time. If your answer to questions 1 and 2 is not “yes,” then you may just have to skip dinner and head straight to a hearty breakfast, because you will be using that time to fix your work.

My thoughts above about dinner and breakfast are only half in jest.

Our customers have reduced bugs in production by over 70%? Want to see how?

Why are you so demanding of your testers?

Wow you really do like speaking truth to power, huh? That is probably also the question that abounds in the minds of the 20% of interview candidates I introduced earlier. The thing is, we really don’t see our approach in that light.

We started out with one very important and achievable goal: to transform software testing from a cost centre to a value generator. There are few ways to achieve this goal unless our teams change the way that software testing is performed.

A crucial obstacle / reality in achieving this goal is the fact that software tests must be as robust and dynamic as the applications that they are built to test. Additionally, modern IT functions are increasingly distributed, outsourced, remote or many other buzzwords that simply mean that your software tests are definitely going to be handled by many different people.

What is the worth of a software test if it requires a dinner date over fine wine to be understood by a person other than she who wrote it?

In football (of the European, not Australian or American variety) the players practice for weeks on end to develop a telepathic understanding of their positioning on the pitch or decision making in pressure situations. For reasons good or bad, modern IT teams, especially in large organisations, have no such luxury. They must get it right the first time or sh*t hits the fan and there’s never enough disinfectant to clean up the mess it makes of some reputations.

Why are you so demanding of your testers?

At work, I don’t believe in making a rule that does not have a firm and achievable goal in mind. But the real results that this principle helps achieves will vary depending on who you ask. For two very recent projects, customers reported the following:
  • One customer reduced testing time from about 4 weeks for every major release to 36 hours after working with us.
  • Another customer has seen a 58% fall in support tickets about bugs in production, after working with us for just 3 months.
I’ll leave you with a simple question, what could such results do for you?

If you need help in cutting testing time and finding more bugs before your application's users find them, speak to us understand how we might be able to help you. Right now, we're offering a free strategy session to help set you on the right path to achieve your goals.
Or contact us on +61 8 8312 1287 or solutions[at]qsometech.com
Publisher: Ayush Trivedi - 12:52