Independent software testing is an Oxymoron in the age of DevOps and continuous delivery

“Inspection neither improves nor guarantees quality. Inspection is too late. The quality, good or bad, is already in the product. Quality cannot be inspected into a product or service; it must be built into it” – Wendell Philips

There are a lot of vendors in the market who peddle software QA services as a standalone offering. Some even make a tall claim of Independent Software Testing. I can understand if the services are of a consulting nature to help improve QA processes, or to help test some non-functional aspects. I can’t imagine how core functional software testing can be done independently, i.e. separately from development, especially in light of the fact that continuous delivery has become the norm. In this blogpost, I would like to explore how software quality assurance has evolved over the years, and what are the QA needs in the age of continuous delivery.

Continuous disruption and shorter innovation cycles are the new norm

Over the past decade, there was a fundamental transformation in the way software is conceptualized, developed, tested and deployed. To a large extent, this transformation is a response to the challenges of continuous technology and business model disruption. Digital accelerating technologies such as SMAC, IoT, AI, Robotics, VR etc. caused unprecedented disruption in terms of scale, speed, impact, and are unlike anything that the world has seen in the past. This technology disruption is the driving force behind the integration of value chains, emergence of digital business ecosystems, digitization of product and service offerings, and evolution of innovative platform business models.

The cumulative impact of this massive technology and business model disruption is a connected world, that demands a culture of rapid innovation, to meet the needs of continuously evolving customer experiences. Speed and shorter innovation cycles are the norm in the digital era, and even more importantly, software is no longer just a business enabler, but has become a core business function.

Shift-left testing

We have come a long way from the days of linear, waterfall approach to software quality assurance, which treated testing as a separate, stand-alone, end of the dev cycle activity . With the advent of agile, linear dependencies of waterfall are eliminated, and the emphasis shifted to process efficiency, geared towards delivering working software in short bursts. Adoption of agile best-practices led to a shift-left approach, i.e. moving to the left of the project timeline, which called for system testing often and early in the development lifecycle. Agile resulted in an overall improvement in quality and reduction of project risks, as testing is integrated into the development process, and is performed early and frequently.

DevOps and continuous delivery

DevOps emerged as a natural extension of agile, and applied the same principles to the rest of the software organization across delivery and operations. This led to a seamless integration of all processes from development, integration, testing, deployment, to feedback. DevOps is a mindset, and has heralded a fundamental cultural change by introducing a continuous culture across the software value chain, i.e. continuous development, continuous testing, continuous integration, continuous delivery, continuous feedback, and continuous improvement. True DevOps culture, along with automation goes a long way in enabling continuous delivery of high quality, functional software. Even though DevOps is considered by many as the natural evolution of agile across the software value chain, there are a few critical differences as shown in the table below:

 Automation is the bedrock of DevOps

Continuous delivery demands that every code change is well-tested, integrated, and is ready to be deployed to a production environment, with as little human intervention as possible. DevOps is based on the premise that manual processes and human intervention give scope for slippage in quality and performance, and the surest way of eliminating them is through automation. While DevOps is much more than automation, it is absolutely essential for enhancing quality, and reduce lead times by automating most of the repetitive tasks across the software value chain. I have covered automation more extensively in an earlier blog: Automation is the bedrock of DevOps, the key features of which can be summarized as follows:

• Everything that can be automated must be automated
• Automated regression and functional testing
• CI/CD
• Virtualized, ready-to-use DTAP street

It is a well-known fact that digital unicorns such as Amazon and Google update their software at an astonishing frequency of thousands of times in a day, often with lead times in minutes. This would be impossible to achieve in the absence of DevOps and automation.

Built-in quality and focus on prevention

The traditional QA approach of discover, identify, and fix defects before the software is released to the market will not work in the age of continuous delivery. With ever-shrinking lead times and shorter innovation cycles, there is also a fundamental shift in focus of software quality assurance from discovery to prevention. Preventing software defects from occurring is only possible with a built-in quality mindset, which calls for adoption of best-in-class practices across the software value chain. Test-driven development, continuous integration, automated regression testing, security & performance testing are some of the best-practices that will go a long way in establishing a culture of built-in quality, and prevent software defects from occurring.

Independent software testing is an Oxymoron

In light of the facts stated above, I strongly believe that in the age of continuous delivery, Independent software testing is an oxymoron. Let me explain my rationale:

• Quality is a culture and mindset
• To achieve built-in quality, you need multi-functional teams with full-stack engineers who are responsible for Dev, QA, and Ops
• Test-Driven Development is very crucial for software quality, and dependent on meaningful unit, integration, and regression tests, which can’t be developed in isolation
• Quality is not an isolated process, but is an integral part of the software value chain – when the goal is to continuously deliver working software, testing is done often and early
• Quality control or check may be possible through independent testing, but not continuous quality assurance
• Quality control or check may be possible through independent testing, but not continuous quality assurance

• In the era of DevOps and integrated software organizations, quality cannot be viewed in isolation from development and operations

Software organizations must realize that there is no magic bullet for software QA, and certainly Independent software testing will not result in sustainable quality improvements. They must work diligently towards achieving built-in quality, which is a critical need in the age of DevOps and continuous delivery.