What exactly does DevOps mean for organizations? Is it just another operating framework to build and release better software products, faster? Not really! DevOps is part of the cultural transformation organizations have to embrace as they transition from traditional (largely manual) to agile, automated application delivery.
The driving force for DevOps continues to be the continuous integration (CI) and continuous delivery (CD) pipeline – a core business process that helps developers rapidly build, package, integrate, test, and release code by automating manual, error-prone steps. After all, bugs in the code are expensive and the cost of rectifying them grows exponentially across the various software development lifecycle (SDLC) stages. A simple issue with a website’s search feature failing to find the right product can have a cost impact of up to GBP 950,000.
Without a tight quality assurance (QA) framework in place, developers largely resort to using Jenkins. It has therefore become the go-to open source standard for managing the CI/CD aspect of DevOps. As an automation and orchestration engine, it is capable of not only managing source code but also delivering new code to production.
A gated, analytics-led QA framework
A robust QA automation framework needs to be centered on a selective accept/reject feature in the software during SDLC, which limits fault tolerance localized to specific development. Developers working with tight release schedules can run fault tolerance tests at a feature level while these features are being developed in parallel. This in turn helps product managers, marketing, and sales teams to not only moderate the end customer’s expectations from the software, but also enable them to plan future upgrades. Supported by advanced bug analytics, the QA framework can help developers quickly assess feature codes, check if they match defined quality metrics, and choose the ones that will make it to the impending release.
Accelerating QA through virtualization and automation
As a direct result of increasing virtualization, hyper-converged infrastructure that supports elastic virtual machines can be used to create test bed environments capable of hosting and simultaneously executing thousands of tests in just a few minutes. Hosted shared services environments such as these make it possible to adjust a VM’s quality of service (QoS) parameters and prepare it to handle any type of payload data needed to perform QA. Automated tests hosted on a test cloud can be modified to become resource-pool aware, in turn helping in discovering, locking, and releasing virtualized test environments. There is no theoretical limit to the number of tests that can be executed in parallel as part of Continuous Integration Testing (CIT) and smoke testing, especially for enterprise software.
Minimizing human intervention
The shift left approach suggests moving high-yield defect tests early in the test cycle to minimize the cost of quality. There are tremendous benefits in doing so as part of CIT and smoke testing. Similarly, shift right suggests moving low-yield defect tests during Continuous Regression Testing (CRT) runs. CRTs are conducted over relatively longer (week, fortnight, month) cycles as compared to CIT and smoke tests. It therefore makes sense to include long running, complex environment test runs during this phase.
However, if the shift left ends up executing a huge number of tests early in the development cycle, then the challenge comes in triaging possibly high number of test failures in a short time. Triaging test run logs, so far, has been a bottleneck in any QA environment, purely due to the manual nature of the activity. Log analytics is a supervised learning technique that uses artificial intelligence (AI) to perform effort-intensive failure analysis. Combined with three-way traceability – correlating a failure among code database, bug database, and test database – it can mine, analyze, and provide insights on the root cause of failure, suggest remedies, and help in creating stronger, meaningful test scenarios. With open source tools like ELK coming to the forefront, we have just begun to realize what AI can do in terms of software QA. A complete machine learning-based solution, which can not only automate but also dynamically adapt to changing test requirements, is still far from becoming a deployment-ready solution.
Just in Time (JIT) QA
QA methodologies for unit test should ideally move away from minimal sanity to comprehensive just-in-time tests, with the highest code coverage through extensive regression suite. However, the test cycle should be short enough so that it does not impede CI/CD functions. Using advanced bug analytics, the JIT QA framework can ensure that the quality benchmarks for a new piece of code has been satisfied even before it is checked into the CI/CD pipeline. In case a bug is discovered, it will prevent progress to suffer.
With the growing adoption of artificial intelligence and machine learning, quality assurance will redefine itself to adapt to digital technologies and largely eliminate the manual intervention in software testing. Digital competency will no longer be classified only a developer’s skill, rather will become a prerequisite for DevOps led projects, driving everyone in the team, irrespective of the role, to use digital intelligence for faster decisions, cost optimization, and ultimately, delivering impeccable software quality.