Lack of a robust test environment management (TEM) continues to be a vulnerable area for software quality assurance (SQA), according to the 2018 world quality report. Test environments play a critical role in maintaining schedules and deadlines, but usually get less attention than software production processes. One of the reasons for this is the confusion in understanding TEM, and I want to address these myths in this post.
One of the much-debated myth is about development teams maintaining, monitoring and troubleshooting test environments. System testing is no longer tied to the development environment. In today’s agile era, development teams focus on their core tasks of design and development, and are reluctant to own, drive and improve test environments. The current fast-paced agile software development demands dedicated TEM teams, who will not only own test environments, but also be responsible for reducing downtime, increasing automation, accepting environment bookings, resolving conflicts, managing configuration, and defining and tracking SLAs, inventory, and knowledge management processes. In short, improve overall efficiency and reduce time to market.
Next comes real-time monitoring—a privilege often confined to production. Test environments—including base infrastructure, databases, application logs, and application availability—too need automated, proactive, and real-time monitoring to prevent defects, identify environment issues early, minimize incidents, and increase test environment availability and predictability. While monitoring tools can auto-create incidents for critical alerts, trend analysis of repeat issues can prevent their recurrence. Similarly, known error databases (KEDB) reduce mean time to repair (MTTR). Artificial intelligence (AI), digital transformation, predictive analytics and auto-healing environments—all require real-time monitoring. TCS’ 360 Degree Assurance provides real-time solutions on problem analysis and root-cause identification, optimization and prediction capabilities using AI methods such as natural language processing, artificial neural networks and linear regression.
The cost myth prevents new test environments from entering IT landscapes. With limited budgets, most QE teams make do with existing, legacy testing environments. In the long run, this technology debt— ineffective infrastructure usage, legacy systems, application duplication and redundant tools—will increase cost of operations and reduce agility. But cloud, with its containerization and virtualization solutions, is changing this perception. New test environments can now be provisioned on-demand. Environments can be deployed and configured from ready-made templates. After the code is deployed and tested, the environment can be re-used for similar testing projects. In a recent case, a large Australian bank reduced cost of delivering non-production environments by 75 percent, using test environments on the Amazon Web Services (AWS) cloud. AWS’s accelerated provisioning enabled the bank to achieve project milestones within budgets.
The final two myths are behavioral in nature, and require process change. In most organizations, information technology infrastructure library (ITIL) standards are applied only for management of production systems. These standards must be extended to TEM as well, especially for service, incident, problem, access, configuration and change management. When effectively deployed, ITIL for TEM can significantly reduce MTTR and improve test environment availability. Similarly, configuration management databases (CMDB)—repository of IT assets, their configuration and relationships—must also be created for test environments. Besides enabling test environment and release managers to take rational Go-No Go decisions on major change requests (CRs), CMDB also helps to analyze the impact and minimize test environment downtime.
As agile makes inroads with speed, dedicated TEM focus is imperative. Your organization could start with—in order of priority—real-time monitoring, cloud adoption, automation, ITIL and CMDB deployment. Then, gradually build a case for self-healing, AI-driven intelligent test environments. As your business braces for unprecedented acceleration, you wouldn’t want your test environment to hold you back, right?