Contact us
We are taking you to another website now.


How Quality Engineering Enables Customer Experience Transformations


The launch was supposed to spur excitement. A large bank redesigned its personal banking web pages to attract new customers and generate more business from existing customers. Instead, the revamped website was an embarrassment. Pages loaded slowly for customers using the Chrome web browser. Website traffic dipped to a record low. Instead of elevating the customer experience, the redesign damaged the bank's reputation and brand.

When a customer interacts with a digital product or service, she is not thinking about using a tool: touching a smartphone, clicking a keyboard or swiping a tablet screen. She is buying products and services, or getting information. A single hiccup in the process - a touch, click or swipe that fails to yield the expected result - quickly becomes a reason not to proceed.

Executives in charge of developing and presenting these digital customer experiences must make these processes seamless. They must deliver great quality service every time, reliably, whether the customer is using a mobile phone, laptop or desktop computer, smart speaker or digital device. The experience must do the job as designed and deliver results quickly.

Ensuring that requires thinking differently about engineering the digital experience. Quality engineering, an approach that relies on agile, DevOps and processes that automate the quality assurance process, enables companies to ensure that customer experiences on websites and apps perform right the first time.

In other words, quality engineering is a catalyst for enhanced CX.

The Growing Challenge of Customer Expectations

The bank's website disaster was not atypical. Many organizations are not providing the digital experiences their customers expect. Sixty-nine percent of customer experiences in the U.S. in 2017 were just "OK" according to a 2017 Forrester Research study of 118,992 online consumers interacting with 314 brands in 21 industries.1 In that study, consumers ranked 17% of their experiences good and only 1% as excellent. Nearly one out of four (23%) were rated poor or very poor.

A delay in website response can be fatal to customer experience improvements. Researchers at Google found it takes about 15 seconds to load a typical mobile web page on a smartphone. That may sound slow, but it is worse than slow in consumers' minds; many will leave a mobile website long before that.2 Average mobile website loading speeds for U.S.-based firms vary by industry. Health care firms were best at 5.4 seconds while technology firms posted times of 6.7 seconds.3 That's not good enough. All are at risk of losing their users.

Security is also crucial to creating a quality customer experience. Not enough firms are validating the security of their customer-facing applications. Some 44% of professionals who develop customer experience applications said their firms had not taken steps to guard against hacking of their mobile or IoT applications, according to a Ponemon Institute survey of IT and application security practitioners.4 In the same survey, 11% said their organization's mobile apps had been breached. Many organizations treat security as an afterthought: 58% said their organization typically waits until their systems are in production to test the security of IoT applications, while 39% said they usually wait until production to test mobile application security.

Another notable characteristic of top-performing customer experiences: They account for those who may have a vision or hearing impairment or other physical or intellectual challenge. Organizations that do not accommodate these users increasingly risk being sued for discrimination. In the United States, there have been more than 240 lawsuits against companies in retail, hospitality and financial services industries alleging violations of laws protecting those who have physical and intellectual challenges.5

In the world of digital experiences, things are not likely to get easier. Customer expectations continue to rise as they embrace the latest mobile apps that use automated technologies and artificial intelligence. These digital experiences raise everyone's expectations. And even if a digital application doesn't incorporate the latest technologies, customers expect the systems to work just like their smartphones - simply, easily. And they expect their digital experiences will be private and secure.

Companies are betting their futures on their ability to develop new, enticing and sticky digital interfaces on time, without glitches in functionality or performance. They are continuously launching new systems and updating existing customer channels. Websites must cater to different browsers. Mobile apps must appear on different smartphone platforms. Voice-activated products - like Amazon Alexa and Google Home - need to be sophisticated at recognizing and answering voice commands. Interfaces need to be embedded in devices and appliances in customer homes, businesses, cars and all the other locations where they live, work and play.

These many avenues for access generate complexity. And when one considers that each channel must handle multiple systems (e.g., various iPhone and Android versions) complexity increases, and avenues to failure proliferate.

To overcome these hurdles, companies must go beyond quality assurance to what we call quality engineering. Delivering an enhanced customer experience through the quality engineering route enables an enterprise to meet three key imperatives:

1. Taking an automation first approach across the whole engineering value chain.

2. Centering the quality engineering approach around the customer experience.

3. Creating an efficient tools strategy for the test environment to further automation that improves the customer experience.

The barriers to achieving these requirements lie in the shortcomings of today's quality assurance approaches.

The Problem with Traditional Quality Assurance

Traditionally, companies set up processes for quality assurance teams to test their customer-facing applications. These processes were appropriate when they were first developed but in today's rapid development environment these quality assurance processes take too long to get results. What's more, they often fail to catch system compatibility glitches that derail the experiences of entire customer groups. Traditional approaches typically focus only on parts of a customer experience, rather the end-to-end process. (Think of a retailer whose omnichannel experience includes in-store, online and mobile.) Traditional quality assurance approaches also can fail to account for the interactions among different applications and services, both existing and new, and whether they are compatible with or disruptive to the customer experience, or how the customer experience in one channel relates to all the other channels a company uses. And while some organizations have benefited simply by executing tests earlier in the development lifecycle (an adjustment called "shift left"), the larger quality challenges remain unaddressed.

Quality engineering takes a different direction.

Quality Engineering: Enhancing the CX

Quality engineering is a superior method for ensuring that customer experiences on websites and mobile apps perform right the first time and every time. The approach recognizes that as enterprise applications (including those connected to new apps and customer-facing systems) continue to grow in complexity, companies need a cost-effective, tool-agnostic platform that can orchestrate software releases. Quality engineering enables the adoption of agile and DevOps development approaches. These approaches enable an organization to design, develop, test and deliver applications rapidly, repeatedly and with tools and technologies that automate the quality assurance process holistically.

Quality engineering starts with establishing holistic automation processes that embrace a customer-centric mindset in every application design choice and software programming decision. This means procedures measure customer experience performance according to such considerations as:

■  How the system operates, and whether it performs according to expectations (such as speed and reliability)

■  How the customer experiences its performance

■  How a specific feature or design fits into the entire customer experience relative to a company's other offerings

■  How a specific feature or design relates to the customer's life and work experience

Before using a quality engineering approach, companies first should compare their CX to industry best practices and determine how to address the weaknesses they find. They should ask what stands in the way of increasing speed-to-market for new and superior customer experiences. Answers to these questions will help them focus a quality engineering program on such elements as:

■  CX quality evaluation to assess and remediate enterprise applications that serve customers, correlating results with key process indicators

■  Technology infrastructure that supports automated systems testing and assesses how CX services perform with a wide range of customer devices, browsers and operating systems

■  Testing for security and regulatory compliance including the safety of personal information and meeting rules related to privacy and differently-able user access

■  Ease-of-use testing with a focus on end-to-end customer journeys through channels such as a mobile apps, mobile websites or websites accessed from a desktop computer

■  Testing for best-in-class performance to measure response times for user clicks and transactions, benchmarked against top performers

Recent AI advances are vital in quality engineering programs, enabling companies to evaluate large systems with numerous potential errors and flaws rapidly. AI-enabled tools can analyze and test customer experience systems much faster than conventional ones and quickly zero in on problems. The approach is designed to include new factors as they come online - whether they be chatbots, new social network connections and more - to ensure these functions work for customers as intended.

How to Get Quality Engineering Right

In our experience, a holistic quality engineering program must address an array of issues - including the deployment of agile approaches and DevOps to automate much of the work and improve its effectiveness - before it can be implemented successfully. Quality engineering covers five dimensions:

1. Compatibility tests ensure an application runs uniformly across all digital mediums. For example, a life insurance policy quote can be started on a tablet and completed seamlessly on a mobile phone.

2. Usability testing addresses quantitative validations like task times, error rates (compared to industry standards) and guidelines on such aspects as human-computer interaction—addressing navigation, presentation, content and interaction. (Are the changes between web pages logical and easy for a customer to follow? Does the customer get the information he requested?) Usability also covers visual consistency across screens. For example, an online retailer planning for peak activity during a Black Friday sale must ensure its digital channels are providing fast, seamless navigation among different web pages, precise content in response customer requests and auto fill capabilities on payment forms to speed the purchase and check-out process.

3. Accessibility testing ensures "inclusive" assurance. It verifies digital assets against the World Wide Web Consortium's Web Content Accessibility Guidelines. For example, a mobile banking app should be built with accessibility features like alternative text (that describe objects that appear on a screen), good contrast ratios and visible focus indicators (that enable a customer to use a keyboard instead of a mouse to access links, search boxes and other features).

4. Performance testing examines the fluidity of user experiences such as search, comparing alternatives offerings, as well as the ability to select, order, schedule delivery and pay instantaneously. Different channels such as mobile and web require their own testing. For example, a retailer planning for a large online sale tied to an event needs best-inclass response times and load times for its channels to prevent crashes during peak demand.

5. Security testing examines the fluidity of user experiences such as search, comparing alternatives offerings, as well as the ability to select, order, schedule delivery and pay instantaneously. Different channels such as mobile and web require their own testing. For example, a retailer planning for a large online sale tied to an event needs best-inclass response times and load times for its channels to prevent crashes during peak demand.

What Great Quality Engineering Looks Like

When you look behind the scenes of organizations that have successfully implemented quality engineering programs and processes, you see the problems that did not make it out into the world. These organizations caught these flaws and fumbles before they reached customers, both internal and external. The examples below quantify the problems and risks avoided through quality engineering.

Repairing customized systems before launch (not after).

Situation: A New York-based media and information company was customizing its Salesforce application to serve as both a CRM and an ERP system for planning, forecasting and financial management.

Quality engineering impact: Testing processes identified flaws before the company implemented the system. Problems uncovered included:

■  Accessibility: Quality engineering impac 4,908 defects across 25 web pages tested on four browsers.

■  Usability: Quality engineering impac 250 defects identified across seven major task screens

■  Browser compatibility: Quality engineering impac 25 URLs with different content when viewed with four different browsers

Result: An improved customer experience for the media company's users. The company adopted techniques like automated cross-browser testing to correct these issues before moving the system into production, saving time and money.

Testing improves the product performance before customers see it live.

Situation: In preparation for the pilot launch of a new financial product, a U.S.-based insurance and financial services company needed to test elements that went into creating the product, including the web application, critical infrastructure and middleware.

Quality engineering impact: The testing process identified flaws the company addressed, which reduced security risks and enabled it to meet web accessibility standards.

Result: By identifying flaws pre-launch, the company reduced its information security costs by 60%. And by identifying browser compatibility and accessibility issues, the firm corrected those issues before launch, saving time and money.

Quality Engineering: The Way to Meet Evolving Customer Expectations

Customers will not wait for companies, even well-known brands, to fix a problem. They will move to another provider or vendor.

A quality engineering approach to testing applications and services before customers experience them is a proven path to designing superior digital experiences.

Customers won’t wait for quality. And neither should you.

1"The US Customer Experience Index For 2017: CX Quality Worsened,” September 27, 2017, accessed at

2Think With Google, Find Out How You Stack Up To New Industry Benchmarks For Mobile Page Speed, February 2018, accessed at

3Google research cited at, March 7, 2018, accessed at, "10 Key Findings From the Ponemon Institute's Mobile & IoT Application Security Testing Study," January 18, 2017, accessed at

5The Wall Street Journal, "Companies Face Lawsuits Over Website Accessibility For Blind User", November 1, 2016, accessed at

About the author(s)

Thank you for downloading