Every large company today is swimming in a rising tide of digital data. That data resides in their information systems and in the ecosystem that surrounds them – the data of suppliers, customers, distribution channel partners, and others with which they do business. Companies that are fastest and best at turning that data into insights, and using those insights to improve their businesses, will be rewarded in the marketplace.
They will stage more effective marketing and sales campaigns and produce more successful new products. They will be faster to identify unmet customer needs and fill them. They will lead in what is being referred to as “The Insight Economy.”
But to do this, they will have to be far better at analyzing digital data, the sources and types of data rising at an exponential rate. Some 90% of the world’s digital data today has been created in the last two years. By itself, the United States on average produces more than 2.6 million gigabytes of Internet data every minute, a number that will only increase as the world of connected devices grows.1 Soon, people routinely will be talking about “zettabytes” – a trillion gigabytes of data – a concept that would have been unfathomable only a few years ago. In 2017 the world produced more than 20 zettabytes (ZB) of data, a number predicted by International Data Corp. to increase eightfold by 2025.2
That data can bring insight, power, and, ultimately, business success. By the end of 2018, half of large enterprises will be selling their digital data, as well as metrics, insights, and recommendations on what to do with it, IDC predicts.3 Such data provides the raw material for operating in a Business 4.0 world, one in which a new generation of agile organizations are embracing automation, robotics, AI, analytics, and cloud computing to rearrange their marketplaces.
In particular, artificial intelligence is poised to turbocharge analytics technologies, which in turn will help managers in every industry make much clearer and faster sense of what’s really happening in their businesses. Three-quarters of 250 executives surveyed believe AI will substantially transform their companies within three years.4
At its most basic level, large volumes of digital data are essential to optimizing analytics, artificial intelligence (AI) and machine learning technologies. With robotics and other intelligent systems, informed by digital data, companies can automate formerly manual processes in the interest of efficiency, cost savings, and improved customer experiences. For example, a manufacturer’s analytics identifies a performance problem with a customer’s product and provides an automatic fix through a software download. Automaker Tesla already provides over-the-air digital updates and upgrades for such features as blind spot detection and performance tuning.5 Now multiple manufacturers are offering or testing field performance and diagnostics services delivered over the internet.
Collecting, processing, analyzing and acting on data has incited a revolution in management techniques, in industries ranging from banking to baseball. The companies that are best at analyzing digital data are leaping ahead of competitors because they generate unique insights in what to produce, who to sell to, and what and when to promote. They are also better at understanding and managing their finances, supply chains, productivity challenges and human resource needs.
Here are a few examples:
In electrical equipment, ABB, the $34 billion industrial firm that services power plants and automates industrial processes, credits its ABB Ability platform of services with driving an 11% increase in orders for its software and services in 2017.6 Using real-time data, the platform analyzes and monitors conditions at industrial facilities such as factories, oil wells, and power plants. It then uses this analysis to reduce operating costs and improve safety and maintenance.
In retail, data scientists and analytics experts are helping Amazon and Walmart outcompete many established grocery and general merchandising companies. Amazon’s food and beverage business is expected to grow by 40% in 2018.7 Its share of all retail sales is predicted to nearly double, from 5% in 20188 to nearly 10% in 2020.9 Though it got a late start, Walmart’s e-commerce business, with a 3.7% share of the U.S. ecommerce market, is rapidly growing thanks to analytics, and is expected to increase 40% this year.10
In media, Facebook, Google, Netflix and Spotify are outpacing incumbents in publishing, movies, and music. The new guard’s data-driven insights are fueling rapid growth. They can deliver potential buyers to advertisers with a precision that traditional mass media companies never could.
In banking, fintech companies are snaring consumer dollars without the overhead of brick-and-mortar branches. For example, deposits at online-only Ally Bank have grown more than 50% over the past three years to just under $100 billion.11 Ally’s disruptive digital offerings are drawing a highly-prized customer base: More than 50% of its new customers over the past year have been Millennials, a number that is expected to rise.12 Fintech also plays a big role in payments, with startups like Ant Financial, Stripe, and Paytm handling transactions for millions of customers around the globe.13
In the last 10 years, the science around data and analytics has moved at warp speed. Advancements in quantum computers promise to open new frontiers in data analysis. Quantum computing could revolutionize the discovery of new materials by simulating matter down to the atomic level.14 Researchers are developing artificial neural networks to endow AI with a sense of imagination—for example, to produce images of objects that the systems have never seen.15 Cloud-based AI is making machine learning widely accessible and far cheaper, which promises to transform a wide variety of businesses.
Corporate boards need to take notice: The extent to which an enterprise excels, or fails to excel at using its data to drive growth, improve profitability, and manage risk will determine its future. Boards must understand the company’s relative maturity as it relates to data analytics.
The Central Role Abundant Data Plays in Today’s Enterprises
To compete in the fast lane, companies must improve how they manage their digital data and analytics dramatically. This is because we are moving from a world of incremental business process improvement to one of transformative data, analytics and insights.
Companies vary in their data and analytics maturity. At one end of a spectrum, there are large enterprises that have launched organization-wide data management initiatives and are benefitting from them. At the other end, many other companies continue to manage data at the departmental level. Many more are somewhere in between. For these companies, siloed data continues to be a significant challenge and barrier. For example, with sales maintaining its own data on prospects, and finance keeping track of who’s been billed and who has paid, a company only can achieve a limited analytical capability. There are only a few big companies that understand all their interactions in a connected manner or possess all the information on any one customer in one place. In an age where companies are leveraging data to improve every aspect of their business, the traditional approach of capturing and holding data in silos will penalize those companies that still practice it.
We see five levels of such data maturity, ranging from siloed to self-optimized:
Companies that dramatically improve their analytics capabilities gain a major competitive advantage. In recent years, organizations largely have focused on developing the maturity of their business processes, not the data that fuels those business processes. Now, excellence in data management is as important as process excellence.
There is so much data—especially unstructured data—that firms now need to manage and analyze. This data has great potential to be tapped in novel ways by analytics.
But this will not be easy; the world of analytics fueled insights is one of continuous disruption compared to the old world of continuous improvement. Analytics-driven insights enable companies to make big changes, fast—in their people, products, services and the business processes that support them. Yet companies still are struggling to analyze their data and put them to use.
In a survey of 196 organizations around the world, Gartner found that 91% had not yet reached a “transformational” level of maturity in data and analytics, despite this being the top investment priority for CIOs in recent years.16 Indeed, 60% rated themselves average or below average in data and analytics maturity. Companies that become masters at generating real-time insights that can be embedded into business processes, enabling them to make rapid changes based on those insights, will pull ahead of their competitors. Think of automatic pricing updates that a retailer, airline, or ride-sharing service provides based on dynamic changes in supply and demand. Or the process for a bank promoting a new service offered to consumers identified by predictive analytics as most likely to respond. These examples illustrate how firms can incorporate their mature data capabilities in their every-day actions.
Managing Analytics to Become an Insights-Driven Business
C-suite executives have a great opportunity before them: to identify ways to increase revenue, improve profitability, reduce risk and potentially disrupt markets by applying analytics insights to their current business practices. However, this won’t happen through analytics alone. It requires a combination of data insights and contextual knowledge. Executives need to use their contextual knowledge—of existing business processes and how they can be improved; of the organization’s culture and how people may be retrained to use the products of data; of the means for selling and servicing products; of customers and their operating environment; of competitors’ strengths and weaknesses; of marketplace openings for new products and services—to maximize the benefits of data analytics. (See sidebar: How different C-suite stakeholders can advocate for effective data use.)
In the past at many organizations, it was the CIO’s responsibility to lead the effort to drive analytics efforts. Now all C-suite stakeholders have great opportunities before them:
For the CEO and heads of strategy, analytics can help them craft business strategy. Insights will help answer the fundamental questions that every company ultimately faces—e.g., whether the business model is about to become obsolete, and how it needs to change. Netflix transformed the media industry by using analytics to provide a steady stream of entertainment personalized for each of its members, in contrast to incumbent media companies that geared their programming toward the masses, essentially appeasing everyone but rarely addressing smaller market segments.17 The result: U.S. consumers now watch Netflix more than any other entertainment platform. Netflix-produced content also was nominated for more Emmy awards than any other network in 2018, a distinction that cable TV powerhouse HBO had held the previous 17 years. For the media industry, Netflix’s dominance has been disruptive, and has forced action from competitors, as reflected in the recent costly takeover battle between Comcast and Disney for 21st Century Fox assets.18
The CMO needs to prepare for the next wave of digital transformation in marketing and sales. Mass personalization will lead to event-driven marketing (EDM), in which companies like banks provide timely, relevant offerings. That helps them show they understand customers and their changing preferences. And, with EDM, a retailer no longer delivers a tailored promotion when it wants to push a product, but rather when the customer has a need. That requires knowing the customer’s need when it arises and fulfilling it at the optimal time: before he or she begins to research a solution.19 As an example, a standing-desk company may deliver a special offer to professionals who search the web for information on back pain. Achieving EDM means sharing data from sales and customer service business processes, as well as incorporating external sources of data. This can dramatically improve the monitoring of customer spending behavior, and where they are in their customer lifecycle, enabling companies to respond quickly with offers to influence their purchasing decisions.
Similarly, the chief customer officer or chief experience officer can use analytics to segment customers to determine what products and offers these smaller groups would respond to, now and in the future. Ultimately, the goal is to use insights to better serve every group. For instance, the MagicBand wrist bands that visitors to Walt Disney World in Orlando, Florida, wear communicate with thousands of sensors throughout the park and stream realtime data to hundreds of systems, Disney’s goal: to “root out all the friction within the Disney World experience.” Every Disney World visitor provides the company with a series of data points that helps it continually improve the individual’s experience and increase revenue. When visitors are waiting in line, they’re not spending money at concession stands.20 Analytics can also help reduce complexity for customers. For example, the mortgage industry could simplify the loan origination process and personalize services by offering products such as customized loans and flexible payment options. The Indian government is using analytics to improve its passport services. In a project that demonstrated the value of fast responses to its analysis of customer feedback data, the government digitized what had been a slow process. India can now issue or renew a passport within a day. Projects such as this highlight how effectively analytics can improve organizational agility and make continuous improvements.
Heads of research and development can use analytics to develop new products and services. Pharma companies are using machine learning in drug discovery to analyze how molecules interact. Ford is using VR to create virtual models of cars and enable designers to improve their collaboration on design changes. The technology reduces the need to build physical prototypes and allows the engineers to more easily explore creative designs.21 Cosmetics companies are testing their products by creating “digital twins” of animal skin, so they can see the reaction in a virtual environment without harming animals.
For supply chain executives, analytics present many opportunities to save money and increase productivity. Oil and gas companies have reduced “maverick spending” by analyzing procurement data and finding waste and fraud; others use analytics to identify less costly suppliers. Automotive companies are using analytics to identify potential supply chain weaknesses, enabling proactive countermeasures before costly problems emerge.22 At GE, a project to address redundancies in negotiated supplier contracts that were previously managed at the business-unit level saved the company $80 million in its first year.23
For CFOs and chief risk officers, analytics technology can significantly expedite tasks such as closing the books, finding bad credit risks, and identifying fraud. It also can be used to find accounting anomalies and comply with anti-money laundering statutes. Currently, RPA and analytics are facilitating the invoice matching process. This involves automating business rules for different matching scenarios, using supporting documents, such as purchase orders, goods receipts, and contracts. RPA tools can capture the details, regardless of invoice format, making reconciliation faster and more accurate. The emerging technology of neural automation promises to do even more complex tasks, such as three-way invoice matching.
Customer service organizations use analytics to get feedback on the life of products once they are in the hands of users. This information is invaluable to executives in customer service, manufacturing, product development, and service performance. With sensors sending data from products in the field, companies can gain unparalleled insights on what features customers are using, when software needs to be updated, and when products need to be repaired. Manufacturers of heavy equipment can use telematics to optimize equipment usage and plan for preventive maintenance.
For COOs, advanced analytics are providing unprecedented insights into how far-flung physical assets are performing. They can determine which branches, stores, hotels, resorts and other places of business are getting the most traffic, which are bottlenecked and at what time, and how different parts of these assets are performing. This can go a long way toward improving the customer experience, thereby boosting profitability.
Legal departments are using analytics to scrutinize contracts more efficiently, augmenting human efforts and reducing routine research. Machines can recognize parts of contracts that resemble ones that have been previously reviewed and can sift through documents far faster than humans can. That saves money and increases accuracy.
For IT departments, analytics can reduce the risk of intrusion and fraud, and can aid recovery after disruptive incidents. Analytics can sort through enormous quantities of data – such as web logs, alerts, traffic captures, and attacker artifacts – to identify anomalies that might signal an intrusion. Then, by tracing the activities of the intruder, the responder can fix vulnerabilities and reduce the chances of future attacks.24
As the list above demonstrates, every C-suite stakeholder has the opportunity to combine their contextual expertise with analytics to make their operations more efficient, profitable, and agile, while helping them identify opportunities for savings and growth.
How to Excel at Analytics and Insights
From our client work, we have found five elements to be critical to excelling at analytics:
A discussion of each of these elements follows.
A company’s first step toward leveraging analytics is to engage key stakeholders to set the scope for the initiative. The scope should be representative of the company’s business goals and include the desired analytics capabilities, and data required to support them.
The work begins by understanding the condition of the company’s digital data. The company needs to ascertain whether it has all the data needed to create insights and to make changes in the business based on those insights. It must also determine what data is missing and where it might be collected—including from external sources such as business partners. For example, wireless telecommunications companies struggle to retain customers in a competitive market. They can use internal data to analyze network usage to predict peak volume times using high-bandwidth apps and video, thereby reducing customer dissatisfaction. Companies could also use external data such as social media streams to gauge customer sentiment and reach out to those at risk of leaving. But to make these adjustments, the company must have that data available and accessible.
External data sources represent an opportunity to augment the data companies possess to create new and deeper insights. The marketplace provides all sorts of information that, properly used, can create tremendous value. The growing data brokerage industry, which collects and markets data on consumers’ habits, is expected to reach $250 billion in 2018.25 New partnerships are being forged to provide useful data. For example, the city of Amsterdam uses GPS data from mobile and navigation devices, gathered by a private company, to create models to study traffic issues in the city.26 In India, ride-share and taxi services can identify the restaurants and establishments where people eat. That data can be sold to other restaurants and businesses that want these people to buy their products and services. The offers, based on insights about customers’ dining histories and locations, create opportunities to collaborate in an urban area digital ecosystem, one that could enhance customer experiences and expand sales, creating a virtuous (and profitable) circle.
New data can come from unexpected places— even washing machines. Consider WASH Multifamily Laundry Systems, a California-based laundry facilities management service provider with 75,000 locations in the United States and Canada. WASH collects a vast amount of data from its extensive network of hundreds of thousands of interconnected washer, dryers, vending machines, and payment systems. Working with apartment building owners, WASH uses this data to test manager’s questions, such as whether it is cost effective to switch from cash to payment cards, before committing to changes.27
Once a company has conducted a data inventory and needs assessment, it can determine the analytics tools it requires to make productive use of its data. To build strong data and analytics capabilities across a company, and not simply in isolated pockets, it must adopt a vision of analytics for all, leveraging best practices to achieve digital and analytics maturity. It must not limit the use of analytics tools to select functions or a small group of experts.
To choose which analytics technologies to use, companies first must realize that the range of possibilities is vast. Analytic solutions vary from traditional business intelligence tools to machine learning algorithms. We think of the options available as a decision fabric that includes choices relating to how systems ingest data for analysis; how they store data, and how they operationalize data. Tools to handle these tasks are evolving quickly and it important for executives to select flexible technologies that will serve their current analytics needs while remaining useful when more advanced tools that provide improved functionality become available.
This is important because data and analytics solutions have been broadening their scope—not just in the business capabilities they provide but also in their underlying operating capability. Their storage and computing power are continually increasing, newer mechanisms for consuming data such as immersive analytics that builds on virtual reality environments are becoming available, as are expanding data sources such as data exchanges and marketplaces for business partners.
Choosing the right tools is key to analytics success. In making these decisions, companies also have to consider the status of their data estates and what it will take to the modernize these assets to support analytics. Data estates supporting analytics can be a mix of multiple generations of data platforms, including mainframes, operational data stores, relational data warehouses, data warehousing appliances, data lakes, and cloudbased data platforms. Like the decision fabric noted above, a company should consider its options for maintaining and modernizing these assets to support analytics—and to continue that support as the company matures in its analytics capabilities.
With a clear vision of its business goals, and an understanding of what data and analytics technologies it needs, a company then must decide what processes and skills it requires to create a successful analytics capability.
This entails assessing staff members’ competency and skills in combination with existing service delivery and governance practices. Executives can assemble a holistic roadmap that includes provisions for making all elements—people, process, technology and data—ready for analytics.
In addition to recruiting analytics talent, the company must decide where these analysts should sit in the organization: Should they be located in a centralized analytics group, within a business function, or in some other arrangement? And who should the analytics function report to?
In the past, analytics typically reported into finance, the CEO or CIO. Now it’s clear that the function needs to have its capabilities integrated throughout the enterprise, with an executive (like a chief data officer or chief analytics officer) leading the effort to bring analytics to every department—and injecting functional knowledge about those departments into the analytics organization. Picture data science and other data experts working with colleagues from various business functions and product lines who can bring contextual meaning to the data. This is the optimal way to structure your analytics talent.
An analytics capability requires collaboration across functions, lines of business and external partners using systems that share data and insights—collaboration that extends beyond a company’s firewalls.
It is critical that data governance, privacy and security measures receive the attention and investment required to protect the security and privacy of sensitive data. Establishing a central data office can serve to implement these controls on security, privacy, and data-sharing while taking responsibility for complying with an ever-changing regulatory landscape. For example, mapping data privacy classifications with specific data supply chains and the analytical outputs supported by them allows visibility into which parts of the data landscape are exposed to what type of user base. The details of this mapping exercise can become the basis for privacy and security controls.28
Assembling these five elements in a stepwise, holistic manner can have a great impact on any organization. Monsanto illustrates the benefits of sharing data and analytics expertise across functional boundaries. To increase revenue and improve profitability, the $14 billion agricultural and biotech company launched a cloud analytics platform it calls “science@scale.” The platform helps the company’s data experts and scientists collaborate, and it has accelerated the development of analytics-driven decision models, says CIO James Swanson.28 And because the platform enables scientific work to be done in hours rather than months, the company estimates it has saved $15 million in operating costs and boosted revenue by $17 million. This principle is applicable across all major industries.
The Democratization of Analytics
Today, companies that wish to excel at analytics are largely dependent on people with advanced analysis capabilities to manage data, create algorithms, and take other steps to turn data into insight. However, this is inadequate in the current data abundant environment. Data is growing so fast that hiring and training enough analytics experts to keep up with the onslaught of information is realistically impossible. Depending on a core of analytics experts inevitably will create bottlenecks, preventing an organization from scaling its use of data.
Instead, companies must democratize analytics, with data tools available to people with varying degrees of analytics expertise at all levels of the organization.
Companies today have numerous people without PhDs in statistics or data science but who have deep familiarity with customers and business processes. In many enterprises, they are an underutilized resource in generating insights from data. Rectifying this gap requires both training and new technologies (some of which are already available) that make analytics technologies simpler and more accessible.
Crowdsourcing is another way to overcome internal skills shortages in analytics. Companies like Netflix and Procter & Gamble have held online crowdsourcing contests that ask people outside their organizations for help in solving technical challenges.
Applications of data-intensive VR (virtual reality) technology can make complicated data sets comprehensible to technicians and even customers. Lowe’s, the large home improvement retail chain in the U.S., is experimenting with VR headsets to help customers choose from a vast array of possibilities for a kitchen remodel.29 Elevator manufacturer ThyssenKrupp is using Microsoft’s HoloLens to visualize an elevator repair before the technician reaches the site. Onsite, technicians can use augmented reality (AR) to view digital overlays of manuals and repair guides while they fix the elevator.30
The pathway to an age of abundance depends on subtraction and addition: removing human beings from the data preparation process, while augmenting human beings so they can more effectively use that data.
Overcoming Barriers to Capitalizing on Analytics
To take advantage of the insight economy, companies need to avoid several pitfalls. Above all, the organization needs to master change. Capitalizing on analytics demands a complete overhaul of the organization’s culture at every level. It’s a matter of moving from intuition to decisions based on data, of getting comfortable with managing robots and learning to accept what they tell you. Line managers need to grow comfortable acting on analytics, as do senior executives.
This transition is challenging and requires a comprehensive change management effort. Business users require education and practice in using tools and the products of data analysis. They must learn that data can show correlations about business trends that are more precise, and actionable, than their gut instincts. (Because these users bring great resources of institutional knowledge, the training is well worth the investment.) Training should continue as the company’s analytics maturity advances; for example, when a firm implements more technologies to automate analytics production, that will require users to adjust (and carry out more advanced tasks).
Becoming a data-driven organization requires coaching and practice, too. Everyone in the company needs to understand the importance of the effort and their role in its success. To ensure that the transformation is successful, employees need to embrace idea that the company’s data is not “owned” by any one function but that all share ownership and responsibility for making good use of it. In some organizations, those who led data efforts in recent past, such as the CIO or chief data officer, now need to collaborate extensively with other functions to make the best use of their singular expertise. In particular, data and analytics experts must work with domain and contextual experts to maximize the benefits of analytics. This will democratize the use of data and the creation of business value from it.
Data management and ownership issues can pose major internal barriers to analytics adoption. Data that is held captive and not shared across the organization can impede progress. On the other end of the spectrum, companies can also struggle with an excess of data choices. Executives need to evaluate options and prioritize the best uses of the data their organizations possess.
Hiring, developing and retaining analytics and AI talent is another initiative vital to analytics success. To do this, companies need to be aware of what motivates these professionals, who may quickly become the organization’s most valuable assets.
Data privacy and security should also concern any organization embarking on a data-driven analytics transformation.
Organizations need to set up systems and appoint individuals responsible for ensuring customer consent and other elements of regulatory compliance. In pursuing the opportunities posed by analytics, the company must attend to the growing body of data privacy statutes and anticipate regulatory changes. Both the European Union’s General Data Protection Regulation (GDPR), which went into effect in May 2018, and California’s recently enacted Consumer Privacy Act (CPA), which will take effect in 2020, include strict provisions and prescribe heavy fines for violations.31 Data security, meanwhile, requires extra attention when a firm stores data in the cloud—and executives must ensure their analytics work accounts for this.
These regulatory statutes apply to organizations doing business in their jurisdictions and processing the personal data of their residents, regardless of where the businesses may be headquartered. Companies should continue to monitor regulatory environments in all areas of the globe where they do business and ensure they thoroughly understand how their data is being used while planning for future statutory developments.
Clearly, implementing analytics (just as with any technology) is not a panacea for every business problem. In addition to selecting the right technologies and tools, success requires a commitment to culture change, investments in skills, and revamping business processes. Implementation, and the transformation it produces, is about people and processes as well as technology.
First Steps Toward the Insight Economy
The insight economy is available to any organization that seizes the opportunity. This starts with assessing your data capabilities, the maturity of your enterprise’s data collection, data management, and analytics capabilities. Then, it’s a matter of determining the best course of action. That means deciding which parts of the business, which functions, and which business models would benefit most from the application of intelligence, automation, agility, and the cloud.
Leaders today need to make the mission clear: to liberate data from its silos and unleash their workforce to gain crucial new insights and move their firms forward. The data and expertise are abundant. The methodologies for leveraging data are proven. The technology and tools are available. The will and imagination are yours to tap.
About the author(s)
Dinanath (Dina) Kholkar is Vice President & Global Head, Analytics & Insights at Tata Consultancy Services (TCS). In this role, Dina guides some of the world’s best companies in their journeys to unlock the potential of their data through the power of analytics and artificial intelligence (AI) to uncover new business opportunities, drive business growth and increase revenue streams.
Dina has been recognized as one of the top 100 influential global data leaders and data visionaries. He advocates ‘data centricity’ as a strategic lever for business growth and transformation. His thought leadership in addition to his team’s expertise and collaborative working with customer organizations is empowering them to realize the power of their data in real-time decision making and ensuring success in their Business 4.0 transformation journeys.
Dina holds a bachelor’s degree in electrical engineering. He has been providing industry leadership to the IEEE Pune section for over 15 years; currently Chair, Industry Relations and Membership development. He also provides leadership, guidance and strategic direction in domains including education, sustainability, agriculture, and ‘data for good’ through his volunteering work at IEEE Pune Section, Pune International Center (PIC), and the Tata Group. Dina is a member of the Board of Governors of his alma mater Veermata Jeejabai Technological Institute (VJTI), Mumbai and actively involved in the institute’s alumni association. He is a review committee member, Indian Statistical Institute, Kolkata and is on the advisory committee at Pune Knowledge Cluster (PKC).