Today’s companies do not struggle to collect a lot of data: They struggle to wring value from that data. Thanks to big data tools becoming mainstream during the last few years, many companies have gathered huge data sets. Few have grabbed the real prize—using that data to make smarter business decisions in new product development, customer service, marketing, sales, and other key business functions. What is holding them back?
Companies across all industry vertical segments, from telecom to manufacturing, see the potential of analytics and seek to gain competitive advantage from it. And they are spending heavily to do so: revenues for sellers of big data and business analytics will rise from $122 billion in 2015 to $187 billion in 2019 globally, according to research firm International Data Corporation (IDC).1
Despite all that spending, several challenges prevent many companies from realizing strong ROI on analytics projects. For starters, companies struggle to find the right metrics for analytics projects. Next, given the amount of data that can be collected—think petabytes in a day—companies often wrestle with carving out subsets for targeted analysis. Notably, companies may ignore or fail to fully tap into rich new data sources, such as social data and mobile app data.
In another significant obstacle, companies face an increasing amount of unstructured data to analyze (such as text, images, and voice calls). This is not ignorable data: some useful business insights (say on customer satisfaction issues) will only come up in unstructured data. Companies can analyze this unstructured data to improve customer satisfaction and brand image association, for example.
But analyzing unstructured data adds a layer of complexity to analytics projects. For example, how do you coordinate this information with structured business data from traditional enterprise software? Additionally, unstructured data comes in from multiple channels (phone calls, email, social media posts, surveys) and each channel has pros and cons. For instance, if you are analyzing social media posts, you may not know what percentage of your customers are using social media. Internal contact center data offers better statistical estimates. Additionally, fake customer reviews are a well-known phenomenon in social media and elsewhere online; companies must address such reviews, but these reviews do not offer value for analytics projects. A judicious approach is to listen to social media posts for signals pointing to particular issues, then collect more data around each issue in a systematic fashion through targeted questions or surveys for valuable analysis. Unstructured data mining can also raise privacy concerns, especially health and insurance data.
Finally, too much analytical data remains in too many business silos. How do you identify and share the data most wanted by front-line managers to enable better decision-making? This is where speed comes in: You cannot be smart if you cannot get insight to the front line fast. One key is mapping analytics work to the right business owners and processes. Pursuing holistic analytics work across departments and functions, rather than doing siloed analytics work, is key to producing strong ROI for the business.
Shortcomings of Current Analytics Approaches and Tools
Today’s analytics tools and techniques can fall short in several respects for companies seeking to turn data into valuable business insights to wield against rivals. Often, judging the ROI of analytics projects proves tough, because some companies track too many metrics, or the wrong metrics. For example, social media analysis often gets stuck with key performance indicators (KPIs) like positive and negative sentiment indices, or number of likes or shares. But the number of likes or shares, on its own, does not indicate how many people have actually read or heard the content.
Traditional industry data streams (for example, from finance and sales software) do not reveal enough. They are tapped out. What is more, every one of your competitors use the same data in the same way, which does not offer much hope for competitive differentiation. This leads smart companies to think creatively about additional data sources. For example, online real estate company Trulia analyzes more than a terabyte of data daily, in its work to give personalized suggestions to potential home buyers. Its multiple data streams include new home listings, public records, and user behavior data.2 When a new listing matches a customer’s preferences, the customer gets an alert.
Traditional BI systems share data through reports. However, well-built analytics dashboards enable business-side managers to ask more and smarter questions, leading to smarter decisions. For example, Walmart analyzes social streams3 (Facebook, Twitter, Pinterest) to feed data into social media analytics dashboards for its product buyers, who can capitalize on customer interest in products such as juice machines or a particular shade of nail polish.
Deploying such dashboards is not easy. Analytics teams may think that if they just get the right BI tool to the business, they have cracked the problem. But business users seek prescriptive advice, not just attractive visualizations, for the tool to gain widespread use and effectiveness.
While more companies now collect unstructured data, many still keep too much of it in departmental or other organizational silos.
While more companies now collect unstructured data, many still keep too much of it in departmental or other organizational silos. For instance, social media data may be collected and analyzed in one business silo, and so will support center and call center data. Today’s BI tools do not help much with visualization of unstructured data, or linking structured data to unstructured data. What is more, models of how to deal with unstructured data are not mature yet, so companies fail to find best practice models.
As a result of all these factors, companies focus too much on handling the volume of data, and not enough on cutting across business silos to find the right information at the right time—the magic moment when business value appears.
A Smarter Analytics Approach
Successful analytics projects seek to flip that focus, and cut across the boundaries that keep valuable information trapped. Let us delve into some techniques companies are using to overcome the challenges and derive more value from analytics work, starting with finding the right metrics.
This starts with rethinking KPIs for analytics projects. If you want different departments to work together, you must look for a holistic or cross-department measure. For example, on customer service analysis projects, do not use call resolution, but single-contact resolution across all channels, such as email, phone, and social media. This ensures that social customer service teams drive towards the ‘one and done’ ideology so customers are not just hopping to another channel for answers.
Rather than creating and tracking a separate set of KPIs for unstructured data, seek to link the analytical insights to business processes, and thus standard business KPIs. This helps prove business value and ROI.
To do this, companies can use software tools to model the relevant business knowledge, organized by business divisions, products or attributes of products, capture the business rules, and match up the data. For example, a company may use tools such as Clarabridge, Clearforest, and Temis to analyze large chunks of text and route key insights to appropriate business units. Companies also use custom-built tools for the purpose, though such cases are rare.
In one project of this kind, a leading U.S. retail chain needed to analyze customer survey text. It used a suite of tools to do information extraction, text labeling, business modeling, and sentiment analysis. The tools automate work that would otherwise have to be done by hand, for instance, categorizing similar groups of products and customers. The retail chain gained insights on different product categories. Issues for each product category were further classified by factors such as price, product quality, associate behavior, store cleanliness, product availability, and variety. Combined with the geographical information about the stores, the retailer identified actionable insights to increase sales.
A grocery store could use a similar approach to analyze huge sets for customer sentiment data. For example, text analysis can group customer complaints from multiple channels (phone, email, social, survey) under labels. Comments such as “lower your prices,” “checkout lines are too long,” or “I love your dairy products, but keep more types of cheese” can be grouped and acted upon. The business labels might be pricing, checkout process, dairy, and variety. Without automated analysis tools and good mapping back to business process, you could try to analyze such data. However, it would take many people, and there would be subjectivity bias in interpreting the comments. Here, if there is a bias, it is uniform across the data. Also, with automation, the company can run reports like this every 24 hours.
Mapping analytics insights to divisions, products, and services leads to valuable input from business-side teams on what business KPIs, such as a customer satisfaction index and sales volume, relate to the project. For example, if you are doing customer sentiment analysis on social data for an online retailer, the ultimate business KPI is sales. Effective metrics could include click-to-conversion rates or lead generation to sales conversion rates.
New Data Sets, New Tools
In addition to rethinking metrics, it is also important to take a fresh look at your data sources, since valuable information lies in mobile app, geographic, and Internet of Things data, for example. What is more, an emerging and intriguing source to consider is news feed data.
News data can inform everyday business activities, especially at the strategic level, and help a company build contextual intelligence, which is highly desirable. For example, if you are acquiring a business in a new geography, you want to build intelligence about economic factors, political issues, and lessons learned by other companies in that locale.4 Multinationals setting up in India or China face many local issues that have to be understood. In another example, an auto company would want to be up-to-date on all news related to changes in pollution-related laws in regions for which it is designing vehicles. The earlier companies come to know of such changes, the better.
Human analysts can only look at so much news. But today, companies such as Bloomberg and RavenPack offer tools that scan and organize large volumes of news content and social media posts for purposes like these, as well as for financial services and stock trading.
In addition to new data sources, companies want to examine additional data analysis methods and tools (to complement traditional ones). For unstructured text analysis, open graph databases and NoSQL databases prove helpful (in addition to natural language processing tools). Why? In a NoSQL database, data is stored as key-value pairs and not in tables with predefined schema. This helps in indexing unstructured data and retrieving it quickly. Open graph databases are all about resources linking to resources, as in Wikipedia’s example. Wikipedia holds a great deal of unstructured data, but a graph on top links resources. This helps resolve certain issues related to handling unstructured text. For example, using Wikipedia, software tools can easily conclude that the entities ‘Barack Obama’ and ‘President of the U.S.’ refer to the same person.
For example, using Wikipedia, software tools can easily conclude that the entities ‘Barack Obama’ and ‘President of the U.S.’ refer to the same person.
Today, many linked open data sources store valuable information curated from the unstructured content of Wikipedia, ready for business use. Companies can use them to obtain valuable information like “Honda is a competitor of Toyota” or “Toyota manufactures Camry” without much effort. A company might deploy tools that use this knowledge to continuously track negative sentiments about its competitors or their products, or positive sentiments about its own brand.
Unstructured Data, Surprise Insights
What kind of results can companies glean from analytics projects that include a fresh approach to KPIs, multiple data sets, and unstructured data? Often, unstructured data yields insights you may not envision at the start of the project.
Consider the case of an insurance company with global presence in more than 70 countries, which operated a toll-free call center to receive customer complaints. After the company observed, over two months, that call durations grew significantly longer than expected, it decided to examine the calls using analytics tools.
Speech-to-text technology helped transcribe the audio calls. In addition to examining the transcript, analysis included non-linguistic parameters like lengths of silence, number of call transfers within one call, and so on. Clustering techniques grouped like communications together, and communications were mapped to services provided by the insurance company.
Analyzing customer complaint content may not only reveal frequent customer problems, but also identify business process flaws.
The insurance company discovered several unexpected insights. Not only did the technology identify the most frequent reasons for customer complaints, deeper analysis revealed flaws in its process for handling new insurance policy requests. The company gained actionable insights for improving that process. Addressable flaws in the call-center operational processes also came to light, adding more ROI on the analytics work.
Conquering Common Barriers
With analytics projects like these, the smartest companies seek to make the data-driven insights available across the company, not just in isolated departments. That is how companies move beyond incremental analytics project success to company-wide business improvements, as has been exemplified by companies such as Procter & Gamble and GE. Potentially valuable insights will perish if stuck inside silos. For example, as cited above, analyzing customer complaint content may not only reveal frequent customer problems, but also identify business process flaws. Often, owners of analytics projects are only concerned about a specific issue—say brand reputation—and do not bother with other insights that come out of the data.
To change this mindset, and to combat cultural resistance to sharing data across organizational boundaries, you need executive support from C-suite leaders. Executives can also pave the way for moving to new KPIs tied closely to business results. Any change to KPIs can create pushback, since it affects compensation for people from management to delivery teams. So people need to understand the business benefit, as articulated by the CEO. A clear business objective will not only put new KPIs in context for the team, but also ease the case for proving ROI in business terms.
Privacy and compliance issues pose a final hurdle that can trip up analytics projects, especially those involving unstructured data. For example, if a customer intended a communication (call, e-mail) to serve one purpose, the company may not be able to harvest additional insight without suitable permissions in place. In banking, a customer communication may reveal the need for a product or service. However, banks face many regulations in using that information. Insurance companies and internal HR departments see similar restrictions around health information. Your company’s internal privacy and compliance experts should be looped in early in the analytics project to stave off wasted efforts or possible privacy violations.
Answering the Analytics Challenge
The time has come to pay attention to unstructured data, such as customer communication data that is being ignored. In the cases of the retail chain and insurance company, these brands surfaced actionable business information from their data before rivals were able to capitalize on weak spots, and realized ROI directly tied to business goals.
Try to look beyond optimizing business operations by analyzing sales and revenue graphs. Smart analytics platforms can link structured and unstructured data about your company and its rivals, to generate new insights about the ‘why’ of business intelligence. Beyond the sales or revenue graphs, unstructured data holds valuable clues to what excites or dismays your customers—issues that may not have been on the radar at all. Combine this with public information about your competitors, and the data mined from sources such as news feeds and social media, and you get a fuller picture of your competitive situation, faster than before.
Companies have become experts at analyzing sales spikes and falls, adding in seasonal variation. The smart ones have also learned to capitalize on unstructured data. For example, clothing retailers in the U.K. jumped on the opportunity to sell blue dresses that looked similar to the one Princess Kate wore in a photo celebrating her engagement, and later, baby clothing modeled after her childrens’ outfits.
What will the next opportunity look like for apparel retailers, or for any company that masters analytics technology? The answer is out there, ready to be analyzed in unstructured data.
1. CIO.com, Big data and analytics spending to hit $187 billion, May 2016, accessed on July 18, 2016, http://www.cio.com/article/3074238/analytics/big-data-and-analytics-spending-to-hit-187-billion.html
2. CIO.com, How an online real estate company optimized its Hadoop clusters, April 2016, accessed on July 18, 2016, http://www.cio.com/article/3058187/big-data/how-an-online-real-estate-optimized-its-hadoop-clusters.html
3. Fusionbrew, How Walmart uses Data Visualization to convert Real-time Social conversations into Inventory? April 2014, accessed on July 18, 2016, https://www.fusioncharts.com/blog/how-walmart-uses-data-visualization-to-convert-real-time-social-conversations-into-inventory/
4. Harvard Business Review, Contextual Intelligence, September 2014, accessed on July 18, 2016, https://hbr.org/2014/09/contextual-intelligence