Data governance has become more critical than ever before. The speed with which data is generated (research projects 1.7MB of data created every second for every person on earth by 2020) and the sheer variety of data in the digital ecosystem mandate a well thought out data governance strategy. Enterprises find themselves in the middle of a data ocean and struggle to know which of their own data will lead to desirable business outcomes. As the focus on data governance grows, a manual approach is no longer feasible because it does not scale to the growing data scope. A Machine First™ approach that leverages artificial intelligence (AI) and machine learning (ML) is the way to achieve sustainable scalability and agility desired in data governance.
Challenges in optimizing data governance
Researchers discovered that 87% of enterprises have low analytics maturity and one of the reasons is the lack of a data governance approach to enterprise data management. Accurately identifying critical data out of the data ocean is the fundamental challenge in becoming a data-driven organization. Beyond the humongous volume of data, enterprises must also address the multifold complexity of business context among diversified data sources. Data sources do not necessarily align with the business language or other data sources of an enterprise. For example, one of the two distinct data sources providing customer address details may contextualize it as billing address while another as shipping address leading to significant adverse impact on shipping and billing processes.
The internet of things and thin boundaries between enterprises and their partners introduce external data into the heterogeneous business context. Continuous discovery of new data sources can add significant value only if enterprises are able to quickly understand the business context and classify data.
If enterprises procrastinate deployment of scalable data governance processes and best practices, they end up facing undesirable consequences such as hefty cost, rework, and the shortfall in achieving desired business results during their data and analytics journey.
Next generation data governance
To put into practice efficient models for scalable data governance, enterprises need to consider a collaborative approach of humans with machines. This can be achieved through the Machine First approach, where machines have the first right of decision, augmenting human capabilities with AI and ML technologies. Data governance becomes much more powerful when assisted by AI-ML models as against simple rule-based automation. Researchers predict that in the next couple of years, AI-ML will reduce manual tasks in enterprise data management by 45%.
Manual data governance processes and actions are time-consuming and overwhelming particularly for critical areas such as data discovery, defining an association of business context with data, data classification, and defining data characteristics for identity. With a Machine First approach, machines can perform the first line of the heavy lifting of these critical areas. The Machine First approach to data governance enables the AI-ML models to learn from previous experience of data vigilance and usage. They then sense and understand the patterns in data and decide on the action plan to achieve scalable and agile data governance.
Machine First approach also enables optimal use of data in an enterprise’s data and analytics journey by assisting users to pick the right data. An enterprise-wide data governance strategy should be modeled on a holistic Machine First approach encompassing data discovery, data preparation (data quality), and data consumption.
Clearly identified and classified data with accurately associated business context lays a strong foundation for success. It also accelerates ROI realization in the data and analytics journey of an enterprise. Following are some of the patterns where the Machine First approach adds significant value:
Pervasive business context: Imagine if financial institutes can identify risks accurately if retailers can define store-opening strategy, or if aircraft manufacturers are able to clear audits in the first pass. One of the key common aspects across all these scenarios is the ability to apply business context on data.
Discovering business context from brochures, process documents, and even tribal knowledge is a highly laborious activity. A business glossary enables the contextual background to data and makes the business context pervasive. By leveraging AI-ML techniques along with NLP, enterprises can create a business glossary to augment human capabilities. AI-ML can classify structured as well as unstructured data and create a business glossary with minimal efforts and time. Human intervention can be reduced, and the available free time can be used for more insightful jobs.
Compartmentalizing data assets: Banks, retailers, healthcare service providers struggle for compliance towards various regulations such as GDPR and HIPPA. Enterprises should have the ability to clearly classify and label data at speed and scale, an impossible task if performed manually.
A machine led data governance approach can ensure accuracy, speed, and wider coverage while performing automatic data classification, and tagging.
Unique identity: Retailers often grapple with identifying substitute products that are similar. Financial institutions need to uniquely identify customers. Traditional pre-defined rule-based algorithms cannot scale as this criteria itself evolves frequently in line with changing business environments. The AI-ML data governance approach can learn from historical data, observe human actions, and discover patterns of its own for unique identification of the data subject (products, customers, etc.)
There are multiple flavors of these patterns applicable across various industries. In addition, there are other scenarios that span the data governance spectrum where a Machine First approach will add value.
Enterprises that embrace the Machine First data governance approach are more likely to derive sustainable desired business benefits. A strategic partnership with a service provider that approaches data governance with a Machine First mindset will enable enterprises to reach their goal faster. The service provider should be able to bring in contextualized frameworks, precise measurements, and customized blueprints to meet specific data governance needs of enterprises. To find out more about how to capitalize on data governance, write to us at firstname.lastname@example.org.
About the author(s)
Ajay Vaidya heads the Information Management solution group within the Analytics and Insights practice of Tata Consultancy Services (TCS). He is responsible for driving thought leadership and strategic growth in Information Management. With over 24 years of experience, Ajay leads large solution teams encompassing information management, master data management, analytics, and machine learning. His experience spans various industry verticals and the entire spectrum of consulting to delivery.