The Very Human Learning Curve for Artificial Intelligence

Research and Innovation

The Very Human Learning Curve for Artificial Intelligence

November 14, 2018

Human intelligence has evolved over a period of time – beginning with discovering how to light fires and inventing the first wheel, to researching and understanding extremely complex phenomena. This process has involved generations of the human race learning over millennia through systematic efforts of discovery, experimentation, deduction, and induction – and, at times, sheer serendipity. 

How easy (or hard) is it then, for artificial intelligence (AI) to emulate human-like behavior? AI systems of today in many ways try to replicate the ‘sense-think-do’ cycle, building a corpus of learning as they go through each step.  Some examples include intelligent bots used for customer interactions, intelligent search engines, and intelligent information extractors, each with their own unique complexities and challenges.

Building the capabilities to perform an end-to-end task successfully often involves stringing together one or more competencies, adding further complexity to the process. Overlooking for a minute the versatility of human beings, let us explore the intricacies involved in building even a single human-like capability. Consider an intelligent system capable of producing theme-specific newsletters for circulation to relevant groups within an enterprise. 

The Search for Good Content

A human being looking for information would begin with an understanding of the sources of information available and the knowledge of how to access that information. An AI solution, on the other hand, would need to know the repositories of information available for it to search in – will it extend to the enterprise’s intranet or the internet? Which enterprise knowledge repositories or information service subscriptions can it access? 

Having identified the sources of information, the AI system would need to figure out the mechanism of accessing information from each of these repositories. Considering that each repository may differ in the way it stores and shares information, the solution will need to have the ability to ‘talk’ to each of the sources – a challenge typically addressed by creating adapters to facilitate the connect.

Beyond identifying the sources of information and knowing how to access the same is the challenge of identifying what to search for. For example, a human searching for information on AI would naturally extend the search to terms such as cognitive intelligence, machine learning, deep learning, and neural networks. An intelligent system will similarly need to extend the search terms to synonyms, related phrases, and similar concepts – an ability that requires defining a dictionary relevant to the domain or theme.

Curating the Best Stories

Once the search yields a comprehensive set of articles, the next step would be to filter out duplicates. While catching exact duplicates is easy, identifying articles similar in their coverage and intent is not. Very often, two articles with a similar set of keywords could present drastically different views – some might be offensive as well. While natural language understanding (NLU) has significantly evolved as a technology, augmenting a human-in-loop approach might be a good solution to these more challenging AI problems.

The next step would be to organize all the information gathered – sorting the articles based on common traits such as sub-themes, timelines, and companies referenced, for instance. For an AI system, this would mean using one or more approaches such as rule-based correlation, unsupervised learning, and supervised learning using human-in-loop to arrive at clusters of related articles.

To ensure that the most interesting articles appear at the top, an intelligent solution would also need to prioritize articles based on parameters such as relevance, author ranking, quantitative data backing, and audience preferences. One would have to leverage technologies such as NLU, deep learning, and rule-based reasoning to build models for ranking articles based on different parameters.

Closing the Loop

Finally, the solution will need to compose the newsletter as a collection of summaries of the various articles that it has collected – duly sanitized, grouped, and prioritized. Summarization typically involves one of two approaches – extraction, which identifies the most prominent sentences based on the frequency of occurrence of key terms or position of sentences in the text; and abstraction, which involves comprehensive understanding of the piece based on NLU and lexicon analysis. Today, extraction-based summarization would be the likely choice, and will likely need to be supported by a human-in-loop. The summaries thus generated will then have to be collated as per their groups and rankings to compose the final newsletter. With this step, will the AI system be done at last? Well, not quite…    

Humans are naturally wired to use their experiences to refine their behavior. To make an intelligent system similarly learn by itself and improve with time and experience, one would need to build mechanisms to capture experiences. This would entail human-in-loop interventions during various stages of producing the newsletter, and then using this learning to fine-tune the underlying AI models. The learning captured could be generic for the entire user community, differentiated by user segments, or specific to each user. All this adds newer challenges related to conflicting inputs, inconsistent user behavior, and user privacy. 

To accelerate the adoption of AI systems and make the technology truly pervasive, there is a need for platforms that can simplify the development of AI solutions by providing the right levels of abstraction of cognitive capabilities, and the ability to assemble, customize, and extend the same. This is why shifting the power of developing AI applications from developers to business users will be the ideal way forward.

Bhasha Khose is an Entrepreneur-in-Residence (EIR) for the Cognitive Transformation Incubation Program at TCS. Her vision is to accelerate enterprise AI adoption in order to build the intelligent applications of tomorrow that can think, correlate, reason, retrieve, interact, and learn naturally, just like humans. With over 15 years of TCS experience, she leads platform and solutions development efforts for the rapid development of business-focused AI applications.