Anyone who has followed the travails of the U.S. newspaper industry knows it has been in a steep decline since the rise of the Web in the 1990s. Between 2003 and 2014, the industry’s annual advertising revenue plunged 56%, from $46 billion to $20 billion. Numerous newspapers have folded (there were 125 fewer U.S. dailies in 2014 than in 2003), and several prominent newspaper chains have gone in and out of bankruptcy over the last two decades.
Nonetheless, for Jim Kennedy, the future of journalism could not be brighter, or more exciting, because he sees technology transforming it. “I think the next 20 to 25 years will be amazing,” says Kennedy, senior vice president of strategy and enterprise development for The Associated Press, the 170-year-old news service. “We will see the rebirth of the journalist.”
The AP is one of the largest newsgathering operations in the world. It is operated as a not-for-profit cooperative, owned by the daily newspapers of the United States and serving 15,000 news outlets worldwide. It reported $568 million in revenue last year.
Kennedy has had an excellent perch from which to observe the recent evolution of the newspaper business. He has been a fixture at AP since 1987, except for an 18-month stint at the turn of the century, as head of planning and product development at The Wall Street Journal’s online edition. In 1995, Kennedy witnessed the birth of Craigslist, one of many websites whose free want-ads siphoned off classified ad revenue from newspapers to the tune of $5 billion alone between 2000 and 2007, according to one estimate.1 A year later, he started the AP’s first department devoted to launching digital products. AP realized back then that it needed to deal with online competitors, and also to start figuring out how to leverage technology to work for, not against, newspapers.
AP’s latest digital initiative is one that harnesses artificial intelligence to write stories.
AP’s latest digital initiative is one that harnesses artificial intelligence to write stories. In 2013, several AP journalists alerted Kennedy to a North Carolina company, Automated Insights, whose AI technology was being used to write releases for the Cleveland Indians baseball team and sports stories for Yahoo. Kennedy and others at AP were intrigued. They put the software to a small test: writing short articles on the weekly statistical performances of National Football League offensive players—stories AP’s newspaper customers could use to supplement their own NFL coverage.
The technology worked. “I thought to myself, ‘That is the acid test,’” Kennedy remembers. “The point was to take a data stream and see if it could be turned it into a narrative blurb [of text] that looked like a human wrote it.” For Kennedy, the experiment validated three important proofs of concept: 1) the software could produce new content that AP’s writers didn’t have time for, yet member newspapers coveted, 2) the content was indistinguishable from stories written by humans, and 3) it could be done rapidly.
The following year, in 2014, the experiment gave AP the confidence to turn the technology loose on another kind of article. AP and Automated Insights trained the AI program to write quarterly earnings stories on publicly held companies. There are 5,300 companies listed on U.S. stock exchanges, far too many for AP’s 65 business journalists to write stories about when the firms announce their quarterly results. In fact, AP’s business reporters wrote about 300 such stories every three months, typically only about the biggest companies (Apple, McDonald’s, and so on) that newspaper readers were interested in. By the end of 2015, after working to train the software to write these 150-word articles using financial data from Zacks Investment Research, the AI was spitting out 3,700 earnings stories every quarter—more than 12 times as many as was possible before.
Importantly, Kennedy notes, AI was not being used to take work away from journalists. It supplemented their work, while giving AP’s daily newspaper owners more of what they wanted: earnings stories covering the smaller companies in their locales, companies with which their local readers had an intimate connection. Training the technology to write short articles about the results of sports games is next, says Kennedy.
To Kennedy, this is merely the beginning of how AI will help AP’s journalists dramatically improve the way they gather, analyze, and produce news. He sees AI giving the AP three big opportunities for the rest of the decade and beyond: continuing to produce content that its reporters do not have the time to write; giving journalists tools to see patterns and make connections from the mountains of data that have been collected about people, organizations, and issues; and enabling new readers to get answers to specific questions about an organization (company, sports team, government agency, and so on), people, and issues based on AP’s ability to tag items in each article at more granular levels.
Tagging will allow AP to “make our news reports more granular so that we can move it into all kinds of applications that would support vertical industry insights, research use cases, and the on-demand consumer world,” Kennedy explains. “We have an opportunity to expand to other customer segments.”
With that in mind, Kennedy believes the newspaper industry’s embrace of AI has just begun. “I predict that eventually the technology will produce more game stories in sports than humans will,” he says. That, in turn, will not only result in more stories being produced, but it will free up journalists to get the story behind the story, focusing on the more complex articles that readers crave but take more time to produce.
“We have to free our people to produce the kinds of content that no one else has,” concludes Kennedy.
1. Robert Seamans (New York University Stern School of Business) and Feng Zhu (Harvard Business School), “Responses to Entry in Multi-Sided Markets: The Impact of Craigslist on Local Newspapers,” Management Science, November 2013. Accessed June 27, 2016. http://pubsonline.informs.org/doi/abs/10.1287/mnsc.2013.1785