Earlier this month in London, more than 1,600 data and analytics leaders and professionals gathered for the Gartner Data & Analytics Summit. It was probably a surprise to no one that artificial intelligence (AI) took center stage. From niche breakout sessions to the packed opening keynote—where “AI” was one of three leading trends along with “data driven” and “privacy”— AI was everywhere.

And yet, the conversation around AI was different in London. The automate! automate! automate! drumbeat was replaced with a more nuanced picture, one where successful AI looks much more like a partnership between machines and humans.

Not Quite Human

There is a hotel in Japan that is completely automated. The front desk is manned by robots (some that look like animatronic dinosaurs), an automated trolley takes baggage to the rooms, and facial recognition replaces the need for manual keycards. Gartner’s Gareth Herschel, VP analyst, pointed out that although “Hotel Weird” presents a compelling novelty, machines just aren’t great at every aspect of service. One guest, for example, was continually woken up at night because the room’s virtual assistant confused his snoring for commands.

Even where AI has made incredible and stunning advancements, humans ultimately need to be involved. In her session, “What CDOs Need to Know About Emotion AI” Annette Zimmermann, VP analyst at Gartner, pointed to some truly incredible breakthroughs. By matching landmarks on the human face or identifying patterns in speech rate, pitch range, intensity, and voice quality, AI is able to detect human emotions — some algorithms can even detect 10-different emotions.

But even in this instance, Zimmermann said that the process isn’t completely automated. In a call center environment, for instance, the AI may detect whether a customer is angry, but after that it’s time for a human to take over. The angry customer is sent to a trained expert who is now prepared to deal with a disgruntled customer. Or in some cases, the call is sent to an employee who hasn’t dealt with an angry customer that day, ensuring that one employee doesn’t have to handle an unfair amount of difficult calls.

Automate the Manual, Promote the Creative

It turns out that machines are very good at certain tasks and pretty bad at others. Humans are also very good at certain tasks and pretty bad at others. Luckily those strengths and weaknesses tend to be complimentary. In her guest keynote “How to be Human in the Age of the Machine,” Hannah Fry, author and associate professor in the Mathematics of Cities at the Centre for Advanced Spatial Analysis at UCL, said that algorithms are great at sensitivity but lacking in specificity. In other words, algorithms are great at spotting anomalies and patterns but can’t hold a candle to the human ability to understand context and nuance.

In one of Fry’s examples, a group of radiologists were given CT scans of lungs to determine which lung nodules could be cancerous. Of the group of 24 radiologists, 20 failed to spot a gorilla superimposed in the last of the five CT scans — a clear lack of sensitivity, even among experts. If the tables were turned, however, algorithms are more likely to spot cancer where none exists — a human radiologist will never look at a normal group of cells and think they are cancerous.

Hannah’s session nicely supported Herschel’s call to “automate the manual, and promote the creative.” AI is not a fad, algorithmic decision-making is inevitable. But although “automate everything” is an easy rule to follow, he recommends focusing on augmenting the expertise and creativity of humans with the tireless ability of algorithms to spot anomalies and patterns.

Human Curation + Machine Learning

The way Herschel, Fry, and Zimmerman talked about AI in many respects reflects our vision for machine learning data catalogs. The Alation Data Catalog leverages machine learning to make data easier to find, understand, trust, use and reuse while allowing the human to ask more questions, share knowledge, and delve into deeper insights.

During his presentation “From Anarchy to Harmony: Use Machine-Learning-Enabled Data Catalogs to Maximize Investments in Distributed Data Assets,” Ehtisham Zaidi, Gartner principal analyst, said that machine-learning-enabled data catalogs are solving the two biggest challenges to data management: identifying data that delivers value and supporting data
governance. How? By leveraging Google-like smart search to find data assets; using automation and self-learning instead of burdening people with the need to manually update metadata in multiple places; and ensuring that metadata is maintained by the whole data community and is not dependent on a centralized IT team.

Zaidi’s vision for the value of machine learning data catalogs closely resembles the data cataloging vision presented by our Cofounder Aaron Kalb at Strata + Hadoop World 2016. What’s more, Zaidi and Gartner believe that this vision of a machine-learning-enabled data catalog creates real value for enterprises. According to Gartner, “By 2020, organizations that offer users access to a curated catalog of internal and external data will realize twice the business value from analytics investments than those that do not.”*

*Gartner, Magic Quadrant for Metadata Management Solutions, Guido De Simoni, Alan Dayley, Roxane Edjlali, 9 August 2018 


What is a Data Catalog eBook