Make Better Decisions, Faster: AI Innovations in Alation

The age of AI is upon us. Today, business leaders are racing to leverage AI for a range of use cases. Whether they seek to improve productivity and efficiency or drive innovation, AI can play a critical supporting role. For this reason, 84% of senior business leaders believe that AI will have a significant impact on their business operations in the future (Cisco AI Readiness Index). For users of Alation, that impact is already being felt. In fact, AI foundations have long formed the bedrock of the product: Alation’s Behavioral Analysis Engine learns from user behaviour to guide data best practices, while machine learning powers Query Log Ingestion (QLI) to make query writing simpler and easier for business users. And with the launch of ALLIE AI, we're making sure AI is infused across the entire product range.

In a recent video brief, I shared an overview of Alation’s AI-fueled intelligent curation feature delivered by ALLIE AI, Alation’s co-pilot for engineers, data analysts, and stewards. Read on to learn what we covered.

Suggested Stewards

For many, the value of a data intelligence platform comes down to trust. Newcomers can trust that the data they see are accurate, verified, and fit for use. Yet this requires a level of curation that demands time and energy from frontline data experts. The number one problem that customers raise is it's just really hard to curate the catalog, it takes time. However, this time investment pays off. Any data analyst can tell you that curation boosts time to value, as it ensures they can comprehend an asset quickly and utilise it effectively.

Now, with the Alation feature Suggested Stewards, users can streamline curation by finding candidate stewards faster. Here’s how it works: If you do not have more than one steward associated with a dataset, the system will go out using machine learning based on the popularity or the usage of the data and will try to assign stewards for you. These are your resident data experts, well-equipped to describe the data’s purpose, history, and applied uses accurately. By recognizing data stewards for the assets they already know well, leaders can connect people with questions to people with answers.

Suggested Descriptions in action

This is not to say that summarising a data table into a succinct description is easy for its designated steward. So how can we make description writing easier? To address the problem of a data steward’s writer’s block, Alation has released suggested descriptions. The steward simply hits a button, and the large language model comes back with a description of what the table contains, including a summary of all the columns, an explanation of what the table does, and suggestions about how it can be used.

How does this work?  To build these descriptions, Alation uses an LLM to analyse metadata securely. We basically bring in some metadata from the instance and then send it over to the large language model with a carefully crafted prompt (question) for the LLM to reply with a description.

Customers can rest assured this process is secure.  People often ask me, ‘Are you storing information in the LLM? How much of my data is moving?’ We do not tune the LLM. We are not storing your information in the LLM. We are just asking the LLM the question in the same way that you might ask ChatGPT a question. We also only use the metadata, we don’t use any of the underlying data. All of this ensures we keep your data and metadata secure.

The AI-fueled data intelligence platform

These AI tools sit under a suite of features we broadly label intelligent curation. The value of intelligent curation is most evident when we use semantic search. A well-curated model supports better search, allowing users to find not just the data but the information about the data they are looking for.

What’s more, these innovations represent just the tip of the AI-for-data-intelligence iceberg. In the future, product leaders are exploring bulk suggested descriptions, suggested descriptions for columns, and suggested descriptions for BI reports. All the while analysing feedback to continuously improve the quality of the features.

See how it works: watch the video brief here.