Analyze Checks¶
Alation Cloud Service Applies to Alation Cloud Service instances of Alation
Maintaining data quality largely depends on minimizing failed checks. Alation Data Quality provides two ways to analyze check results where you can analyze failed sampled records and do a root cause analysis.
This requires user account credentials to fetch data for analysis. User credentials are used to establish a direct connection and honors your existing data source permissions.
Note
Alation does not store your user credentials.
Prerequisites¶
To add a user to establish an active connection to the data source, the user must already exist in the database with the necessary permissions to the table.
You must have JDBC URI connection details. To establish a direct connection with the data source, Alation Data Quality uses the JDBC URI connection details from Compose. For more information, see Configure Compose for OCF Data Sources and Working with Data Source Connections.
Failed Record Analysis¶
Failed records analysis allows you to investigate the specific data rows that caused a check to fail. By examining actual failed records, identifying patterns, and determining appropriate remediation strategies.
Alation Data Quality automatically generates the appropriate SQL query to retrieve failed records for the following check or filter types:
For other checks, Alation Data Quality requires user input to define which records are considered failures.
When you create a monitor and add checks, you can specify a custom query for retrieving failed records. If no query is provided, a SELECT (*) query is used to return all rows from a table.
To analyze failed records:
Navigate to the Alation Data Quality application and click the Monitors tab.
Search for the monitor you need to update.
Click on the monitor name to open the Monitors page.
Go to the Check Results tab that provides a list of job runs with its status for a monitor.
From the list of job runs, select the job you want to analyze and click View Details.
In the Actions column, click View Sample Failed Rows icon.
Choose a connection:
Company Connections:
Default ConnectionPrivate Connections: Select the one if you have configured it before. Otherwise, click + Add New.
Define the connection name.
Define the URI path for the data source. Use the format:
schema://path:portdepending on the data source as used in Compose.Click Save.
Connect as (Select User):
Select an existing user from the list.
To add a new user:
Click + Add New, then enter the username and password.
Click Save.
Click Set Active Connection to fetch sampled records.
A window appears with failed sample records and deviations.
View the Target Object, Check Type, Check Definition, and failed sample records.
You can also search for the data from the sampled records.
For an offline analysis, click Export CSV.
To close the analysis page, click Close.
Important
If you need to further fine-tune your sampled records, adjust the failed row sampling number in Monitors > Monitor Name > Settings > Advanced Settings > Failed Row Sampling. Set limits based on your typical analysis needs and query performance considerations. For more information, see Manage Monitors.
Root Cause Analysis (Beta)¶
Root cause analysis (RCA) capabilities leverage large language models to identify correlations and patterns within failed records, helping you quickly understand why checks are failing and what underlying factors may be contributing to data quality issues.
Alation Data Quality uses a privacy-preserving approach to analyze root cause for the failed records:
The RCA report uses the data from Failed Sample Rows and find correlation with the other existing rows using the large language model generated SQL query.
Only the masked data is sent to the LLM for SQL query generation to identify correlations and patterns.
The masked data in the generated SQL query is then replaced with the actual values using the user credentials and its permission framework.
The root cause is established based on the SQL query execution and actionable insights in a detailed report (including various prompts used by the LLM).
Note
No customer data is sent to any large language model (LLM). Only masked representations are analyzed, ensuring your sensitive data remains within your environment.
To find the root cause of a failed check:
Navigate to the Alation Data Quality application and click the Monitors tab.
Search for the monitor you need to update.
Click on the monitor name to open the Monitors page.
Go to the Check Results tab, which provides a list of job runs and their statuses for a monitor.
From the list of job runs, select the failed job you want to analyze and click View Details.
In the Actions column, click View Root Cause Analysis icon.
Choose a connection:
Company Connections:
Default ConnectionPrivate Connections: Select the one if you have configured it before. Otherwise, click + Add New.
Define the connection name.
Define the URI path for the data source. Use the format:
schema://path:portdepending on the data source as used in Compose.Click Save.
Connect as (Select User):
Select an existing user from the list.
To add a new user:
Click + Add New, then enter the username and password.
Click Save.
Click Set Active Connection to initiate the root cause analysis.
After processing, a window appears with details on failed sample data and AI analysis.
View the root cause analysis:
Analysis Details: Contains sampled failed data, masked data sample, schema profile, rule compiler prompt, rule compiler prompt, analyst prompt, analyst complete, SQL author prompt, SQL author complete.
Impact Analysis: Contains sample violation rate, failing rows, and target table.
Top Explanations: Explains the root cause of the error with further breakdown into segments and contexts.
To close the analysis page, click Close.