Alation Data Quality Standards and Library

Alation Cloud Service Applies to Alation Cloud Service instances of Alation

Alation Data Quality Standards provide a centralized, reusable framework for defining, governing, and enforcing enterprise-wide data quality expectations at scale. As data estates grow, manually configuring checks at the individual column level becomes unsustainable and leads to inconsistent governance.

DQ Standards solve this by allowing you to define reusable canonical expectations, the approved organization-wide rules for specific types of data, once in a central library and apply them consistently across thousands of columns.

  • Scalability: Rapidly grow your data quality coverage without increasing operational overhead by applying checks in bulk rather than one-by-one.

  • Governance and Ownership: By assigning clear owners to DQ Standards, you ensure that subject matter experts maintain the logic and that quality rules remain aligned with business requirements.

  • Composability: For maximum flexibility, design your standards to be narrowly focused. These standards can then be mixed and matched during monitor setup to meet complex data requirements.

  • High-Impact Discovery: Focus your initial efforts on high-impact standards for ubiquitous data types such as primary keys (IDs), email addresses, and financial dates to achieve immediate visibility into core data health.

Common Use Cases for DQ Standards

This section outlines how organizations can leverage the DQ Standard library to resolve systemic data integrity issues through automated, governed rule enforcement.

Automated Pattern Validation for String Fields

By defining a standard for validity check with a specific pattern-match check, you can enforce technical compliance across all columns typed as VARCHAR or String that are identified as containing email data. Applying this at scale ensures that downstream applications receive strings that conform to valid syntax requirements, preventing ingestion errors in customer-facing systems.

Integrity Controls for Financial Numeric Fields

By creating a standard for a consistency and completeness check for currency-related numeric fields, you can mitigate the risk of data corruption in financial reporting.

  • Range Validation: Prevents the ingestion of extreme outliers or negative values in fields where only positive values are logically valid.

  • Null Constraint Enforcement: Ensures that critical amount or transaction fields do not contain NULL or empty values, which would otherwise result in inaccurate aggregations during financial consolidation.

Standardized Monitoring for Sensitive Data (PII)

By creating a PII Governance Standard, you can apply consistent Uniqueness and Validity checks to sensitive columns across disparate environments (for example, Snowflake and Databricks). This provides a technical audit trail showing that all PII assets meet the same compliance-driven thresholds for data integrity.

DQ Coverage for New Asset Ingestion

By applying the programmatic or bulk application of existing DQ checks to metadata as soon as it is extracted and ingested into the Alation Catalog, you no longer need to manually author checks for a new table. The DQ Standards allows the immediate assignment of library-based monitors to the new assets. This ensures that newly discovered columns have active monitors and health scores from day one of their lifecycle in the catalog.

Create a Data Quality Standard

The Data Quality Standard library is the centralized repository where these definitions are created and managed. When building a standard, follow these steps to ensure long-term discoverability:

  1. In the Alation Data Quality application, go to Standards > Create New Standard.

  2. Specify a name and description that reflects the business intent (for example, “PII Column Minimum Quality”).

  3. Select the Column Data Type: All, Numeric, String, Date, or Boolean.

  4. Select the users you want to give access to the defined standards and click Next.

  5. In Configure Checks section, click Create Check.

  6. Define the check name and select one of the check categories:

    • Completeness: Missing counts or percentages.

    • Validity: Patterns, formats, or allowed values.

    • Freshness: Timeliness of data delivery.

    • Uniqueness: Duplicate detection.

  7. Define the threshold logic or passing condition using operators (for example, “Is Less Than”) and a numerical value.

  8. Review the configured settings, click Save to keep it as a Draft for internal review or click Publish to make it discoverable for the selected users to reuse.

Add a Data Quality Standards Monitor

Applying a Data Quality Standard is the process of taking a pre-defined set of rules from the Standards library and enforcing them across multiple data assets. This is achieved by creating a DQ Standard Monitor, which serves as the execution engine that runs the checks, evaluates the data against your thresholds, and manages downstream incidents .

To add a new monitor and apply defined standards in bulk:

  1. Navigate to the Alation Data Quality application or the catalog page of a specific table where you intend to apply standards and click Add Monitor.

  2. Select Apply DQ Standards to apply your pre-configured checks in bulk.

  3. In the Select Standard section, do the following:

    1. Browse the shared library for published standards (for example, “Null Check Standard” or “Email - Valid Format”).

    2. View the specific checks included (Completeness, Validity, etc.) and the date the standard was last updated.

    3. Use the built-in filters: All, Numeric, String, Date, Boolean to find standards compatible with your target data.

  4. In the Select Objects section, do the following:

    1. Search for specific columns or tables using the search bar.

    The UI displays the Name, Data Type, Source, and Breadcrumb (lineage path) for every column to ensure you are selecting the correct asset.

    1. Select multiple objects across different schemas or data sources simultaneously.

  5. (Optional) In Apply Filters (Performance Optimization) section, configure the filter:

    • Global Time Filter: Restricts the check to a specific time window. You may skip it if you need to evaluate the entire historical depth of a table for higher coverage.

      1. Toggle this on to apply a uniform time constraint across all selected assets.

      2. Define operators (for example, Less Than or Equal To), numerical values, and units (Days, Weeks, Months).

        Note

        The system automatically suggests partition columns (for example, sale_date) and provides a Confidence score (High or Medium) for the match.

  6. In the Preview section, review the configured settings:

    • Rule Validation: Review a human-readable summary of the checks to be created (for example, “Exactly 0 values should be missing in [Column]”).

    • Filter Audit: Verify that the partition columns and time-based filters are correctly mapped for each source before finalization.

  7. In the Configure Monitor section, configure the operational behavior of the monitor:

    • Name: Define a monitor name and provide a brief description.

    • Schedule: Set the frequency (Daily, Weekly, Monthly) and specific run-time.

    • Notifications: Select the means for notifications: None, Slack, Email, or Teams. For more information, see Configure Data Quality Alerts.

    • Priority: Select the priority level for the checks: Critical, High, Medium, or Low.

    • Incident Management: Enable Auto-Create Incident and integrate with Jira or ServiceNow. You can pre-configure the Incident Title and Description.

    • Advanced Settings: Configure Failed Row Sampling (default: 500 rows) to balance performance and detail.

    • Edit Permissions: Add specific users or groups allowed to edit and manage the monitor.

Manage Updates and Evolution

When a Data Quality Standard is modified, those changes do not automatically overwrite existing checks already deployed in the field. This prevents unintended breaking changes to your monitors and historical health scores. Instead, Alation provides a governed update workflow to propagate changes.

  • Existing Checks: By default, applied checks remain unchanged to maintain historical continuity.

  • Update Options: Users can choose to apply the new version to new columns only or reapply the updated standard to all existing columns.