OpenLineage Integration

Alation Cloud Service Applies to Alation Cloud Service instances of Alation

OpenLineage is an open standard for collecting lineage metadata from data pipelines. It defines a common format for capturing job runs and the datasets they read and write.

Alation supports OpenLineage events from any system that can emit them according to the OpenLineage specification. This includes:

• Orchestration platforms (Apache Airflow, Dagster, Prefect, etc.)

• Data processing frameworks (Spark, Flink, etc.)

• Custom applications and scripts

• Any system that can send OpenLineage-compliant events over HTTPS

This section explains how to configure systems to send OpenLineage events to Alation.

OpenLineage Integration Workflow

1. Your system emits OpenLineage events during job execution.

   • Events follow the OpenLineage specification format

   • Events include job context, inputs, outputs, and metadata

2. Events are sent to Alation’s ingestion endpoint over HTTPS.

   • Uses Alation API token for authentication

   • Events can come from any system or platform

3. Alation processes events and builds lineage.

   • Creates cross-source lineage between data sources

   • Appears on Lineage tab of catalog objects

   • Enables Impact Analysis across systems

4. Lineage becomes visible in the catalog.

   • Shows data flow across pipelines

   • Includes job metadata and execution details

   • Works across different data sources

Prerequisites

To send OpenLineage events from any supported system to Alation:

Event Format Requirements

• Events must follow the OpenLineage specification.

• Each event must include both input datasets (sources) and output datasets (targets).

• Events must be sent as COMPLETE events (failed events don’t create lineage).

Network and Authentication

• HTTPS connectivity from the supported system to Alation (TCP port 443).

• API access token from Alation.

• Alation ingestion endpoint: https://<your-tenant>.alationcloud.com/open_lineage_event/

Requirements in Alation

• Data sources must be cataloged in Alation.

• Metadata extraction must have been run at least once.

• Alation resolves the host and port from the OpenLineage namespace. The host and port in the namespace must match the JDBC URI in the Alation data source, or a value configured in Additional Datasource Connections.

Requirements in Your System

• Ability to emit OpenLineage events (via library, plugin, or custom code).

• Configuration to point events to Alation’s endpoint.

• Namespace configuration. Specify a URL-like namespace whose host and port match the corresponding Alation data source.

Integration Guides

See detailed integration guides for commonly-used systems:

• Platform-specific integrations:

  • Apache Airflow: See Apache Airflow Integration (Beta) for Airflow configuration (self-managed, MWAA, Cloud Composer).

• Direct API integration:

  • Custom Systems & Direct HTTP POST: See Direct API Integration for OpenLineage Events for technical details, API reference, code examples, and implementation guidance. Use this if your system doesn’t have a pre-built OpenLineage integration.

• Other supported systems: If your system has an existing OpenLineage client library or provider (such as Dagster, Prefect, Spark, Flink), configure it to point to Alation’s endpoint using the general prerequisites above and your system’s OpenLineage documentation.

For help with OpenLineage integration issues, see OpenLineage Integration Troubleshooting or contact Alation Support.