Overview

Alation Cloud Service Applies to Alation Cloud Service instances of Alation

Customer Managed Applies to customer-managed instances of Alation

Enhanced Connector Enhanced connectors add extended capabilities and require a separate entitlement in addition to your Alation platform license.

The Azure Data Explorer enhanced connector is used to catalog Azure Data Explorer data sources in Alation.

The connector extracts Azure Data Explorer objects, such as:

  • Cluster and database (cataloged as schemas)

  • Tables

  • Materialized views

  • Functions

  • Columns and column data types

After extraction, search and find Azure Data Explorer objects, curate the corresponding catalog pages, and understand data transformations using lineage diagrams.

The connector is distributed as a .zip package that you upload and install in the Alation application. Download the latest connector package from the Alation Customer Portal (Customer Portal > Connectors).

Team

Work with the following administrators to install and configure this connector:

  • Azure Data Explorer administrator

    • Creates a service account for Alation.

    • Provides connection details, including the JDBC URI.

    • Provides authentication details and assists with authentication configuration.

  • Alation Server Admin

    • Ensures that Alation Connector Manager is installed and running (or installs it).

    • Installs the connector.

    • Creates and configures the Azure Data Explorer data source in Alation.

    • Performs initial extraction and prepares the data source for Alation users.

Scope

The table below lists core features supported by the Azure Data Explorer Enhanced connector.

Feature

Scope

Availability

Authentication

Azure AD token authentication

Authentication using an Azure AD service account

Yes

Metadata extraction (MDE)

Default MDE

Extraction of metadata based on default queries built into the connector

Yes

Selective extraction

Ability to limit extraction using filters

Yes

Full extraction

Ability to run extraction across all accessible Azure Data Explorer databases

Yes

Extracted metadata objects

Cluster and database

List of clusters and databases

Yes

Table

List of tables

Yes

Materialized view

List of materialized views

Yes

Function

List of functions

Yes

Column

List of columns and their data types

Yes

Sampling and profiling

Table sampling

Retrieval of data samples from extracted tables

Yes

Column sampling

Retrieval of data samples from extracted columns

No

Custom query-based table sampling

Ability to use custom queries for sampling specific tables

Yes

Custom query-based column sampling

Ability to use custom queries for sampling specific columns

No

Deep column profiling

Value distribution statistics (profiling V2)

No

Dynamic profiling

Individual users connect with their own accounts to retrieve samples and profiles

Yes

Query log ingestion (QLI)

Table-based QLI

Ingestion of query history from a table or view

No

Custom query-based QLI

Ingestion of query history based on a custom query

No

JOIN and filter calculation

Calculation of JOIN and filter information based on ingested query history

No

Lineage

Table-level lineage

Auto-calculation of lineage based on MDE, and Compose queries

Yes

External table file lineage

Lineage for external tables

Yes

Column-level lineage

Calculation of lineage data at the column level (requires the Column-level Lineage Parser add-on)

Yes

Compose

Compose on customer-managed deployments

Customer-managed Compose on on-premise Alation instances

Yes

Compose on Alation Cloud Service

Compose on Alation Cloud Service instances (may require Alation Agent depending on network)

Yes

OAuth SSO in Compose

SSO via OAuth

No

Object mapping

Azure Data Explorer metadata extracted by the connector maps onto the Alation object model for RDBMS sources:

Data Source → Schema → Table → Column

Source object

Object in Alation

Cluster and database

Schema

Table

Table

Materialized view

Table

Function

Table

Column

Column