Prerequisites¶
Alation Cloud Service Applies to Alation Cloud Service instances of Alation
Customer Managed Applies to customer-managed instances of Alation
Enhanced Connector Enhanced connectors add extended capabilities and require a separate entitlement in addition to your Alation platform license.
Supported Deployment Types¶
Alation Cloud Service (ACS)
Customer-managed Alation
Connector Requirements¶
Alation Connector Manager (ACM) must be installed and running for OCF connectors.
For ACS, ensure the Alation Agent is installed and running.
The connector is compatible with Alation version 2024.3 and higher.
Network and Firewall Configuration¶
Ensure the following network connectivity from the connector host:
Endpoint |
Port |
Protocol |
Purpose |
|---|---|---|---|
|
443 |
HTTPS |
MaxCompute OpenAPI |
|
443 |
HTTPS |
DataWorks API |
|
80 |
HTTP |
JDBC endpoint |
|
443 |
HTTPS |
Alibaba Cloud STS endpoint (required when using STS with AssumeRole authentication) |
Replace <region> with your Alibaba Cloud region (for example, cn-hangzhou, us-east-1).
Authentication Requirements¶
The connector supports the following authentication types for metadata extraction (MDE), query log ingestion (QLI), and profiling/sampling that use the configured service account credentials:
Access Key (default): The connector uses a long-lived RAM user Access Key ID and Access Key Secret for service-account-based calls to MaxCompute and DataWorks.
STS with AssumeRole (available from version 1.1.0): The connector uses a RAM user Access Key only to call the Alibaba Cloud Security Token Service (STS)
AssumeRoleoperation. The connector then uses the short-lived temporary credentials returned by STS for service-account-based calls to MaxCompute and DataWorks. Temporary credentials are refreshed automatically before they expire.
Compose does not use the service account credentials, so these authentication types do not apply to Compose calls.
Both authentication types require Alibaba Cloud RAM (Resource Access Management) credentials:
Access Key ID: The RAM user’s Access Key ID
Access Key Secret: The RAM user’s Access Key Secret
Creating a RAM User¶
Log in to the Alibaba Cloud RAM Console.
Navigate to Users and click Create User.
Enable Programmatic Access for the user to generate Access Keys.
Save the Access Key ID and Access Key Secret securely.
Access Requirements¶
The MaxCompute connector requires multiple levels of access. Configure the following permissions based on the features you plan to use.
1. RAM Policy: AliyunMaxComputeFullAccess¶
Used by: Test Connection, Filter Extraction (project discovery), MDE
Attach the AliyunMaxComputeFullAccess managed policy to the RAM user, or create a custom policy with the following permissions:
API Action |
Feature |
Purpose |
|---|---|---|
|
Test Connection, Filter Extraction, QLI |
Discover all projects in the tenant |
|
MDE |
List tables in each project |
|
MDE |
Get table details (columns, types, comments) |
|
MDE |
List UDFs in each project |
2. RAM Policy: AliyunDataWorksFullAccess¶
Used by: Lineage
Attach the AliyunDataWorksFullAccess managed policy to the RAM user, or create a custom policy with the following permissions:
API Action |
Feature |
Priority |
Purpose |
|---|---|---|---|
|
Lineage |
Required |
Map MaxCompute projects to DataWorks project IDs |
|
Lineage |
Required |
List tables with metadata (owner, description, create/modify time) |
|
Lineage |
Required |
List columns with types, comments, PK/FK/partition info |
|
Lineage |
Required |
Discover data sources in each DataWorks project |
|
Lineage |
Required |
Get table-level and column-level lineage relationships |
|
Lineage |
Optional |
Enrich dataflow with script content, status, owner, execution times |
|
Lineage |
Optional |
Enrich dataflow with scheduling metadata (cron, script path, recurrence) |
3. MaxCompute Tenant-Level Role: infoschema_admin¶
Used by: QLI (Query Log Ingestion)
Requirement |
Details |
|---|---|
Role |
|
Assigned via |
MaxCompute Console > Tenants > Users > Assign Role |
Grants access to |
|
Enables |
|
Not a RAM policy |
This is a MaxCompute internal role, not an Alibaba Cloud RAM policy |
Role Policy:
{
"Statement": [{
"Action": ["odps:Describe", "odps:Select"],
"Effect": "Allow",
"Resource": ["acs:odps:*:catalogs/system_catalog/schemas/information_schema/tables/*"]
},
{
"Action": ["odps:List"],
"Effect": "Allow",
"Resource": ["acs:odps:*:catalogs/system_catalog/schemas/information_schema"]}
],
"Version": "1"
}
4. MaxCompute Project-Level Membership (JDBC Access)¶
Used by: QLI (entry point), Profiling, Compose
The RAM user must be added as a member to each project they need JDBC access to. Assign membership via MaxCompute Console > Project > Members.
Requirement |
Feature |
Purpose |
|---|---|---|
|
QLI |
JDBC entry point for SYSTEM_CATALOG query |
|
Profiling (sampling) |
Execute SELECT MIN(), MAX(), AVG()… queries per-project |
|
Compose |
Execute user-written SQL queries |
Grant project-level access using the MaxCompute console or the following command:
-- Grant read access to a user
GRANT Read, CreateInstance ON PROJECT <project_name> TO USER <ram_user>;
-- Or assign a built-in role
GRANT role_project_reader TO USER <ram_user>;
5. RAM Permissions for STS with AssumeRole¶
You require the following permissions for STS with AssumeRole authentication (available from version 1.1.0):
The connector calls the Alibaba Cloud STS
AssumeRoleAPI to obtain temporary credentials.The credentials are used for service-account features such as MDE, Lineage, QLI, and profiling/sampling.
This requires both a caller RAM user and a target RAM role, configured as described below.
Caller RAM User
The RAM user whose Access Key is configured in the connector must have permission to assume the target role. Attach a custom policy with the following permission to the caller RAM user:
{
"Version": "1",
"Statement": [
{
"Effect": "Allow",
"Action": "sts:AssumeRole",
"Resource": "acs:ram::<account-id>:role/<role-name>"
}
]
}
Replace <account-id> with your Alibaba Cloud account ID and <role-name> with the name of the RAM role the connector will assume.
Target RAM Role
Create a RAM role that the caller RAM user assumes at runtime. Configure the role as follows:
Trust Policy: The role’s trust policy must permit the caller RAM user as a trusted principal. For example:
{ "Statement": [ { "Action": "sts:AssumeRole", "Effect": "Allow", "Principal": { "RAM": ["acs:ram::<account-id>:user/<caller-ram-user>"] } } ], "Version": "1" }
Permissions: Attach the permissions required for the role or the features you plan to use. Include the RAM policies and the
infoschema_admintenant role as needed. Grant project-level membership to the assumed role only for service-account JDBC features, such as query log ingestion (QLI) and profiling/sampling. Compose is not affected by STS and continues to use per-user credentials, so each Compose user still needs their own project access.Maximum Session Duration: At the role’s maximum session duration, set a value greater than or equal to the session duration you configure in the connector. The connector accepts session durations between 900 and 43200 seconds (default: 3600).
DataWorks Configuration¶
For lineage extraction and enhanced metadata, ensure:
DataWorks Data Map is enabled for your workspace.
The MaxCompute projects are registered as data sources in DataWorks.
The RAM user has access to the DataWorks workspace.
Note
Lineage extraction uses the DataWorks Public API. If DataWorks is not configured or the RAM user lacks permissions, lineage extraction will be skipped but metadata extraction will continue.
JDBC Requirements¶
For QLI, Compose, and Profiling, the connector uses MaxCompute JDBC:
JDBC Driver:
com.aliyun.odps.jdbc.OdpsDriverversion 3.10.5 (bundled with the connector)JDBC URL Format:
jdbc:odps:http://service.<region>.maxcompute.aliyun.com/api?project=<project_name>(thejdbc:prefix is optional)
Authentication:
Compose: Per-user authentication. Each user provides their own Access Key ID (as username) and Access Key Secret (as password). Service account credentials are not used.
Sampling and Profiling: Uses service account Access Key ID and Access Key Secret.
QLI: Uses service account Access Key ID and Access Key Secret with tenant-level
infoschema_adminrole.