Troubleshoot the OCF S3 Connector Configuration

Alation Cloud Service Applies to Alation Cloud Service instances of Alation

Customer Managed Applies to customer-managed instances of Alation

Terraform Configuration

1. AWS Error while executing a script:

   • Make sure that you have provided a valid permission to the user which is used in the script

2. Bucket missing error:

   • Make sure the buckets present in the source-bucket-list.json file exist in S3

3. Access denied error:

   • Make sure you have required access while executing a script

   • Make sure you have added a correct region

4. Cannot create a bucket error:

   • Make sure you have given required permission and also a unique name to target_bucket_for_inventory

5. Cannot create a Lambda function error:

   • Make sure you have given required permission and also a unique name to lambda_function_name

Alation Configuration

Note

Prior to using this section, Alation recommends you to refer to the error messages displayed in the user interface and take suggested actions.

1. Test connection issue:

   • Ensure the AWS region provided is accessible, valid, and active in the user’s AWS account

   • Ensure the AWS access key and secret key are correct

   • Ensure you have provided the correct region and the S3 endpoint of the region is accessible

   • Make sure the user has access as described in Configure Permissions for Authentication

   • For STS-based authentication, ensure the STS endpoint is accessible

2. No inventory reports found:

   • Ensure the destination or inventory bucket is exist, is correct, and has required permissions

   • Ensure the inventory bucket is in the same region as specified in the General Settings tab

   • Ensure If the inventory folder has folders that have reports of the source buckets for which inventory was configured.

   • If the destination bucket is correct then wait 48 hours after setting up inventory for inventory generation and try again

3. MDE or filter extraction failure due to access issue:

   • Make sure the user has access to destination bucket

4. Duplicate columns when column name contains special characters:

   • After you upgrade the Amazon S3 OCF connector to version 3.8.6 or later, you may notice duplicate columns for columns with names containing special characters. To remove duplicates, enable Remove Buckets from the catalog that are not captured by the lists below before you run the next schema extraction.

     Note

     This will also remove all the buckets that are not part of the selective extraction list.

5. MDE or bucket list error:

   • Error:

     • File is initialized without providing a rootPath

     • File path can’t be null or empty

   • Troubleshooting:

     • Do not reuse buckets for the destination inventory bucket for this connector

     • Ensure that all buckets in the destination bucket are named properly

     • Verify the Inventory Configuration for each source bucket you are ingesting

     • If there is already a bucket path that has a blank name or a forward slash / in the destination inventory bucket, remove it. Review the inventory configuration for each source bucket

Lambda Function Logs

1. Go to Amazon S3 console > Lambda Functions section.

2. Select the lambda function that you have created and go to the Monitor section.

3. Click View Logs in Cloudwatch. It will open in a new tab.

4. In Cloudwatch, search for a particular log based on a date or prefix.

Also refer to Troubleshoot OCF Connectors.

Schema Extraction Did Not Start

If both Extract Files at each bucket level and Use Schema Path Pattern are disabled, the Amazon S3 OCF connector will not initiate schema extraction, even if Test Connection and Metadata Extraction (MDE) are successful.

This is expected behavior. With both options disabled, there are no columns available for extraction at either the file or directory level.

Refer to the table below for details on how schema extraction behaves based on different combinations of the Use Schema Path Pattern and Extract Files settings.

Use Schema Path Pattern	Extract Files	Behavior
Enabled	Enabled / Disabled	Schema is extracted at the directory level for matching patterns. Columns appear only on the directory catalog page.
Disabled	Enabled	Schema is extracted for each individual file. Columns appear on the file catalog page.
Disabled	Disabled for all filtered buckets	No schema is extracted as there are no columns to extract.

MDE Job Duration Increases After Disabling File Extraction for the first run

This is an expected behavior. On the first run after disabling Extract Files, Alation removes previously extracted files, which can result in a longer processing time.