Amazon Web Services PrivateLink

Amazon Web Services’ PrivateLink is an AWS service that provides private connectivity between VPCs without exposing traffic to the public Internet. Keeping traffic in the Amazon network reduces the data security risk associated with exposing your Warehouse traffic to the Internet.

During the Private Beta, you can set up AWS PrivateLink for Databricks, RDS Postgres, and Redshift.

Databricks

Prerequisites

Before you can configure AWS PrivateLink for Databricks, complete the following prerequisites in your Databricks workspace:

• Databricks account must be on the Enterprise pricing tier and use the E2 version of the platform.
• Databricks workspace must use a Customer-managed VPC and Secure cluster connectivity.
  • Configure your VPC with DNS hostnames and DNS resolution
  • Configure a security group with bidirectional access to 0.0.0/0 and ports 443, 3306, 6666, 2443, and 8443-8451.

Configure PrivateLink for Databricks

To configure PrivateLink for Databricks:

1. Follow the instructions in Databricks’ Enable private connectivity using AWS PrivateLink documentation. You must create a back-end connection to integrate with Segment’s front-end connection.
2. After you’ve configured a back-end connection for Databricks, request access to Segment’s PrivateLink integration by reaching out to your Customer Success Manager (CSM).
3. Your CSM sets up a call with Segment R&D to continue the onboarding process.

The following Databricks integrations support PrivateLink:

• Databricks storage destination
• Databricks Reverse ETL source

RDS Postgres

Prerequisites

Before you can configure AWS PrivateLink for RDS Postgres, complete the following prerequisites in your Databricks workspace:

• Set up a Network Load Balancer (NLB) to route traffic to your Postgres database: Segment recommends creating a NLB that has target group IP address synchronization, using a solution like AWS Lambda.
• Configure your NLB with one of the following settings:
  • Disable the Enforce inbound rules on PrivateLink traffic setting
  • Add an inbound rule that allows traffic belonging from Segment’s us-east-1 PrivateLink/Edge CIDR: 10.248.64.0/18

Configure PrivateLink for RDS Postgres

1. Create a Network Load Balancer VPC endpoint service using the instructions in the Create a service powered by AWS PrivateLink documentation.
2. Reach out to your Customer Success Manager (CSM) for more details about Segment’s AWS principal.
3. Add the Segment AWS principal as an “Allowed Principal” to consume the Network Load Balancer VPC endpoint service you created in step 1.
4. Reach out to your CSM and provide them with the Service name for the service that you created above. Segment’s engineering team provisions a VPC endpoint for the service in the Segment Edge VPC.
5. After creating the VPC, Segment provides you with private DNS so you can update the Host in your Segment app settings or create a new Postgres integration.
   The following RDS Postgres integrations support PrivateLink:
   • RDS Postgres storage destination
   • RDS Postgres Reverse ETL source

Redshift

Prerequisites

• You’re using the RA3 node type: To access Segment’s PrivateLink integration, use an RA3 instance.
• You’ve enabled cluster relocation: Cluster relocation migrates your cluster behind a proxy and keeps the cluster endpoint unchanged, even if your cluster needs to be migrated to a new Availability Zone. A consistent cluster endpoint makes it possible for Segment’s Edge account and VPC to remain connected to your cluster. To enable cluster relocation, follow the instructions in the AWS Relocating your cluster documentation.
• Your cluster is using a port within the ranges 5431-5455 or 8191-8215: Clusters with cluster relocation enabled might encounter an error if updated to include a port outside of this range.

Configure PrivateLink for Redshift

Implement Segment’s PrivateLink integration by taking the following steps:

1. Let your Customer Success Manager (CSM) know that you’re interested in PrivateLink. They will share information with you about Segment’s Edge account and VPC.
2. After you receive the Edge account and VPC, grant cluster access to Segment’s Edge account and VPC.
3. Reach back out to your CSM and provide them with the Cluster identifier for your cluster and your AWS account ID.
4. Segment creates a Redshift managed VPC endpoint within the Segment Redshift subnet on your behalf, which creates a PrivateLink Endpoint URL. Segment then provides you with the internal PrivateLink Endpoint URL.
5. After Segment provides you with the URL, use it to update or create new Redshift integrations. The following integrations support PrivateLink:
   • Redshift storage destination
   • Redshift Reverse ETL source