Configuring a Private Network

You can configure your data catalog to access data sources hosted in private networks.

By configuring data catalog to access a private network, you can:

  • Harvest Oracle Cloud Infrastructure data sources that are only accessible privately.
  • Harvest on-premise data sources that are connected to an Oracle Cloud Infrastructure Virtual Cloud Network (VCN) using VPN Connect or FastConnect.
Important

You can access and harvest on premise or private data sources in Data Catalog using only their Fully Qualified Domain Name (FQDN). For example, <hostname>.<subnet DNS label>.<VCN DNS label>.oraclevcn.com

To allow your data catalog to access a private network, you must:

  1. Create a private endpoint for your data catalog.
  2. Attach the private endpoint to your data catalog.
  3. Use the private endpoint while creating a data asset.

Prerequisites

One of the ways Oracle Cloud Infrastructure lets you configure private access for your resources is using private endpoints.

Data Catalog uses private endpoints to access the private network where your data sources are hosted. You must have the required data catalog permissions to use the Data Catalog private endpoints.

Additionally, to create, update, or delete private endpoints in Oracle Cloud Infrastructure, you need to obtain certain permissions in Oracle Cloud Infrastructure Identity and Access Management (IAM) for the relevant compartments in your tenancy. The following table lists the required permissions for virtual networking resources in Oracle Cloud Infrastructure for the private endpoint operations.

Operation Required Access on Underlying Resources
Create a private endpoint

For the private endpoint compartment:

  • Create VNIC (VNIC_CREATE)
  • Delete VNIC (VNIC_DELETE)
  • Update members in a network security group (NETWORK_SECURITY_GROUP_UPDATE_MEMBERS)
  • Associate a network security group (VNIC_ASSOCIATE_NETWORK_SECURITY_GROUP)

For the subnet compartment:

  • Attach subnet (SUBNET_ATTACH)
  • Detach subnet (SUBNET_DETACH)
Update a private endpoint

For the private endpoint compartment:

  • Update VNIC (VNIC_UPDATE)
  • Update members in a network security group (NETWORK_SECURITY_GROUP_UPDATE_MEMBERS)
  • Associate a network security group (VNIC_ASSOCIATE_NETWORK_SECURITY_GROUP)
Delete a private endpoint

For the private endpoint compartment:

  • Delete VNIC (VNIC_DELETE)
  • Update members in a network security group (NETWORK_SECURITY_GROUP_UPDATE_MEMBERS)

For the subnet compartment:

  • Detach subnet (SUBNET_DETACH)
Important

If you are managing the data catalog private endpoints resource, we recommend that you also have the manage work requests permission. This ensures that you are able to view the logs and error messages that are encountered while working with private endpoints.

Creating a Private Endpoint

Oracle Cloud Infrastructure lets you create private endpoints within your service so that you can access resources that are only available using private IPs. In Data Catalog, you create a private endpoint to configure the private network where your data source is hosted.

Before you create a private endpoint in Data Catalog, you must have the following details:

  • The name of the Virtual Cloud Network (VCN) used to access your data source.
  • The name of the subnet in the VCN.
  • The list of DNS zones used to resolve the Fully Qualified Domain Names (FQDNs) of the data sources that you want to harvest.
    Important

    You must only specify the FQDNs of the data sources in this list. For private autonomous databases, use the FQDN of the database as the DNS zone. For custom data sources running on Oracle Cloud Infrastructure compute Virtual Machines (VMs), you can specify the FQDN of the VM, or the domain name of the subnet in which the VM is provisioned, or the domain name of the VCN.

Here's how you create a private endpoint:

  1. In the Console, open the navigation menu, and then under Data and AI, click Data Catalog.
  2. From the Data Catalog service page, click Private Endpoints.
  3. Click Create Private Endpoint.
  4. Select the compartment where you want to create the private endpoint. You can create the private endpoint in a different compartment than the compartment where your data catalog is created.
  5. Enter a name to identify the private endpoint.
  6. Select the VCN that is created to provide private access to your data source.
  7. Select the Subnet that has the private endpoint to access your data source.
  8. Enter the DNS zones to resolve. The DNS zone should be a Fully Qualified Domain Name (FQDN). You can enter upto 30 DNS zones.
  9. (Optional) Add tags to identify this private endpoint resource.
  10. Click Create.
The private endpoint is created. The create process can take a couple of minutes. When the private endpoint is created successfully, the private endpoint is in ACTIVE status.

Viewing Private Endpoints

All private endpoints created in Data Catalog are listed in the Private Endpoints page. To view details for a specific private endpoint, click the private endpoint name. The private endpoint details page displays. Alternatively, click the Actions icon (three dots) for the private endpoint and select View Details.

Additionally, from the Private Endpoints list, you can edit, move, or delete a private endpoint.

A private endpoint can be in one of the following statuses:

Creating
The private endpoint is being created.
Active
The private endpoint is successfully created and ready for use.
Updating
The private endpoint details, such as name, DNS zones, or compartment, are being updated.
Moving
The private endpoint is being moved to a new compartment.
Deleting
The private endpoint is being deleted.
Deleted
The private endpoint is successfully deleted.
Failed
The private endpoint was not created, updated, or deleted successfully.

Attaching a Private Endpoint

You attach a private endpoint to a data catalog to enable the data catalog data assetsĀ  to be harvestedĀ  using the configured private network. You can attach a private endpoint to only one data catalog and you can attach only one private endpoint to a data catalog.

Before you attach a private endpoint to a data catalog, you must create a private endpoint.

Here's how you attach a private endpoint to a data catalog.

  1. From the Data Catalogs list in the Console, click the Actions icon (three dots) for the data catalog where you want to attach a private endpoint and then select Attach Private Endpoint.
  2. From the Attach Private Endpoint dialog, select the private endpoint that you want to attach to the data catalog. If the private endpoint was created in a different compartment than the data catalog, then you have to change the compartment before selecting the private endpoint.
  3. Click Attach.

A notification displays indicating that the private endpoint is being attached to the data catalog. A notification is also displayed after the private endpoint is attached successfully to the data catalog.

Using a Private Endpoint

When you create a data asset in the data catalog service console, you can specify that you want to use the private endpoint you have created and attached to the data catalog.

Caution

If you specify that you want to use a private endpoint without creating and attaching a private endpoint to the data catalog, you receive an error while creating the data asset.

Editing a Private Endpoint

Here's how you edit the private endpoint name and DNS zones:

  1. From the Data Catalog page in the Console, click Private Endpoints.
  2. From the Private Endpoints list, click the Actions icon (three dots) for the private endpoint you want to edit and select Edit. Alternatively, you can click the name of the private endpoint to open the private endpoint details page and then click Edit.
  3. Modify the private endpoint name and DNS zones.
  4. Click Save Changes.
A notification displays indicating that your changes are saved successfully.

Moving a Private Endpoint

You can move the private endpoint resource from the compartment you created it in to a different compartment.

Here's how you move a private endpoint to a different compartment:

  1. From the Data Catalog page in the Console, click Private Endpoints.
  2. From the Private Endpoints list, click the Actions icon (three dots) for the private endpoint you want to move and select Move Resource.
  3. Select the new compartment for the private endpoint resource.
  4. Click Move Resource.
A notification displays indicating that the private endpoint resource is moved to the new compartment successfully. You might notice the private endpoint status change to Moving. After the move is completed successfully, the private endpoint status changes back to Active.

Detaching a Private Endpoint

You must detach a private endpoint from the data catalog it is attached to before you can delete the private endpoint. Also, if you want to attach a different private endpoint to a data catalog, you must detach any private endpoint already attached to the data catalog.

Here's how you detach a private endpoint from a data catalog.

  1. From the Data Catalogs page in the Console, click the Actions icon (three dots) for the data catalog where you want to detach a private endpoint and then select Detach Private Endpoint.
  2. From the Detach Private Endpoint dialog, click Yes, Detach.

A notification displays indicating that the private endpoint is being detached from the data catalog. A notification is also displayed after the private endpoint is detached successfully from the data catalog.

Important

The data catalog data assets that use the private endpoint that you detached from the data catalog can no longer be harvested again.

Deleting a Private Endpoint

You can delete a private endpoint only if it is not attached to any data catalog.

Caution

If you attempt to delete a private endpoint that is still attached to a data catalog, you receive a warning that the private endpoint can't be deleted. You must first detach the private endpoint from that data catalog and then delete the private endpoint.

Here's how you delete a private endpoint:

  1. From the Data Catalog page in the Console, click Private Endpoints.
  2. From the Private Endpoints list, click the Actions icon (three dots) for the private endpoint you want to delete and select Delete.
  3. In the Delete Private Endpoint dialog, type DELETE to confirm that you want to delete the private endpoint and then click Delete.
A notification displays indicating that the private endpoint is deleted successfully.
Important

The data catalog data assets that use the private endpoint that you deleted can no longer be harvested again.