Oracle Cloud Infrastructure Documentation

Preparing for Appliance Data Transfers

Prepare phase indicator for appliance transfer

This topic describes the tasks associated with preparing for the Appliance-Based Data Transfer. The Project Sponsor role typically performs these tasks. See Roles and Responsibilities.

Note

You can only run Oracle Cloud Infrastructure CLI commands from a Linux host. This differs from running CLI commands for other Oracle Cloud Infrastructure Services on a variety of host operating systems. Appliance-based commands require validation that is only available on Linux hosts.

Creating the Required IAM Users, Groups, and Policies

Each service in Oracle Cloud Infrastructure integrates with IAM for authentication and authorization.

To use Oracle Cloud Infrastructure, you must be given the required type of access in a An IAM document that specifies who has what type of access to your resources. It is used in different ways: to mean an individual statement written in the policy language; to mean a collection of statements in a single, named "policy" document (which has an Oracle Cloud ID (OCID) assigned to it); and to mean the overall body of policies your organization uses to control access to resources. written by an administrator, whether you're using the Console or the REST API with an SDK, CLI, or other tool. If you try to perform an action and get a message that you don’t have permission or are unauthorized, confirm with your administrator the type of access you've been granted and which A collection of related resources that can be accessed only by certain groups that have been given permission by an administrator in your organization. you should work in.

Access to resources is provided to groups using policies and then inherited by the users that are assigned to those groups. Data transfer requires the creation of two distinct groups:

  • Data transfer administrators who can create and manage transfer jobs.
  • Data transfer upload users who can upload data to Object Storage. For your data security, the permissions for upload users allow Oracle personnel to upload standard and multi-part objects on your behalf and inspect bucket and object metadata. The permissions do not allow Oracle personnel to inspect the actual data.

The Data Administrator is responsible for generating the required RSA keys needed for the temporary upload users. These keys should never be shared between users.

For details on creating groups, see Managing Groups.

An administrator creates these groups with the following policies:

  • The data transfer administrator group requires an authorization policy that includes the following:

    Allow group <group_name> to manage data-transfer-jobs in compartment <compartment_name>
    Allow group <group_name> to manage buckets in compartment <compartment_name>
    Allow group <group_name> to manage objects in compartment <compartment_name>

    Alternatively, you can consolidate the manage buckets and manage objects policies into the following:

    Allow group <group_name> to manage object-family in compartment <compartment_name>
  • The data transfer upload user group requires an authorization policy that includes the following:

    Allow group <group_name> to manage buckets in compartment <compartment_name> where all { request.permission='BUCKET_READ' }
    Allow group <group_name> to manage objects in compartment <compartment_name> where any { request.permission='OBJECT_CREATE' , request.permission='OBJECT_OVERWRITE' , request.permission='OBJECT_INSPECT' }

Important

For security reasons, we recommend that you create a unique IAM data transfer upload user for each transfer job and then delete that user once your data is uploaded to Oracle Cloud Infrastructure.

The Oracle Cloud Infrastructure administrator then adds a user to each of the data transfer groups created. For details on creating users, see Managing Users.

Requesting the Data Transfer Appliance Entitlement

If your tenancy has not been entitled to perform Appliance-Based Data Transfers, you are required to request it before creating an appliance-based transfer job. The Data Transfer Appliance Entitlement is a tenancy-wide entitlement that you need to request once for each tenancy.

Use the following policy to enable users in a specific group to request a Data Transfer Appliance Entitlement in your tenancy.

Allow group <group_name> to {DTA_ENTITLEMENT_CREATE} in tenancy

Creating Object Storage Buckets

The Object Storage service is used to upload your data to Oracle Cloud Infrastructure. Object Storage stores objects in a container called a bucket within a A collection of related resources that can be accessed only by certain groups that have been given permission by an administrator in your organization. in your tenancy. For details on creating the bucket to store uploaded data, see Managing Buckets.

Creating Transfer Jobs

This section describes how to create a transfer job as part of the preparation for the data transfer. See Transfer Jobs for complete details on all tasks related to transfer jobs.

Tip

You can use the Console or the Oracle Cloud Infrastructure CLI to create a transfer job.

A transfer job represents the collection of files that you want to transfer and signals the intention to upload those files to Oracle Cloud Infrastructure. Identify which compartment and Object Storage bucket that Oracle is to upload your data to. Create the transfer job in the same compartment as the upload bucket and supply a human-readable name for the transfer job. Avoid entering confidential information when providing transfer job names.

Note

It is recommended that you create a compartment for each transfer job to minimize the required access your tenancy.

Creating a transfer job returns a job ID that you specify in other transfer tasks. For example:

ocid1.datatransferjob.region1.phx..<unique_ID>
To create a transfer job using the Console
To create a transfer job using the CLI

Requesting the Data Transfer Appliance Entitlement

Tip

You can use the Console or the Oracle Cloud Infrastructure CLI to request the Data Transfer Appliance Entitlement.

If your tenancy is not entitled to use the Data Transfer Appliance, you must request the Data Transfer Appliance Entitlement before creating an appliance-based transfer job.

To request the Data Transfer Appliance Entitlement using the Console
To request the Data Transfer Appliance Entitlement using the CLI

Preparing Upload Configuration Files

The Project Sponsor is responsible for creating or obtaining configuration files that allow the uploading of user data to the transfer appliance. Send these configuration files to the Data Administrator where they can be placed in the Control Host (if there are separate Control and Data Hosts).The config file is for the data transfer administrator, the IAM user with the authorization and permissions to create and manage transfer jobs. The config_upload_user file is for the data transfer upload user, the temporary IAM user that Oracle uses to upload your data on your behalf.

Create a base Oracle Cloud Infrastructure directory and two configuration files with the required credentials.

Creating the Data Transfer Directory

Create a Oracle Cloud Infrastructure directory (.oci) on the same Control Host machine where the Oracle Cloud Infrastructure CLI is installed. For example:

mkdir /root/.oci/

The two configuration files (config and config_upload_user) are placed in what ever location you choose.

Note

You can store the configuration files anywhere on your Control Host. The root directory is only given as an example.

Creating the Data Transfer Administrator Configuration File

Create the data transfer administrator configuration file /root/.oci/config with the following structure:

[DEFAULT]
user=<The OCID for the data transfer administrator>
fingerprint=<The fingerprint of the above user's public key>
key_file=<The _absolute_ path to the above user's private key file on the host machine>
tenancy=<The OCID for the tenancy that owns the data transfer job and bucket>
region=<The region where the transfer job and bucket should exist. Valid values are: us-ashburn-1, us-phoenix-1, eu-frankfurt-1, and uk-london-1.>

For example:

[DEFAULT]
user=ocid1.user.oc1..<unique_ID>
fingerprint=4c:1a:6f:a1:5b:9e:58:45:f7:53:43:1f:51:0f:d8:45
key_file=/home/user/ocid1.user.oc1..exampleuniqueID.pem
tenancy=ocid1.tenancy.oc1..<unique_ID>
region=us-phoenix-1

For the data transfer administrator, you can create a single configuration file that contains different profile sections with the credentials for multiple users. Then use the ‑‑profile option to specify which profile to use in the command. Here is an example of a data transfer administrator configuration file with different profile sections:

[DEFAULT]
user=ocid1.user.oc1..exampleuniqueID
fingerprint=4c:1a:6f:a1:5b:9e:58:45:f7:53:43:1f:51:0f:d8:45
key_file=/home/user/ocid1.user.oc1..exampleuniqueID.pem
tenancy=ocid1.tenancy.oc1..exampleuniqueID
region=us-phoenix-1
[PROFILE1]
user=ocid1.user.oc1..exampleuniqueID
fingerprint=4c:1a:6f:a1:5b:9e:58:45:f7:53:43:1f:51:0f:d8:45
key_file=/home/user/ocid1.user.oc1..exampleuniqueID.pem
tenancy=ocid1.tenancy.oc1..exampleuniqueID
region=us-ashburn-1

Important

Creating an upload user configuration file with multiple profiles is not supported.

By default, the DEFAULT profile is used for all CLI commands. For example:

oci dts job create --compartment-id ocid.compartment.oc1..exampleuniqueID --bucket MyBucket --display-name MyDisplay --device-type appliance

Instead, you can issue any CLI command with the --profile option to specify a different data transfer administrator profile. For example:

oci dts job create --compartment-id ocid.compartment.oc1..exampleuniqueID --bucket MyBucket --display-name MyDisplay --device-type appliance --profile MyProfile

Using the example configuration file above, the <profile_name> would be profile1.

If you created two separate configuration files, use the following command to specify the configuration file to use:

oci dts job create --compartment-id <compartment_id> --bucket <bucket_name> --display-name <display_name> 

Creating the Data Transfer Upload User Configuration File

The config_upload_user configuration file is for the data transfer upload user, the temporary IAM user that Oracle uses to upload your data on your behalf. Create this configuration file with the following structure:

[DEFAULT]
user=<The OCID for the data transfer upload user>
fingerprint=<The fingerprint of the above user's public key>
key_file=<The _absolute_ path to the above user's private key file on the host machine>
tenancy=<The OCID for the tenancy that owns the data transfer job and bucket>
region=<The region where the transfer job and bucket should exist. Valid values are: us-ashburn-1, us-phoenix-1, eu-frankfurt-1, and uk-london-1.>

For example:

[DEFAULT]
user=ocid1.user.oc1..exampleuniqueID
fingerprint=4c:1a:6f:a1:5b:9e:58:45:f7:53:43:1f:51:0f:d8:45
key_file=/home/user/ocid1.user.oc1..exampleuniqueID.pem
tenancy=ocid1.tenancy.oc1..exampleuniqueID
region=us-phoenix-1

Configuration File Entries

The following table lists the basic entries that are required for each configuration file and where to get the information for each entry.

Note

Data Transfer Service does not support passphrases on the key files for both data transfer administrator and data transfer upload user.

Entry Description and Where to Get the Value Required?
user

OCID of the data transfer administrator or the data transfer upload user, depending on which profile you are creating. To get the value, see Required Keys and OCIDs.

Yes
fingerprint

Fingerprint for the key pair being used. To get the value, see Required Keys and OCIDs.

Yes
key_file

Full path and filename of the private key.

Important: The key pair must be in PEM format. For instructions on generating a key pair in PEM format, see Required Keys and OCIDs.

Yes
tenancy

OCID of your tenancy. To get the value, see Required Keys and OCIDs.

Yes
region

An Oracle Cloud Infrastructure region. See Regions and Availability Domains.

Data transfer is supported in US East (Ashburn), US West (Phoenix), Germany Central (Frankfurt), and UK South (London).

Yes

You can verify the data transfer upload user credentials using the following command:

oci dts job verify-upload-user-credentials --bucket <bucket_name>

Requesting the Transfer Appliance

This section describes how to request a transfer appliance from Oracle for copying your data to Oracle Cloud Infrastructure See Appliances for complete details on all tasks related to transfer jobs.

Tip

You can use the Console or the Oracle Cloud Infrastructure CLI to request a transfer appliance.

Oracle Cloud Infrastructure customers can use data transfer appliances to migrate data for free. You are only charged for Object Storage usage once the data is successfully transferred to your designated bucket. All appliance requests still require approval from Oracle.

Tip

We recommend that you identify the data you intend to upload and make data copy preparations before requesting the transfer appliance.

Creating a transfer appliance request returns an Oracle-assigned appliance label. For example:

XA8XM27EVH
To request a transfer appliance using the Console
To request a transfer appliance using the CLI

When you submit an appliance request, Oracle generates a unique label (name) to identify the transfer appliance and your request is sent to Oracle for approval and processing.

Notifying the Data Administrator

When you have completed all the tasks in this topic, provide the Data Administrator of the following:

  • IAM login credentials
  • Oracle Cloud Infrastructure CLI configuration files
  • Transfer job ID
  • Transfer job label

What's Next

You are now ready to configure your system for the data transfer. See Configuring Appliance Data Transfers.