Oracle Cloud Infrastructure Documentation

Copying the Data

Copy phase indicator for disk transfer

This topic describes the tasks associated with running the data transfer from the Data Host to the physical disk that has been configured as a transfer disk. The Data Administrator role typically performs these tasks. See Roles and Responsibilities.

Information Prerequisites

Before performing any disk copying tasks, you must obtain the following information:

  • Disk block device path. The Infrastructure Engineer typically provides this information.
  • IAM login information, Data Transfer Utilityconfiguration files, transfer job ID, and job label. The Project Sponsor typically provides this information.

Generate and Upload RSA Key Pairs

The Data Administrator is responsible for generating and uploading the RSA key pairs. Do not share these generated RSA keys between users. See Creating a Key Pair.

Install and Configure the Data Transfer Utility

The Data Transfer Utility provides a set of command line-based tools for configuring and running disk-based data transfers. Use the Data Transfer Utility as an alternative to running commands from the Console. Sometimes you must use the Data Transfer Utility to complete certain tasks as there is no Console equivalent. See Data Transfer Utility for details on how to install and configure the Data Transfer Utility for use with disk-based data transfers.

Creating the Transfer Disk

The transfer disk is the logical representation of the physical disk that has been configured for use for receiving data as part of the disk-based data transfer. See Transfer Disks for complete details on all tasks related to transfer disks.

Tip

You can only use the Data Transfer Utility to create a transfer disk.

When you create a transfer disk for use with the disk on which you are copying your files, the Data Transfer Utility:

  • Sets up the disk for encryption using the passphrase
  • Creates a file system on the disk
  • Mounts the file system at /mnt/orcdts_<label>

For example:

/mnt/orcdts_DJZNWK3ET

When you register a transfer disk, Oracle Cloud Infrastructure generates a strong encryption passphrase that is used to encrypt the contents on the disk. The encryption passphrase is displayed to standard output to the data transfer administrator user and cannot be retrieved again. Create a local, secure copy of the encryption passphrase, so you can reference the passphrase again.

Creating a transfer disk requires the job ID returned from when you created the transfer job and the path to the attached disk (for example, /dev/sdb).

To create a transfer disk using the Data Transfer Utility

Copying Files

Attach the disks to the Data Host and copy files to the mount point created by the transfer disk through the Data Transfer Utility.

You can only copy regular files to disks. Special files (links, sockets, pipes, and so forth) cannot be copied directly. To transfer special files, create a tar archive of the files and copy the tar archive to the disk.

Note

Copy all Files Before Disconnecting the Disk

Do not disconnect the disk until you copy all files from the Data Host and generate the manifest file. If you accidentally disconnect the disk before copying all files, you must unlock the disk using the encryption passphrase. The encryption passphrase was generated and displayed when you created the transfer disk. If the generated encryption passphrase is not available, you must delete the transfer disk from the transfer job and re-create the transfer disk. All data previously copied to that disk is lost.

Generating the Manifest

Tip

You can only use the Data Transfer Utility to generate a manifest file.
The amount of time to generate the manifest file depends on the size of the upload files, disk speed, and available processing power.

After copying your data to a transfer disk, generate a manifest file using the Data Transfer Utility. The manifest contains an index of all of the copied files and generated data integrity hashes. The Data Transfer Utility copies the config_upload_user configuration file and referenced IAM credentials to the encrypted transfer disk. This configuration file describes the temporary IAM data transfer upload user. Oracle uses the credentials and entries defined in the config_upload_user file when processing the transfer disk and uploading files to Oracle Cloud Infrastructure Object Storage.

Note

Data Transfer Service Does Not Support Passphrases on Private Keys

While we recommend encrypting a private key with a passphrase when generating API signing keys, Data Transfer does not support passphrases on the key file required for the config_upload_user. If you use a passphrase, Oracle personnel cannot upload your data.

Oracle cannot upload data from a transfer disk without the correct credentials defined in this configuration file. See Data Transfer Utility for more information about the required configuration files.

To create a manifest file using the Data Transfer Utility
Note

Do You Need to Regenerate the Manifest File?

If you add, remove, or modify any files on the disk after generating the manifest file, you must regenerate the file. If the manifest file does not match the contents of the target bucket, Oracle cannot upload the data.

Locking the Transfer Disk

Tip

You can only use the Data Transfer Utility to lock a transfer disk.

Locking a transfer disk safely unmounts the disk and removes the encryption passphrase from the Data Host.

To lock a transfer disk using the Data Transfer Utility

If you need to unlock the transfer disk , you are prompted for the encryption passphrase that was generated when you created the transfer disk.

To unlock a transfer disk using the Data Transfer Utility

Creating the Transfer Package

A transfer package is the virtual representation of the physical package of disks that you are shipping to Oracle for upload to Oracle Cloud Infrastructure. See Transfer Packages for complete details on all tasks related to transfer packages.

Tip

You can use the Console or the Data Transfer Utility to create a transfer package.

Creating a transfer package requires the job ID returned from when you created the transfer job. For example:

ocid1.datatransferjob.region1.phx..exampleuniqueID
To create a transfer package using the Console
To create a transfer package using the Data Transfer Utility

Attaching the Transfer Disk to the Transfer Package

Attach a transfer disk to a transfer package after you have performed the following tasks:

  1. Copied your data onto the disk
  2. Generated the required manifest file
  3. Run and reviewed the dry-run report
  4. Locked the transfer disk in preparation for shipment

Tip

You can use the Console or the Data Transfer Utility to attach a transfer disk to a transfer package.

A disk can be attached to one package, detached, and then attached to another package.

To attach a transfer disk to a transfer package using the Console
To attach a transfer disk to a transfer package using the Data Transfer Utility

You have attached a transfer disk to a transfer package, but have changed your mind about shipping that disk with the transfer package. You can also detach a transfer disk from one transfer package and attach that disk to a different transfer package.

To detach a transfer disk to a transfer package using the Console
To detach a transfer disk to a transfer package using the Data Transfer Utility

Setting Tracking Details on the Transfer Package

Tip

You can use the Console or the Data Transfer Utility to update the transfer package with tracking information.

After delivering the transfer package to the shipping vendor, update the transfer package with the tracking information.

Important

Oracle cannot process a transfer package until you update the tracking information.

To update the transfer package with tracking information using the Console
To update the transfer package with tracking information using the Data Transfer Utility

Notifying the Infrastructure Engineer

After completing the tasks listed in this topic, notify the Infrastructure Engineer of the following:

  • Disconnect the physical disk from the Data Host
  • Package the disk for shipment

What's Next

You are now ready to ship your disk with the copied data to Oracle. See Shipping the Disk.