Oracle Cloud Infrastructure Documentation

Appliance Data Transfer Reference

This topic provides complete task details for certain components associated with Appliance-Based Data Transfers. Use this topic as a reference to learn and use commands associated with components included in the Appliance-Based Data Transfer procedure.

Transfer Jobs

A transfer job represents the collection of files that you want to transfer and signals the intention to upload those files to Oracle Cloud Infrastructure. A transfer job combines at least one transfer disk with a transfer package. Identify which compartment and Object Storage bucket that Oracle is to upload your data to.

Note

It is recommended that you create a compartment for each transfer job to minimize the required access your tenancy.

Tip

You can use the Console or the Oracle Cloud Infrastructure CLI to create a transfer job.

 

Creating Transfer Jobs

Create the transfer job in the same compartment as the upload bucket and supply a human-readable name for the transfer job. Avoid entering confidential information when providing transfer job names.

Creating a transfer job returns a job ID that you specify in other transfer tasks. For example:

ocid1.datatransferjob.region1.phx..<unique_ID>
To create a transfer job using the Console
To create a transfer job using the CLI

Displaying Transfer Jobs

To display the list of transfer jobs using the Console
To display the list of transfer jobs using the CLI

Displaying Transfer Job Details

To display the details of a transfer job using the Console
To display the details of a transfer job using the CLI

Editing Transfer Jobs

To edit the name of a transfer job using the Console
To edit the name of a transfer job using the CLI
To edit the tags associated with a transfer job using the CLI
To delete the tags associated with a transfer job using the CLI

Deleting Transfer Jobs

You can delete transfer jobs when they are in the Initiated, Preparing, and Close states.

To delete a transfer job using the Console
To delete a transfer job using the CLI

Closing Transfer Jobs

Tip

You can use the Console or the Oracle Cloud Infrastructure CLI to close a transfer job.

Typically, you would close a transfer job when no further transfer job activity is required or possible. Closing a transfer job requires that the status of all associated transfer packages be returned, canceled, or deleted. In addition, the status of all associated transfer disks must be complete, in error, missing, canceled, or deleted.

To close a transfer job using the Console
To close a transfer job using the CLI

Appliances

This section describes tasks associated with the Oracle-provided appliance.

Requesting Appliances

Tip

You can use the Console or the Oracle Cloud Infrastructure CLIs to request an appliance.

Tip

We recommend that you identify the data you intend to upload and make data copy preparations before requesting the appliance.

Creating an appliance request returns an Oracle-assigned appliance label. For example:

XA8XM27EVH
To request an appliance using the Console
To request an appliance using the CLIs

When you submit an appliance request, Oracle generates a unique label (name) to identify the appliance and your request is sent to Oracle for approval and processing.

Monitoring the Appliance Request Status

The time it takes to approve, prepare, and ship your appliance request varies and depends on various factors, including current available inventory. Oracle provides status updates daily throughout the appliance request and ship process.

To monitor the status of your appliance request using the Console
To monitor the status of your appliance request using the CLI

Displaying the List of Appliances

Tip

You can use the Console to get a list of appliances that are associated with a job.

 

To display the list of appliances using the Console
To display the list of appliances using the CLI commands

Displaying the Details of an Appliance

To display the details of an appliance using the Console
To display the details of an appliance using the CLI commands

Editing the Appliance Request Shipping Information

You can only edit the shipping information when the status is Requested.

To edit the appliance request shipping information using the Console
To edit the appliance request shipping information using the CLI

Deleting an Appliance Request

You can delete an appliance request before Oracle approves the request—the status must be Requested. For example, you initiated the transfer by creating a transfer job and requested an appliance, but changed your mind.

To delete an appliance request using the Console
To delete an appliance request using the CLI commands

Datasets

A dataset is a collection of files that are treated similarly. You can write up to 100 million files onto the appliance for transfer. We currently support one dataset per appliance.

Creating Datasets

Appliance data transfer supports NFSversion 3, 4, and 4.1 to write data to the appliance. In preparation for writing data, create and configure a dataset to write to.

To create a dataset using the CLI

Activating the Dataset

Activation creates the NFS export, making the dataset accessible to NFS clients.

To activate the dataset

Configuring Export Settings on the Dataset

To configure export settings on a dataset

Deactivating the Dataset

Note

Deactivating the dataset is only required if you are running appliance commands using the Data Transfer Utility. If you are using the Oracle Cloud Infrastructure CLI to run your Appliance-Based Data Transfer, you can skip this step and proceed to Sealing the Dataset.

After you are done writing data, deactivate the dataset. Deactivation removes the NFS export on the dataset, disallowing any further writes.

To deactivate the dataset

Sealing the Dataset

Sealing a dataset stops all writes to the dataset. Sealing a dataset is a long running process that can take some time to complete. The completion time depends upon the number of files and total amount of data that was copied to the appliance.

If you issue the seal command without the --wait option, the seal operation is triggered and runs in the background. You are returned to the command prompt and can use the seal-status command to monitor the sealing status. If you issue the seal command with the --wait option, the seal operation is triggered and continues to provide status updates until sealing completion.

Important

You can only copy regular files to transfer appliances. Special files (for example, symbolic links, device special, sockets, and pipes) cannot be copied directly. To transfer special files, create a tar archive of these files and copy the tar archive to the transfer appliance.

The sealing operation generates a manifest across all files in the dataset. The manifest contains an index of the copied files and generated data integrity hashes.

To seal the dataset using the CLI
To monitor the dataset sealing process using the CLI

Downloading the Dataset Seal Manifest

After sealing the dataset, you can optionally download the dataset's seal manifest to a user-specified location. The manifest file contains the checksum details of all the files. The transfer site uploader consults the manifest file to determine the list of files to upload to object storage. For every uploaded file, it validates that the checksum reported by object storage matches the checksum in manifest. This validation ensures that no files got corrupted in transit.

To download the dataset seal manifest file using the CLI

Reopening a Dataset

Tip

You can only use the CLI command to reopen a dataset.

If changes are necessary after sealing a dataset or finalizing an appliance, you must reopen the dataset to modify the contents. Make the required changes and again seal the dataset. Resealing the dataset generates a new manifest.

Note

If an appliance is rebooted or power cycled, follow the instructions in this topic to reopen the dataset.

Step 1: Unlocking the Appliance

Before you can write data to the transfer appliance, you must unlock the appliance. Unlocking the transfer appliance requires the strong passphrase that is created by Oracle Cloud Infrastructure for each appliance. Unlocking can be accomplished in two different ways:

  • If you provide the --job-id and --appliance-label when running the unlock command, the data transfer system retrieves the passphrase from Oracle Cloud Infrastructure and sends it to the transfer appliance during the unlock operation.
  • You can query Oracle Cloud Infrastructure for the passphrase and provide that passphrase when prompted during the unlock operation.
To retrieve the passphrase to unlock the appliance
To query Oracle Cloud Infrastructure for the passphrase to provide to unlock the appliance

Step 2: Reopening the Appliance

Reopen the dataset to write data to the appliance again.

To reopen an NFS dataset

Step 3: Repeat Steps to Write Data to the Appliance

Repeat the same tasks you performed when you originally wrote data to the appliance beginning with activating the dataset in the Copying Files to the NFS Share section.