Export and Import of Objects

In Data Integration, the object types that you can export from a workspace and import to a workspace include projects and folders, tasks, applications, data flows, pipelines, and user-defined functions and functions libraries.

In an export operation, objects are exported to a zip file that's created and saved in an Oracle Cloud Infrastructure Object Storage bucket. Objects to be imported must be an export zip file that's saved in an Object Storage bucket.

Data Integration includes referenced objects in an export request. For example, when you export a data flow, by default Data Integration includes referenced objects such data assets. In an import request, you have the option to exclude existing data assets.

Instead of exporting and importing different object types separately, you can export and import a whole project and an entire workspace. Before you start exporting and importing, see Considerations When Using Export and Import for best practices and support information.

Data Integration needs specific policies to facilitate export and import, and access resources in Object Storage. See Required Setup and Policies.

After setting up the required policies, see the following pages for the type of object that you want to export or import:

• Exporting Objects
  • Exporting a Project
  • Exporting a Folder, Task, Data Flow, or Pipeline
  • Exporting a User-Defined Functions Library or User-Defined Function
  • Exporting an Application
  • Exporting a Workspace
• Importing Objects
  • Importing a Project
  • Importing a Folder, Task, Data Flow, or Pipeline into an Existing Project
  • Importing a User-Defined Functions Library or User-Defined Function
  • Importing an Application
  • Importing a Workspace

The following pages describe management tasks that can be performed with export and import requests:

• Monitoring Export and Import Requests
  • Viewing the Details of an Export Request
  • Viewing the Details of an Import Request
• Stopping an Export or Import Request

Required Setup and Policies

Review the following before you start exporting and importing objects:

• Ensure that you have an Object Storage bucket.

• To allow the group to use export and import in Data Integration, add either one of the following policy statements depending on your requirements:

  allow group <group-name> to manage dis-family in tenancy

  allow group <group-name> to manage dis-family in compartment <compartment-name>

  Alternatively, instead of providing full manage permissions, you can use the policy that restricts usage access to export and import only. For example:

  allow group <group-name> to {DIS_WORKSPACE_OBJECT_EXPORT,DIS_WORKSPACE_OBJECT_IMPORT} in tenancy

  allow group <group-name> to {DIS_WORKSPACE_OBJECT_EXPORT,DIS_WORKSPACE_OBJECT_IMPORT} in compartment <compartment-name>

• Create policies that allow Data Integration to access Object Storage resources such as objects and buckets.

  For the Object Storage bucket that is to be used for export and import, ensure that you have the required read, write, and list permissions to access the bucket or be part of the group that has the required access.

  The following are policy examples that you can use. Ensure that you write policies that meet your needs.

  If the Data Integration workspace and the Object Storage bucket are in the same tenancy, create these policies:

  allow group <group-name> to read objectstorage-namespaces in tenancy

  allow group <group-name> to <read or use> buckets in tenancy

  allow group <group-name> to manage objects in tenancy

  If the Data Integration workspace and the Object Storage bucket are in different tenancies, create the relevant policies in the workspace tenancy and the Object Storage tenancy:

  In the workspace tenancy:

  Define tenancy <tenancy-name> as <Object-Storage-tenancy-OCID>

  Endorse group <group-name> to inspect compartments in tenancy <tenancy-name>

  Endorse group <group-name> to use object-family in tenancy <tenancy-name>

  In the Object Storage tenancy:

  Define tenancy <tenancy-name> as <workspace-tenancy-OCID>

  Define group <workspace-tenancy-group-name> as <workspace-tenancy-group-OCID>

  Admit group <group-name> of tenancy <tenancy-name> to inspect compartments in tenancy

  Admit group <group-name> of tenancy <tenancy-name> to <use or manage> object-family in compartment <compartment-name>

• For a REST task that uses the workspace OCI resource principal for authentication to perform an export or import operation, the following policies are required on the workspace resource that calls the REST task:

  Allow any-user to {DIS_WORKSPACE_OBJECT_EXPORT,DIS_WORKSPACE_OBJECT_IMPORT} in compartment <compartment-name> where ALL {request.principal.type='disworkspace', request.principal.id='<workspace-ocid>'}
  

  Allow any-user to read objectstorage-namespaces in tenancy where ALL {request.principal.type = 'disworkspace', request.principal.id = '<workspace-ocid>'}
  

  Allow any-user to read buckets in compartment <compartment-name> where ALL {request.principal.type = 'disworkspace', request.principal.id = '<workspace-ocid>'}
  

  Allow any-user to manage objects in compartment <compartment-name> where ALL {request.principal.type = 'disworkspace', request.principal.id = '<workspace-ocid>'}