bulk-download

Description

Downloads all objects which match the given prefix to a given directory.

Usage

oci os object bulk-download [OPTIONS]

Required Parameters

-bn, --bucket-name [text]

The name of the bucket.

--download-dir [text]

The directory where retrieved objects will be placed as files. This directory will be created if it does not exist.

Optional Parameters

--delimiter [text]

When this parameter is set, only objects whose names do not contain the delimiter character (after an optionally specified prefix) are returned. Scanned objects whose names contain the delimiter have part of their name up to the last occurrence of the delimiter (after the optional prefix) returned as a set of prefixes. Note: Only '/' is a supported delimiter character at this time.

--exclude [text]

Only download objects which do not match the provided pattern. Patterns are taken relative to the DOWNLOAD directory. This option can be provided mulitple times to match on mulitple patterns. Supported pattern symbols are: 

*: Matches everything

?: Matches any single character

[sequence]: Matches any character in sequence

[!sequence]: Matches any character not in sequence

--from-json [text]

Provide input to this command as a JSON document from a file using the file://path-to/file syntax.

The --generate-full-command-json-input option can be used to generate a sample json file to be used with this command option. The key names are pre-populated and match the command option names (converted to camelCase format, e.g. compartment-id --> compartmentId), while the values of the keys need to be populated by the user before using the sample file as an input to this command. For any command option that accepts multiple values, the value of the key can be a JSON array.

Options can still be provided on the command line. If an option exists in both the JSON document and the command line then the command line specified value will be used.

For examples on usage of this option, please see our "using CLI with advanced JSON options" link: https://docs.cloud.oracle.com/iaas/Content/API/SDKDocs/cliusing.htm#AdvancedJSONOptions

--include [text]

Only download objects which match the provided pattern. Patterns are taken relative to the DOWNLOAD directory. This option can be provided mulitple times to match on mulitple patterns. Supported pattern symbols are: 

*: Matches everything

?: Matches any single character

[sequence]: Matches any character in sequence

[!sequence]: Matches any character not in sequence

--multipart-download-threshold [integer range]

Objects larger than this size (in MiB) will be downloaded in multiple parts. The minimum allowable threshold is 128 MiB.

-ns, --namespace, --namespace-name [text]

The top-level namespace used for the request. If not provided, this parameter will be obtained internally using a call to 'oci os ns get'

--no-overwrite

If a file with the same name as an object already exists in the download directory, do not overwite it. If neither this flag nor --overwrite is specified, you will be prompted each time a file would be overwritten

--overwrite

If a file with the same name as an object already exists in the download directory, overwrite it. If neither this flag nor --no-overwrite is specified, you will be prompted each time a file would be overwritten.

--parallel-operations-count [integer]

The number of parallel operations to perform. Decreasing this value will make bulk downloads less resource intensive but they may take longer. Increasing this value may improve bulk download times, but the upload process will consume more system resources and network bandwidth. [default: 10]

--part-size [integer range]

Part size (in MiB) to use when downloading an object in multiple parts. The minimum allowable size is 128 MiB.

--prefix [text]

Retrieve all objects with the given prefix. Omit this parameter to get all objects in the bucket

Examples

Download all objects in the bucket

 oci os object bulk-download -ns mynamespace -bn mybucket --download-dir path/to/download/directory

Download all objects that match a given prefix

 oci os object bulk-download -ns mynamespace -bn mybucket --download-dir path/to/download/directory --prefix myprefix

You can download all objects that match a given prefix by specifying the --prefix flag. In the above example, "--prefix myprefix" would match object names such as myPrefix_textfile1.txt, myPrefix_myImage.png etc.

If you have named your objects so that they exist in Object Storage as a hierarchy, e.g. level1/level2/level3/myobject.txt, then you can download objects at a given level (and all sub levels) by specifying a prefix:

 oci os object bulk-download -ns mynamespace -bn mybucket --download-dir path/to/download/directory --prefix level1/level2/

This will download all objects of the form level1/level2/<object name>, level1/level2/leve3/<object name>, level1/level2/leve3/level4/<object name> etc.

Limiting downloaded objects using a prefix and delimiter

 oci os object bulk-download -ns mynamespace -bn mybucket --download-dir path/to/download/directory --prefix level1/level2/ --delimiter /

If you have named your objects so that they exist in Object Storage as a hierarchy, e.g. level1/level2/level3/myobject.txt, and you only want to download objects at a given level of the hierarchy, e.g. example everything of the form level1/level2/<object name> but not level1/level2/leve3/<object name> or any other sub-levels, you can specify a prefix and delimiter. Currently the only supported delimiter is /

Overwriting or skipping files

 oci os object bulk-download -ns mynamespace -bn mybucket --download-dir path/to/download/directory --overwrite
 oci os object bulk-download -ns mynamespace -bn mybucket --download-dir path/to/download/directory --no-overwrite

If files with the same name as the objects being downloaded already exist in the download directory, you can opt to overwrite them with the --overwrite option, or preserve them with the --no-overwrite option.