Oracle Cloud Infrastructure Documentation

Interacting With Object Storage

This topic helps you understand the Oracle Cloud Infrastructure Object Storage environment and how it interacts with a Storage Gateway.

Creating the Required IAM Users, Groups, and Policies

An Oracle Cloud Infrastructure administrator must perform prerequisite tasks in preparation for data movement between Storage Gateway and Object Storage. If you are new to Oracle Cloud Infrastructure, we recommend that you read Setting Up Your Tenancy.

To use Oracle Cloud Infrastructure, you must be given the required type of access in a An IAM document that specifies who has what type of access to your resources. It is used in different ways: to mean an individual statement written in the policy language; to mean a collection of statements in a single, named "policy" document (which has an Oracle Cloud ID (OCID) assigned to it); and to mean the overall body of policies your organization uses to control access to resources. written by an administrator, whether you're using the Console or the REST API with an SDK, CLI, or other tool. If you try to perform an action and get a message that you don’t have permission or are unauthorized, confirm with your administrator the type of access you've been granted and which A collection of related resources that can be accessed only by certain groups that have been given permission by an administrator in your organization. you should work in.

Access to resources is provided to groups using policies and then inherited by the users that are assigned to those groups. For details on creating groups, see Managing Groups.

For Storage Gateway, an administrator creates these groups with the following policies:

Allow group <group_name> to manage buckets in compartment <compartment_name>
Allow group <group_name> to manage objects in compartment <compartment_name> 

Content Consistency Between Storage Gateway and Object Storage

Changes to the files in Storage Gateway, including create, write, update, and delete, eventually are consistent with Object Storage. Uploads are asynchronous and buffered for performance, so Storage Gateway file changes might not yet be reflected in Object Storage.

You can access, modify, and upload objects directly to a bucket using Object Storage native APIs, SDKs, the CLI, the Console, or the HDFS connector. Objects modified in these ways do not appear as files in Storage Gateway until you click Refresh in the Storage Gateway management console.

Name Restrictions

Storage Gateway file and file system names must adhere to Object Storage bucket and object name restrictions and guidelines.

Use the following guidelines for naming file systems:

  • Use from 1 to 256 UTF-8 characters.
  • Valid characters are letters (upper or lower case), numbers, hyphens, underscores, and periods.

    Important

    Names cannot contain a slash (/) character because this character delimits Object Storage bucket and object names.

  • Do not include confidential information.
  • Make the name unique within a Storage Gateway instance.

Use the following guidelines for naming files:

  • Use from 1 to 1024 characters.
  • Valid characters are letters (upper or lower case), numbers, and characters other than linefeed, newline, and NULL.
  • Use only Unicode characters for which the UTF-8 encoding does not exceed 1024 bytes. Clients are responsible for URL-encoding characters.
  • Do not include confidential information.
  • Make the name unique within the bucket. Do not use the name of an existing object within the bucket when naming an object unless you intend to overwrite the existing object with the contents of the new or renamed object.

Custom Metadata

POSIX file and directory attributes are stored in custom metadata. These attributes include uid, gid, mode, atime, ctime, and mtime. If existing objects in Object Storage are missing the required custom metadata, Storage Gateway assigns the following default values:

  • uid=0
  • gid=0
  • mode=0644 for file and 0755 for directory

The custom metadata is not updated in Object Storage until a file operation triggers Storage Gateway to update the file in Object Storage.Timestamp metadata (atime, ctime, and mtime) are expressed in milliseconds. Access modes are expressed in octal and include file/directory bit.

The custom metadata names follow these guidelines:

  • Only ASCII characters.
  • A maximum of 128 bytes.

The custom metadata values follow these guidelines:

  • Only UTF-8 characters.
  • A maximum of 256 bytes.

Understanding Directory and File Hierarchy Translations in Object Storage

Within an Object Storage namespace, buckets and objects exist in a flat hierarchy. Storage Gateway flattens the file system directory hierarchy into nested object prefixes in Object Storage.

For directories:

  • A Storage Gateway file system called myFS that contains a directory called myDir, appears in Object Storage as:

    n/<os_namespace>/b/myFS/o/myDir/
  • A Storage Gateway file system called myFS that contains a myDir subdirectory called mySubDir, appears in Object Storage as:

    n/<os_namespace>/b/myFS/o/myDir/mySubDir/

You can distinguish a Storage Gateway directory from a Storage Gateway file in the following ways:

  • Directories have a trailing slash /.
  • Directory size or length is 0 (zero).

For files:

  • A Storage Gateway file system called myFS that contains a directory called myDir with a file called file1, appears in Object Storage as:

    n/<os_namespace>/b/myFS/o/myDir/file1
  • A Storage Gateway file system called myFS that contains a myDir subdirectory called mySubDir with a file called file2, appears in Object Storage as:

    n/<os_namespace>/b/myFS/o/myDir/mySubDir/file2

You can distinguish a Storage Gateway file from a Storage Gateway directory in the following ways:

  • Directories have a trailing / and files do not.
  • File length can be 0 (zero) or non-zero, but directory length is always 0 (zero).

Internal Storage Gateway Objects

Storage Gateway creates some special internal objects in Object Storage. These objects have a /gateway directory prefix. For example:

/n/<object_storage_namespace>/b/<bucket>/o//gateway

Important

Do not modify or remove the objects in the special /gateway directory. These objects are critical for Storage Gateway operation.