Amazon S3 Compatibility API

Using the Amazon S3 Compatibility API, customers can continue to use their existing Amazon S3 tools (for example, SDK clients) and partners can make minimal changes to their applications to work with Object Storage. The Amazon S3 Compatibility API and Object Storage datasets are congruent. If data is written to the Object Storage using the Amazon S3 Compatibility API, the data can be read back using the native Object Storage API and conversely.

Differences between the Object Storage API and the Amazon S3 Compatibility API

The Object Storage Service provided by Oracle Cloud Infrastructure and Amazon S3 use similar concepts and terminology. In both cases, data is stored as objects in buckets. The differences are in the implementation of features and tools for working with objects.

The following highlights the differences between the two storage technologies:

  • Compartments

    Amazon S3 doesn't use compartments. By default, buckets created using the Amazon S3 Compatibility API or the Swift API are created in the root compartment of the Oracle Cloud Infrastructure tenancy. Instead, you can designate a different compartment for the Amazon S3 Compatibility API or Swift API to create buckets in.

  • Global bucket namespace

    Object Storage doesn't use a global bucket namespace. Each tenant is associated with one default namespace that spans all compartments within a region. The namespace serves as a container for all of your buckets and objects. You control bucket names within your namespace, however, bucket names must be unique within each region. You can have a bucket named MyBucket in US West (Phoenix) and a bucket named MyBucket in Germany Central (Frankfurt).

  • Encryption

    The Oracle Cloud Infrastructure Object Storage service encrypts all data at rest by default. Encryption can't be turned on or off using the API.

  • Object Level Access Control Lists (ACLs)

    Oracle Cloud Infrastructure does not use ACLs for objects. Instead, IAM policies are used to manage access to compartments, buckets, and objects.

For more information, see Overview of the Object Storage service.

Amazon S3 Compatibility API Prerequisites

To enable application access from Amazon S3 to Object Storage, you need to set up access to Oracle Cloud Infrastructure and modify your application.

Setting up access to Oracle Cloud Infrastructure:

Modifying your application:

  • Configure a new endpoint for the application that includes the namespace and the region identifier. For example: mynamespace.compat.objectstorage.us-phoenix-1.oraclecloud.com.

  • Set the target region as one of the Oracle Cloud Infrastructure regions.

    Important

    If your application does not support setting the region identifier to the correct Oracle Cloud Infrastructure identifier, you must either set the region to us-east-1 or leave it blank. Using this configuration, you can only use the Amazon S3 Compatibility API in your Oracle Cloud Infrastructure home region.
    If you can manually set the region, you can use the application against any Oracle Cloud Infrastructure region.

  • Configure the application to use the Customer Secret key.
  • The application must use path -based access. Virtual host-style access (accessing a bucket as bucketname.namespace.compat.objectstorage.region.oraclecloud.com) is not supported.

You can now use the Amazon S3 Compatibility API to access Object Storage in Oracle Cloud Infrastructure.

Amazon S3 Compatibility API Support

Amazon S3 Compatibility API support is provided at the bucket level and object level.

Bucket APIs

The following bucket APIs are supported:

Object APIs

The following object APIs are supported:

Multipart Upload APIs

The following multipart upload APIs are supported:

Tagging APIs

The following tagging APIs are supported:

SSE-C Support

Using optional API headers, you can provide your own 256-bit AES encryption key that is used to encrypt and decrypt objects uploaded to and downloaded from Object Storage.

If you want to use your own keys for server-side encryption, specify the following three request headers with the encryption key information:

Headers Description APIs Supported
x-amz-server-side-encryption-customer-algorithm Specifies "AES256" as the encryption algorithm.

GetObject

HeadObject

PutObject

InitiateMultipartUpload

UploadPart

x-amz-server-side-encryption-customer-key Specifies the base64-encoded 256-bit encryption key to use to encrypt or decrypt the data.
x-amz-server-side-encryption-customer-key-md5 Specifies the base64-encoded 128-bit MD5 digest of the encryption key. This value is used to check the integrity of the encryption key.

Object Storage has distinct APIs for copying objects and copying parts. Amazon S3 uses the presence of the following headers in PutObject and UploadPart to determine copy operations. To copy a source object that is encrypted with an SSE-C key, you must specify these three headers so that Object Storage can decrypt the object.

Headers Description APIs Supported
x-amz-copy-source-server-side-encryption-customer-algorithm Specifies "AES256" as the encryption algorithm to use to decrypt the source object.

PutObject

UploadPart


x-amz-copy-source-server-side-encryption-customer-key Specifies the base64-encoded 256-bit encryption key to use to decrypt the source object.
x-amz-copy-source-server-side-encryption-customer-key-md5 Specifies the base64-encoded 128-bit MD5 digest of the encryption key used to decrypt the source object.

Supported Amazon S3 Clients

Here are some examples of configuring various client applications to talk to Object Storage's Amazon S3-compatible endpoints. Use an existing or create a special signing key to authenticate with Amazon S3, which is an Access Key/Secret Key pair. See Managing User Credentials for details.

AWS SDK for Java

The following is an example of configuring AWS SDK for Java.

                // Get S3 credentials from the console and put them here
                
AWSCredentialsProvider credentials = new AWSStaticCredentialsProvider(new BasicAWSCredentials(
"gQ4+YC530sBa8qZI6WcbUbtH8oar0exampleuniqueID",
"7fa22331ebe62bf4605dc9a42aaeexampleuniqueID"));

// Your namespace
String namespace = "namespace";

// The region to connect to
String region = "us-ashburn-1";

// Create an S3 client pointing at the region
String endpoint = String.format("%s.compat.objectstorage.%s.oraclecloud.com",namespace,region);
AwsClientBuilder.EndpointConfiguration endpointConfiguration = new AwsClientBuilder.EndpointConfiguration(endpoint, region);
AmazonS3 client = AmazonS3Client.builder()
.standard()
.withCredentials(credentials)
.withEndpointConfiguration(endpointConfiguration)
.disableChunkedEncoding()
.enablePathStyleAccess()
.build();

AWS SDK for Javascript

The following is an example of configuring AWS SDK for Javascript.

s3 = new AWS.S3({
  region: 'us-ashburn-1',
  endpoint: 'https://' + mynamespace + '.compat.objectstorage.us-ashburn-1.oraclecloud.com',
  accessKeyId: 'gQ4+YC530sBa8qZI6WcbUbtH8oar0exampleuniqueID',
  secretAccessKey: '7fa22331ebe62bf4605dc9a42aaeexampleuniqueID',
  s3ForcePathStyle: true,
  signatureVersion: 'v4',
});

AWS SDK for Python (Boto 3)

The following is an example of configuring AWS SDK for Python (Boto 3).

import boto3
  
s3 = boto3.resource(
    's3',
    aws_access_key_id="gQ4+YC530sBa8qZI6WcbUbtH8oar0exampleuniqueID",
    aws_secret_access_key="7fa22331ebe62bf4605dc9a42aaeexampleuniqueID",
    region_name="us-phoenix-1", # Region name here that matches the endpoint
    endpoint_url="https://mynamespace.compat.objectstorage.us-phoenix-1.oraclecloud.com" # Include your namespace in the URL
)
  
# Print out the bucket names
for bucket in s3.buckets.all():
    print bucket.name