Object Storage Amazon S3 Compatibility API Support

AWS SDK for Java

AWS SDK for Javascript

The AWS SDK for Javascript repository, documentation links, and installation instructions are available on GitHub: https://github.com/aws/aws-sdk-js.

Here is an example of configuring AWS SDK for Javascript to use Object Storage

s3 = new AWS.S3({
  region: 'us-ashburn-1',
  endpoint: 'https://' + mynamespace + '.compat.objectstorage.us-ashburn-1.oraclecloud.com',
  accessKeyId: 'gQ4+YC530sBa8qZI6WcbUbtH8oar0exampleuniqueID',
  secretAccessKey: '7fa22331ebe62bf4605dc9a42aaeexampleuniqueID',
  s3ForcePathStyle: true,
  signatureVersion: 'v4',
});

AWS SDK for Python (Boto3)

The AWS SDK for Python (Boto3) repository, documentation links, and installation instructions are available on GitHub: https://github.com/boto/boto3.

Here is an example of configuring AWS SDK for Python to use Object Storage

import boto3
  
s3 = boto3.resource(
    's3',
    aws_access_key_id="gQ4+YC530sBa8qZI6WcbUbtH8oar0exampleuniqueID",
    aws_secret_access_key="7fa22331ebe62bf4605dc9a42aaeexampleuniqueID",
    region_name="us-phoenix-1", # Region name here that matches the endpoint
    endpoint_url="https://mynamespace.compat.objectstorage.us-phoenix-1.oraclecloud.com" # Include your namespace in the URL
)
  
# Print out the bucket names
for bucket in s3.buckets.all():
    print bucket.name

Mounting Object Storage Buckets Using s3fs

s3fs lets Linux and macOS mount Object Storage as a file system. The s3fs repository, documentation links, installation instructions, and examples are available on GitHub: https://github.com/s3fs-fuse/s3fs-fuse.

s3fs isn't suitable for all applications. Understand the following limitations:

To mount an Object Storage bucket as a file system

1. Follow the installation instructions provided on GitHub: https://github.com/s3fs-fuse/s3fs-fuse.

   If you're unable to install using a pre-built package, follow the compilation instructions here: https://github.com/s3fs-fuse/s3fs-fuse/blob/master/COMPILATION.md.

2. Review and perform the prerequisites in Amazon S3 Compatibility API Prerequisites. You need an Access Key/Secret Key pair and a proper IAM policy that lets you mount a bucket as a file system. For example:

   Allow group s3fsAdmins to manage object-family in compartment MyCompartment

3. Enter your Access Key/Secret Key pair credentials in a ${HOME}/.passwd-s3fs credential file:

   cat ${HOME}/.passwd-s3fs
   access_key:secret_key

   For example:

   cat ${HOME}/.passwd-s3fs
   gQ4+YC530sBa8qZI6WcbUbtH8oar0exampleuniqueID:7fa22331ebe62bf4605dc9a42aaeexampleuniqueID

   Then, set owner-only permissions for the credential file:

   chmod 600 ${HOME}/.passwd-s3fs

4. Create a mount point to mount an Object Storage bucket:

   mkdir /path/to/<local_directory_name>
   s3fs bucket_name local_directory_name -o passwd_file=${HOME}/.passwd-s3fs -o url=https://<namespace_name>.compat.objectstorage.<region_ID>.oraclecloud.com -o use_path_request_style -o kernel_cache -o multipart_size=128 -o parallel_count=50 -o multireq_max=100 -o max_background=1000 [-o endpoint=<region_ID>]

   Where:

   • bucket_name is the name of the bucket that you want to mount.
   • local_directory_name is the name of the local directory where you want to mount the bucket.
   • namespace_name is the unique system-generated assigned to your tenancy at account creation time. You can use the CLI or the Console to obtain your namespace name. See Object Storage Namespaces for details.
   • region_ID is the region identifier where the bucket resides. See Regions and Availability Domains for details.
   • endpoint: If you want to mount a bucket that was created in your home region, you do not need to specify the endpoint parameter. If you want to mount a bucket that was created in a different region, you need to specify the endpoint parameter.

5. To automatically mount the bucket as a file system on system startup using s3fs, add following to the /etc/fstab file:

   bucket_name /path/to/local_directory_name fuse.s3fs use_path_request_style,passwd_file=/root/.s3fs-password,url=https://namespace_name.compat.objectstorage.region_ID.oraclecloud.com,endpoint=region_ID kernel_cache,multipart_size=128,parallel_count=50,multireq_max=100,max_background=1000,_netdev

6. To verify the s3fs bucket mount, run the df -h command. The output shows the new mount point for the bucket. Navigate to the new mount point and run the ls command to list all objects in the bucket.

To troubleshoot mounting an Object Storage bucket

• If you get authorization errors, review your IAM policies and ensure you have one that lets you mount a bucket as a file system. For example:

  Allow group s3fsAdmins to manage object-family in compartment MyCompartment

• Ensure that you're using the correct namespace name in the URL in the s3fs command. To verify your namespace name, see Object Storage Namespaces.
• Ensure that the named bucket that you are trying to mount exists and is in a compartment that you have access to. Use one of the following ways to verify the bucket name:
  • Log into the Console and find the named bucket in the compartment that you have access to.
  • Use the CLI command oci os bucket list --namespace <object_storage_namespace> --compartment-id <target_compartment_id>.
• To mount a bucket that was created in a region other than your home region, you need to specify that other region in both the url and endpoint parameters.
• If you mount a bucket as the root user, other users can't list or access objects in the bucket unless you add -o allow_other to the s3fs command or allow_other to the /etc/fstab mount options. You can also supply specific UID and GID parameters to specify user access details.

• If you reviewed and verified the troubleshooting solutions and need to contact Support, run the mount command again in DEBUG mode to get more failure details. Add the following to the end of the command and save the output:

  -o dbglevel=info -f -o curldbg

To unmount an Object Storage bucket from a file system

umount /path/to/<local_directory_name>