This topic highlights key features of Storage Gateway.
Using Storage Gateway, your applications can interact with Oracle Cloud Infrastructure Object Storage through standard NFSv4 protocols. You connect Storage Gateway file systems to Object Storage buckets. Storage Gateway stores files as objects in an Oracle Cloud Infrastructure Object Storage bucket and supports multipart uploads for large objects. Object Storage does not, however, support symbolic links, hard links, or special device files.
The built-in data integrity checks ensure that data is validated as it moves through the data path from Storage Gateway to Oracle Cloud Infrastructure Object Storage. Checksum verification helps ensure data integrity. Metadata integrity checks ensure that the metadata is in a consistent state. The checksum for each file can be read using a custom interface.
The Oracle Cloud Infrastructure Object Storage service supports multipart uploads for faster, more efficient, and resilient uploads. Storage Gateway can use multipart upload for files larger than 128 MB. With multipart uploads, individual parts of an object can be uploaded in parallel to reduce the amount of time you spend uploading. Multipart uploads also minimize the impact of network failures by letting you retry a failed part upload instead of requiring you to retry an entire object upload. See Using Multipart Uploads for details.
Beginning with version 1.3, Storage Gateway provides partial update capabilities to:
- Reduce upload latency.
- Improve the use of available network bandwidth.
- Reduce the minimum required cache size.
- Enable ingestion of single files that are larger than the Storage Gateway cache size.
Storage Gateway's partial update capability leverages the Object Storage service multipart upload functionality to break a large file into smaller parts, and then upload the parts in parallel. After uploading, the service reconstructs them as a single object. With partial update, the service can upload only the modified file parts and reconstruct the object using the unchanged parts that already exist in Object Storage. The service does not need to overwrite the existing object by uploading the entire file.
Since the full file does not have to exist within the Storage Gateway cache at one time, partial update enables the service to ingest single files that are larger than the allocated cache size.
The default file part size is 1GB, with a maximum size of 10TB for the full file. Although the part size is configurable per file system, Oracle does not recommend that you use custom sizes. Custom part sizes do not improve upload performance, but can reduce the maximum supported full file size. The maximum file size is 10,000 times the file part size, with a hard cap of 10TB. If you customize the file part size, you do not need to restart the Storage Gateway, but you must reconnect to the affected file system.
All file parts in the cache are managed by the least recently used (LRU) cache management policy.
To use partial update for objects stored in the Archive Storage tier, you must first restore the object.
See Upgrading Storage Gateway for information about upgrading your Storage Gateway system to use partial update.
In addition to uploading to buckets in the Object Storage Standard tier, Storage Gateway supports uploading to and restoring objects from buckets in the Archive Storage tier.
When you create a file system, you specify the storage tier in which to create the corresponding Object Storage bucket.
- The default Standard Object Storage tier is used for storing data to which you need fast, immediate, and frequent access.
- The Archive Storage tier is used for storing data that is accessed infrequently and requires long retention periods.
While Archive Storage is more cost effective than Object Storage for preserving cold data, you must first restore the objects before you can access them. The restoration process can take up to four hours depending on the size of the object. See Overview of Archive Storage and Restoring Files and Objects from Archive Storage for details.
Storage Gateway supports Oracle Cloud Infrastructure Object Storage object lifecycle policies to manage the archiving and deletion of objects in a bucket according to a pre-defined schedule. Using object lifecycle policies, you can specify bucket creation in the Standard Object Storage tier, and then create a policy to schedule the subsequent movement of data to the Archive Storage tier. This lifecycle policy archival method is useful if you have on-premises applications that generate intermediary or temporary files and directories that are inappropriate for immediate archival. See Using Object Lifecycle Management for details.
Automated Object Deletion
When you delete a Storage Gateway file from a file system, the corresponding object in Object Storage is automatically deleted.
Storage Gateway lets you pin files to the file system cache for quick access. You can pin files to the cache for file systems connected to either the Object Storage Standard or Archive tier.
When you write a file to your Storage Gateway file system, the file is initially stored in the file system cache, and then asynchronously uploaded to your Oracle Cloud Infrastructure bucket. After a file has been uploaded, the cache manager can remove the file from the file system cache. To meet the cache threshold specified for the file system, cache is reclaimed using the Least Recently Used (LRU) cache management policy. If you want specific files to be available in the cache for quick access, you can pin the files to the file system cache. Once pinned, files are not removed from the file system cache until you explicitly unpin them.
The Storage Gateway performs automated "health checks" on the system to monitor the status of the following:
- Storage Gateway services and resources
- Local storage, file system cache, metadata storage, and log storage
Storage Gateway provides an integrated cloud transfer and synchronization feature called Cloud Sync that lets you back up and transfer files on local storage to and from Oracle Cloud Infrastructure Object Storage buckets. This new feature replaces the independent, downloadable cloud sync utility that was available in the previous Storage Gateway version.
You can use the Storage Gateway management console or CLI to create, monitor, and manage Cloud Sync jobs similar to other enterprise NAS backup/replication offerings. Cloud Sync runs as part of the Storage Gateway software inside the Docker instance on the host.
- Features of Storage Gateway
- POSIX-Compliant NFS Access to Oracle Cloud Infrastructure Object Storage
- Data Integrity with Checksum Verification
- Large File Support
- Support for Data Archival
- Automated Object Deletion
- Quick Access to Select Files with Cache Pinning
- Storage Gateway Health Check
- Integrated Cloud Transfer and Synchronization (Cloud Sync)