Using Object Lifecycle Management

Object Lifecycle Management lets you automatically manage the archiving and deletion of objects. By using Object Lifecycle Management to manage your Object Storage and Archive Storage data, you can reduce your storage costs and the amount of time you spend managing data.

Object Lifecycle Management works by defining rules that instruct Object Storage to archive or delete objects on your behalf within a given bucket. A bucket's lifecycle rules are collectively known as an object lifecycle policy. For example, you could have Object Storage automatically move objects to Archive Storage 30 days after creation, and then automatically delete the archived objects 120 days after creation.

Each Object Storage or Archive Storage bucket can have a single lifecycle policy consisting of up to 1,000 rules. Rules can have object name prefix and pattern matching conditions. You can create, edit, delete, enable, and disable individual rules in the Console as needed. To update a lifecycle policy using the CLI or API, overwrite the entire policy with a new policy. Ensure that the new policy is inclusive of all the policy rules that you want to apply to the bucket.

Required IAM Policies

Important

You cannot use Object Lifecycle Management until you authorize the Object Storage service to archive and delete objects on your behalf. See Service Permissions for more information.

User Permissions

To use Oracle Cloud Infrastructure, you must be given the required type of access in a policy  written by an administrator, whether you're using the Console or the REST API with an SDK, CLI, or other tool. If you try to perform an action and get a message that you don’t have permission or are unauthorized, confirm with your administrator the type of access you've been granted and which compartment  you should work in.

The policy Let Object Storage admins manage buckets and objects lets the specified group do everything with buckets and objects, including adding and managing lifecycle policies. See Details for Object Storage, Archive Storage, and Data Transfer for more information on Object Storage user permissions.

Service Permissions

To execute object lifecycle policies, you must authorize the service to archive and delete objects on your behalf. To do so, create the following policy in the root compartment of your tenancy:

Allow service objectstorage-<region_identifier> to manage object-family in compartment <compartment_name>

Because Object Storage is a regional service, you must authorize the Object Storage service in each region you use lifecycle policies. Object Storage ensures that your data is not read from any unauthorized region.

If you don't have permissions to write policies for the root compartment of your tenancy, contact your Oracle Cloud Infrastructure administrator. To determine the region identifier value of an Oracle Cloud Infrastructure region, see Regions and Availability Domains.

Instead of using the policy verbmanage, you can grant individual permissions to the service. For example:

Allow service objectstorage-<region_identifier> to {BUCKET_INSPECT, BUCKET_READ, OBJECT_INSPECT, OBJECT_CREATE, OBJECT_DELETE} in compartment <compartment_name>

If you're new to policies, see Getting Started with Policies and Common Policies.

Options

When creating object lifecycle policy rules, you have the following options:

  • When a lifecycle rule is created, the system generates a default name for that rule, for example lifecycle-rule-20190321-1559. This rule name identifies the current year, month, day, and time that the rule was created. You can use that system-generated name for your new rule or you can specify a different name for it.
  • You can use a rule to either archive or delete objects and specify the number of days until the specified action is taken.
  • You can apply a rule to all objects in a bucket. Alternatively, you can use object name filters to specify which objects the lifecycle rule applies to. You can select objects using both object name prefixes and pattern matching. See Using Object Name Filters for details.
  • You can decide whether a new rule is enabled or disabled upon creation.

Using Object Name Filters

Use object name filters to specify which objects the lifecycle rule applies to.

Important

If you want the rule to apply to all objects in the bucket, do not specify any object name filters.

You can add object filters in any order. Object Lifecycle Management evaluates the precedence of the rules as follows:

  1. Pattern exclusions
  2. Pattern inclusions
  3. Prefix inclusions

Using Prefix Matching to Filter Objects

When naming objects, you can use prefix strings without a delimiter so that certain bulk operations can be performed by matching on the prefix portion of the object name. For example, in the object names below, the string gloves_27_ serves as a prefix for matching purposes when performing lifecycle management archive or deletions:

gloves_27_dark_green.jpg
gloves_27_light_blue.jpg
gloves_27_deep_purple.jpg
gloves_27_bright_orange.jpg

See Object Naming Using Prefixes and Hierarchies for complete object naming details.

Using Pattern Matching to Filter Objects

Object Storage supports the following pattern matching characters to either include or exclude objects:

Character Description Pattern Examples Matches Doesn't Match
* Matches 0 or more characters *.tmp

foo.tmp

foo/bar/baz.tmp

tmp

Atmp

*.xls

.xls

/home/user/file.xlsx

xls

.xl

/archive/*

/archive/sub/dir/

/archive/1/2/3/4/foo.txt

/src/archive/a

archive/b

? Matches any one character X?Z

XyZ

X_Z

XZ

XYYZ

\ Escapes the next character \\dir\\sub\\*

\dir\sub\ABC

\dir\sub\

dir\sub\abc

dirsub

[...]

Matches a group of characters, which can be:

  • A set of characters, for example: [Zafg9@]. Matches any character in the brackets.
  • A range of characters, for example: [a-f]. Matches any character in the range:
    • [a-f] is equivalent to [abcdef].

    • For character ranges only the CHARACTER-CHARACTER pattern is supported:

      • [ab-yz] is not valid.
      • [a-mn-z] is not valid.
    • Character ranges cannot start with ^ or colon (:).

    • To include a hyphen (-) in the range, make it the first or last character.

[-ab3]

-

a

b

3

-a

-ab

3b

backup.tar.gz.[0-9]

backup.tar.gz.0

backup.tar.gz.5

backup.tar.gz.9

backup.tar.gz10

backup.tar.gz

page-[0-9]*

page-0

page-2

page-22

page-2X

page-

page-A1

\[a-z\] [a-z]

a

z

[a-z

Patterns are limited to 1024 characters. The following are examples of invalid patterns:

  • \
  • [^a-z]
  • [z-a]
  • [:isalpha:]

Scope and Constraints

Understand the following scope and constraints regarding object lifecycle policies:

  • When you create a lifecycle policy for a bucket, Object Storage applies that policy to all objects that exist in the bucket unless you add object name filters.
  • A rule that deletes an object always takes priority over a rule that would archive that same object.
  • When creating a lifecycle policy rule that deletes objects from Archive Storage, Archive Storage has a minimum retention requirement of 90 days. Objects deleted from Archive Storage that have not met the 90-day retention minimum are billed for 90 days of storage. For more information, see Overview of Archive Storage
  • You can create up to 1,000 lifecycle rules per bucket.

Working with Object Lifecycle Management Policies

You can create, delete, edit, or disable lifecycle policy rules using the Console, the Command Line Interface (CLI), an SDK, or the API.

Warning

Objects deleted on your behalf by lifecycle policies cannot be recovered. Be sure when creating and editing your lifecycle policies that you are not unintentionally deleting data you want to retain. Oracle recommends that you test your lifecycle policy on development data before using the policy in production.

Using the Console

To create a lifecycle policy rule
  1. Open the navigation menu. Under Core Infrastructure, click Object Storage.
  2. Choose the compartment containing bucket for which you want to create a lifecycle rule.
  3. Click the bucket name.
  4. Click Lifecycle Policy Rules under Resources to access the lifecycle policy rule list.
  5. Click Create Rule.
  6. Provide the following information:
    • Name: Required. The system generates a default rule name that reflects the current year, month, day, and time, for example lifecycle-rule-20190321-1559. If you change this default to any other rule name, use letters, numbers, dashes, underscores, and periods. Avoid entering confidential information.
    • Lifecycle Action: Select rule type Archive or Delete.
    • Number of Days: The number of days until the specified action is taken.
  7. Optionally, you can add one or more Object Name Filters to specify which objects the lifecycle rule applies to. You can choose objects using prefixes and pattern matching. If no object name filters are specified, the rule applies to all objects in the bucket.

    To create an object name filter:

    1. Click Add Filter.
    2. Select the Filter Type.
    3. Enter the Filter Value.
    4. Click Add Another Filter to add as many filters as you need for this rule.
  8. Select whether the rule is enabled or disabled upon creation using the State selector.
  9. Click Create.
To edit a lifecycle policy rule
  1. Open the navigation menu. Under Core Infrastructure, click Object Storage.
  2. Choose the compartment where the bucket is.
  3. Click the bucket name.
  4. Click Lifecycle Policy Rules under Resources to access the rule list.
  5. For the rule you want to edit, click the Actions icon (three dots), and then click Edit.
  6. In the Edit Lifecycle Rule dialog box, edit the following as needed for each rule you want to change:
    • Name: A user-friendly name for the rule. Avoid entering confidential information.
    • Lifecycle Action: Rule type Archive or Delete.
    • Number of Days: The number of days until the specified action is taken.
    • Object Name Filters: Edit, delete, or add a prefix or pattern filter.
  7. Click Save Changes.
To enable, disable, or delete a lifecycle policy rule

You can disable and enable a rule on demand using the Console. The system stops the execution of disabled or deleted rules immediately.

  1. Open the navigation menu. Under Core Infrastructure, click Object Storage.
  2. Choose the compartment where the bucket is.
  3. Click the bucket name.
  4. Click Lifecycle Policy Rules under Resources to access the rule list.
  5. For the rule you want to manage, click the Actions icon (three dots), and then click one of the following:
    • Enable (only displays if the rule is disabled)
    • Disable (only displays if the rule is enabled)
    • Delete

Using the Command Line Interface (CLI)

For information about using the CLI, see Command Line Interface (CLI). For a complete list of flags and options available for CLI commands, see the Command Line Reference.

To create or replace a lifecycle policy for a bucket

Open a command prompt and run oci os object-lifecycle-policy put to create or replace the object lifecycle policy for a bucket. To edit individual rules, replace the bucket's existing policy with a new version of the policy that includes the changes to your rules.

oci os object-lifecycle-policy put -ns <object_storage_namespace> -bn <bucket_name> --items <json_formatted_lifecycle_policy>
Tip

The --items option requires that you provide key-value pair input as valid formatted JSON. See Passing Complex Input and Using a JSON File for Complex Input for information about JSON formatting.

For example, the following lifecycle policy archives objects after 30 days and deletes them after 180 days:

oci os object-lifecycle-policy put -ns MyNamespace -bn MyBucket --items
'[
    {
        "action": "ARCHIVE",
        "is-enabled": true,
        "name": "ArchiveAfter30Days",
		"object-name-filter": {
          "exclusion-patterns": [
            "*.jpg"
          ], 
          "inclusion-patterns": [
            "*.doc"
          ], 
          "inclusion-prefixes": [
            "documents/"
          ]
        }, 
        "time-amount": 30, 
        "time-unit": "DAYS"
      }, 
      {
        "action": "DELETE", 
        "is-enabled": true, 
        "name": "DeleteAfter180Days", 
        "object-name-filter": {
          "exclusion-patterns": null, 
          "inclusion-patterns": null, 
          "inclusion-prefixes": null
        }, 
        "time-amount": 180, 
        "time-unit": "DAYS"
      }
]'

On Windows, to pass complex input to the CLI as a JSON string, you must enclose the entire block in double quotes. Inside the block, each double quote for the key and value strings must be escaped with a backslash (\) character.

For example:

oci os object-lifecycle-policy put -ns MyNamespace -bn MyBucket --items "[{\"action\":\"ARCHIVE\",\"is-enabled\":true,\"name\":\"Archive After 30 Days\",\"object-name-filter\":{\"exclusion-patterns\":[\"*.jpg\"],\"inclusion-patterns\":[\"*.doc\"],\"inclusion-prefixes\":[\"documents/\"]},\"time-amount\":30,\"time-unit\":\"DAYS\"},{\"action\":\"DELETE\",\"is-enabled\":true,\"name\":\"DeleteAfter180Days\",\"object-name-filter\":{\"exclusion-patterns\":null,\"inclusion-patterns\":null,\"inclusion-prefixes\":null},\"time-amount\":180,\"time-unit\":\"DAYS\"}]"
To delete a bucket's lifecycle policy

Open a command prompt and run oci os object-lifecycle-policy delete to delete a bucket's object lifecycle policy.

oci os object-lifecycle-policy delete -ns <object_storage_namespace> -bn <bucket_name>
To get a bucket's lifecycle policy

Open a command prompt and run oci os object-lifecycle-policy get to get a bucket's object lifecycle policy.

oci os object-lifecycle-policy get -ns <object_storage_namespace> -bn <bucket_name>