Patch and Update an Exadata Database Service on Cloud@Customer System

Learn to update and patch the Exadata Database Service on Cloud@Customer System

Perform User Managed Maintenance Updates

Maintaining a secure Exadata Cloud@Customer system in the best working order requires you to perform the following tasks regularly:
  • Patching the Oracle Grid Infrastructure and Oracle Database software on the VM Cluster virtual machines. For information and instructions, see Patching and Updating an Exadata Cloud@Customer System.
  • Updating the operating system on the VM Cluster virtual machines. For information and instructions, see Updating Guest VM Operating System and Oracle Clusterware Configuration and Administration.

Patching and Updating an Exadata Database Service on Cloud@Customer System

Learn how to perform patching operations on Exadata database virtual machines and Database Homes by using the Console, API, or the CLI.

For information and instructions on patching the system by using the dbaascli utility, see Patching and Updating an Exadata Database Service on Cloud@Customer System Manually.

For more information and examples for applying database quarterly patches on Exadata Cloud@Customer refer to My Oracle Support note: How to Apply Database Quarterly Patch on Exadata Cloud Service and Exadata Cloud at Customer Gen 2 (Doc ID 2701789.1).

For more guidance on achieving continuous service during patching operations, see the Application Checklist for Continuous Service for MAA Solutions white paper.

Patching and Updating VM Clusters and Database Homes

Learn how to perform patching operations on VM Cluster Grid Infrastructure (GI) and Database Homes using the Console or API

About Patching and Updating VM Cluster's GI and Database Homes

Patching a VM cluster updates components on each of the VM guests in the VM cluster. VM cluster patching updates the grid infrastructure (GI) and Database Home patching updates the Oracle Database software shared by the databases in that home.

For more information on available patches, see My Oracle Support note https://support.oracle.com/epmos/faces/DocContentDisplay?id=2333222.1.

Consider the following best practices:
  • Because patching a system requires a reboot, plan to run the operations at a time when they will have minimal impact on users.
  • Oracle recommends that you back up your databases before you apply any patches. For information about backing up the databases, see Managing Database Backup and Recovery on Exadata Database Service on Cloud@Customer.
  • Oracle recommends your Oracle Grid Infrastructure RU version be no more than 6 months behind your latest database RU version. When updating the database version, you should also update the GI version if possible.
  • To patch a database to a version other than the database version of the current home, move the database to a Database Home running the target version. See Using the Console to Move a Database to Another Home.
Prerequisites for Patching and Updating an Exadata Database Service on Cloud@Customer System

Check and apply the latest Cloud patches that are dowloaded and made available by Oracle on the CPS host.

Ensure that the following conditions are met to avoid patching failures:
  • The /u01 directory on the database host file system has at least 15 GB of free space for the execution of patching processes.
  • The Oracle Clusterware is up and running on the VM cluster.
  • All nodes of the VM cluster are up and running.
Using the Console for Patching and Updating VM Cluster's GI and Database Homes

Learn how to use the console to view the history of patch operations on VM cluster and Database Homes, apply patches, and monitor the status of patch operations.

Oracle recommends that you use the precheck action to ensure your VM cluster or Database Home has met the requirements for the patch you want to apply.

Using the Console to Perform a Grid Infrastructure Patch Operation on a VM Cluster

Learn to apply Grid Infrastructure patches on a VM cluster.

  1. Open the navigation menu. Under Oracle Database, click Exadata Database Service on Cloud@Customer.
    VM Clusters is selected by default.
  2. Choose your Compartment.
    A list of VM Clusters is displayed for the chosen Compartment.
  3. In the list of VM clusters, click the VM cluster on which you want to perform a patch operation.
  4. Under Oracle Grid Infrastructure Version, click View Updates.
  5. Review the scope:
    • VM Cluster: Automatically set to the context from which you have launched this page.
    • Database Home: Automatically set to the context from which you have launched this page. If you have not set the context, then select the Database Home first.
  6. Review the list of available patches for the VM cluster.
  7. Click the Actions icon (three dots) for the patch you are interested in, and then click one of the following actions:
    • Precheck: Check for any prerequisites to make sure that the patch can be successfully applied. Oracle highly recommends that you run this operation before you apply a patch. Precheck does not cause any availability impact to the cluster, everything remains operational.
    • Apply Patch: Applies the selected patch.
  8. Confirm when prompted.

The patch list displays the status of the operation. While the precheck is running, the patch's status shows Checking. While a patch is being applied, the patch's status shows Applying and the VM cluster's status shows Updating. During patching, lifecycle operations on the VM cluster and its resources are temporarily unavailable. If patching completes successfully, the patch's status changes to Applied and the VM cluster's status changes to Available. You can view more details about an individual patch operation by clicking Update History. Grid Infrastructure patching is done in a rolling fashion, node by node, and the cluster resources will be stopped and restarted on each node.

Using the Console to Perform a Patch Operation on a Database Home

Learn to apply patches on a Database Home.

  1. Open the navigation menu. Under Oracle Database, click Exadata Database Service on Cloud@Customer.
    VM Clusters is selected by default.
  2. Choose your Compartment.
    A list of VM Clusters is displayed for the chosen Compartment.
  3. In the list of VM clusters, click the VM cluster where the Database Home is located.
  4. Under Resources, click Database Homes.
  5. In the list of Database Homes, click the Database Home on which you want to perform a patch operation.
  6. Under Database Software Version, click View Patches.
  7. Review the scope:
    • Database Home: Automatically set to the context from which you have launched this page.
  8. Review the list of available patches for the Database Home.

    The Oracle Provided Database Software Images tab displays generally-available Oracle Database software images that you can use to patch your database. Oracle images that can be used for patching have the update Type of "Patch".

    The Custom Database Software Images tab allows you to select a database software image that you have created in advance. Use the Select a Compartment selector to specify the compartment that contains the database software image.

  9. Click the Actions icon (three dots) for the patch you are interested in, and then click one of the following actions:
    • Precheck: Check for any prerequisites to make sure that the patch can be successfully applied. Oracle highly recommends that you run this operation before you apply a patch. The Precheck does not cause any availability impact to the cluster, everything remains operational.
    • Apply: Applies the selected patch.
  10. Confirm when prompted.

The patch list displays the status of the operation. While the precheck is running, the patch's status shows Checking. While a patch is being applied, the patch's status shows Applying, the status of the Database Home and the databases in it display as Updating, and lifecycle operations on the VM cluster and its resources are temporarily unavailable. Patches are applied to the Database Home in a rolling fashion, node by node, and each database in the home is stopped and then restarted. This may result in temporary service disruption. If patching completes successfully, the patch's status changes to Applied and the Database Home's status changes to Available. You can view more details about an individual patch operation by clicking Update History.

Using the Console to View Update History

Each update history entry represents an attempted patch operation and indicates whether the operation was successful or failed. You can retry a failed patch operation. Repeating an operation results in a new patch history entry.

Update history views in the Console do not show patches that were applied by using command line tools such as dbaascli.

Using the Console to View the Update History of a VM Cluster

Learn how to view the history of patches applied on a VM cluster.

  1. Open the navigation menu. Under Oracle Database, click Exadata Database Service on Cloud@Customer.
    VM Clusters is selected by default.
  2. Choose your Compartment.
    A list of VM Clusters is displayed for the chosen Compartment.
  3. In the list of VM clusters, click the VM cluster you are interested in.
  4. Under Oracle Grid Infrastructure Version, click View Patches.
  5. Click Update History.

The history of patch operations for that VM cluster is displayed, along with the history of patch operations on its Database Homes.

Using the Console to View the Update History of a Database Home

Learn how to view the history of patches applied on a Database Home.

  1. Open the navigation menu. Under Oracle Database, click Exadata Database Service on Cloud@Customer.
    VM Clusters is selected by default.
  2. Choose your Compartment.
    A list of VM Clusters is displayed for the chosen Compartment.
  3. In the list of VM clusters, click the VM cluster where the Database Home is located.
  4. Under Resources, click Database Homes.
    A list of Database Homes is displayed.
  5. In the list of Database Homes, click the Database Home you are interested in.
  6. Under Database Software Version, click View Patches.
  7. Click Update History.

The history of patch operations for that Database Home is displayed, along with the history of patch operations on the VM cluster to which it belongs.

Using the Console to Move a Database to Another Home

You can update the version of a VM cluster database by moving it to a Database Home that is running the version of Oracle Database you are interested in.

  1. Open the navigation menu. Under Oracle Database, click Exadata Database Service on Cloud@Customer.
    VM Clusters is selected by default.
  2. Choose your Compartment.
    A list of VM Clusters is displayed for the chosen Compartment.
  3. In the list of VM clusters, click the VM cluster where the database you want to move is located.
  4. Under Resources, click Database Homes.
  5. In the list of Database Homes, click the Database Home you are interested in.
    A list of databases is displayed.
  6. In the list of databases, click the database you are interested in.
  7. Click Move Database.
  8. Select the target Database Home.
  9. Click Move Database.
    The database will be stopped in the current home and then restarted in the destination home.
  10. Confirm the move operation.

The database is moved in a rolling fashion. The database instance will be stopped, node by node, in the current home and then restarted in the destination home. While the database is being moved, the Database Home and Database statuses display as Updating. The Database Home location, shown under Database Version, displays as Moving Database. When the operation completes, Database Home is updated with the current home. Datapatch is executed automatically, as part of the database move, to complete post-patch SQL actions for all patches, including one-offs, on the new Database Home. If the database move operation is unsuccessful, then the status of the database displays as Failed, and the Database Home field provides information about the reason for the failure.

Using the API for Patching and Updating VM Cluster and Database Homes

Use various API features to help manage patching an Oracle Exadata Database Service on Cloud@Customer system.

For information about using the API and signing requests, see "REST APIs" and "Security Credentials". For information about SDKs, see "Software Development Kits and Command Line Interface".

Use these API operations to manage patching VM clusters, Database Homes and Databases.

VM cluster:

  • UpdateVmCluster

Database Homes:

  • CreateDbHome
  • UpdateDbHome
  • DeleteDbHome

Database:

  • CreateDatabase
  • UpdateDatabase
  • DeleteDatabase

Use UpdateVMCluster to patch the Oracle Grid Infrastructure on the VM Cluster. Use UpdateDbHome to patch the Database Software of the Database Home. Use UpdateDatabase to move a database to a different Database Home, thereby updating the database to the same version as the target Database Home.

For the complete list of APIs for the Database service, see "Database Service API".

Updating Guest VM Operating System

Learn to update the operating system image on Exadata Cloud@Customer VM cluster nodes in an automated manner from the OCI console and APIs.

This automated feature simplifies and speeds up VM cluster patching, makes patching less error-prone, and eliminates the need to use Patch Manager.

When you apply a patch, the system runs a precheck operation to ensure that your Exadata Cloud@Customer VM cluster, database system, or Database Home meets the requirements for that patch. If the precheck is not successful, then the patch is not applied, and the system displays a message that the patch cannot be applied because the precheck failed. A separate precheck operation that you can run in advance of the planned update is also available.

Supported Software Versions and Update Restrictions

Minimum requirements for updating to Exadata image release 23.1.0.0.0 (Oracle Linux 8-based image):

Note

These are just the minimum requirements. If you want to update Grid Infrastructure and/or Oracle Database to meet the Exadata 23.1 requirements, then the recommendation is to update to the latest available versions of Grid Infrastructure and Oracle Database, and not to the minimum.
  • Exadata Image (Guest OS): Exadata image release 22.1.0 (May 2022) or 21.2.10 (March 2022). Systems running versions older than 21.2.10 will first need to upgrade to at least 22.1.0 (May 2022) or 21.2.10 (March 2022) before updating to 23.1.0.0.0. This applies to both storage and database servers.
    • In addition to performing minor version updates to the Exadata VM Cluster images, you can update to a new major version if the currently installed version is 19.2 or higher. For example, if the VM cluster is on version 20, then you can update it to version 21.
    • The latest 4 (N to N-3) or more minor versions of each major version of the VM Cluster images are available through the console to apply.
  • Oracle Grid Infrastructure: Exadata image release 23.1.0.0.0 supports the following minimum or newer Oracle Grid Infrastructure versions.
    • Release 19c: Version 19.15, April 2022 Release Update (RU) and newer (Default)
    • Release 21c: Version 21.6, April 2022 Release Update (RU) and newer
  • Oracle Database: Exadata System Software 23.1 supports the following minimum versions or newer for new database installations.
    • Release 19c: Version 19.15, April 2022 Release Update (RU) and newer (Default)
    • Additional supported database releases under Market Driven Support or Quarterly Updates exception approval:
      • Release 12.2.0.1, Release Update (RU) 12.2.0.1.220118 (Jan 2022)
      • Release 12.1.0.2, Bundle Patch 12.1.0.2.220719 (Jul 2022) - requires patch 30159782
      • Release 11.2.0.4, Bundle Patch 11.2.0.4.210119 (Jan 2021) - requires patch 30159782, patch 33991024
  • If you have an Exadata infrastructure maintenance operation scheduled to start within the next 24 hours, then the Exadata Image update feature is not available.
  • Once the VM cluster is upgraded to Exadata Database Service Guest VM OS 23.1, you will be able to add a new VM or a new database server to this VM cluster if Exadata Cloud@Customer Infrastructure is running an Exadata System Software version 22.1.16 and later.
    Note

    Upgrade to Exadata System Software 23.1 for Exadata Cloud@Customer Infrastructure will be available with February 2024 update cycle.
Using the Console to Update Guest VM Operating System

To update the guest VM operating system with the Console, be prepared to provide values for the fields required.

Note

Once the VM cluster is upgraded to Exadata Database Service Guest VM OS 23.1, you will be able to add a new VM or a new database server to this VM cluster if Exadata Cloud@Customer Infrastructure is running an Exadata System Software version 22.1.16 and later.

Upgrade to Exadata System Software 23.1 for Exadata Cloud@Customer Infrastructure will be available with February 2024 update cycle.

  1. Open the navigation menu. Under Oracle Database, click Exadata Database Service on Cloud@Customer.
  2. Choose your Compartment.
    A list of VM Clusters is displayed for the chosen Compartment.
  3. In the list of cloud VM clusters, click the name of the cluster you want to patch to display the cluster details.
  4. In the Version section, to the right of the Updates Available, click View Updates to display the Updates page.
  5. Review the list of available software updates and locate the operating system patch you are applying.
  6. Click the Actions icon (three dots) at the end of the row listing the patch you are interested in, and then click one of the following actions:
    • Run Precheck. Precheck checks the prerequisites to ensure that the patch can be successfully applied. Oracle highly recommends that you run the precheck operation before you apply a patch. The reason is that things can change in a database any time, and the precheck you run just before running a patch may find errors that the previous precheck did not find.
      Note

      If the precheck fails, the system displays a message in the Apply Exadata OS Image Update dialog that the last precheck has failed. Oracle recommends that you run the precheck again. Click the Actions icon (three dots)at the end of the row listing the OS patch to view the dialog.
    • Apply Exadata OS Image Update. This link displays the Apply Exadata Image Update dialog that you use to apply the patch. The dialog shows the name of the database system you are patching, the current version of the database, and the new version of the database after the patch is applied. To start the process, click Apply Exadata OS Image Update.
    • Copy OCID. This copies the Oracle Cloud ID. This can be used when troubleshooting a patch or to give to Support when contacting them.
      Note

      While the patch is running:
      • Run Precheck and Apply OS Image Update are not available. When the patch has completed, these actions are available again.
      • If the Exadata infrastructure containing this VM cluster is scheduled for maintenance that conflicts with the patching operation, the patch fails and the system displays a message explaining why. After the infrastructure maintenance is complete, run the patch operation again.
  7. Confirm when prompted.

    The patch list displays the status of the operation in the Version section of the database details page. Click View Updates to view more details about an individual patch status and to display any updates that are available to run. If no new updates are available, the system displays a message that says No Updates Available.

Using the Console to Rollback or Retry Failed Guest VM Operating System Update

To update the guest VM operating system with the Console, be prepared to provide values for the fields required.

  1. Open the navigation menu. Under Oracle Database, click Exadata Database Service on Cloud@Customer.
  2. Choose your Compartment.
    A list of VM Clusters is displayed for the chosen Compartment.
  3. In the list of VM Clusters, click the name of the cluster you want to patch to display the cluster details.

    If applying update has failed, then on the VM Cluster Details page, a banner with the options Roll Back and Retry Apply is displayed.

    Choose an appropriate option.

    1. Click Retry Apply.

      Apply Exadata OS Image Update dialog is displayed with the options Apply Exadata Image Update and Run Precheck.

      Choose an appropriate option.

      (or)

    2. Click Roll Back.

      Confirm Rollback Operation dialog is displayed.

      Click Roll Back to confirm.

  4. You can also Apply Exadata Image Update from the Updates page.
    1. In the Version section, to the right of the Updates Available, click View Updates to display the Updates page.
    2. Click the Actions icon (three dots) and then select the Apply Exadata OS Image Update option.
Using the API to Update Guest VM Operating System

Review the list of API calls to update guest VM operating system.

For information about using the API and signing requests, see REST APIs and Security Credentials. For information about SDKs, see Software Development Kits and Command Line Interface.

Use these API operations to upgrade the Oracle Grid Infrastructure in a VM cluster and view the cluster's update history:
  • ListVmClusterUpdates
  • GetVmClusterUpdate
  • ListVmClusterUpdateHistoryEntries
  • GetVmClusterUpdateHistoryEntry
  • UpdateVmCluster

For the complete list of APIs, see Database Service API.

Upgrading Oracle Grid Infrastructure on an Exadata Cloud@Customer VM Cluster

Learn to upgrade Oracle Grid Infrastructure on an Exadata Cloud@Customer VM cluster using the Oracle Cloud Infrastructure Console or API.

Upgrading enables you to provision Oracle Database Homes and databases that use the most current Oracle Database software.

About Upgrading Oracle Grid Infrastructure

Upgrading the Oracle Grid Infrastructure (GI) on a VM cluster involves upgrading all the compute nodes in the instance. The upgrade is performed in a rolling fashion, with only one node being upgraded at a time.

  • Oracle recommends running an upgrade precheck to identify and resolve any issues that would prevent a successful upgrade.
  • You can monitor the progress of the upgrade operation by viewing the associated work requests.
  • If you have an Exadata infrastructure maintenance operation scheduled to start within the next 24 hours, then the GI upgrade feature is not available.
  • During the upgrade, you cannot perform other management operations such as starting, stopping, or rebooting nodes, scaling CPU, provisioning or managing Database Homes or databases, restoring a database, or editing IORM settings. The following Data Guard operations are not allowed on the VM cluster undergoing a GI upgrade:
    • Enable Data Guard
    • Switchover
    • Failover to the database using the VM cluster (a failover operation to standby on another VM cluster is possible)
Using the Console to Manage Oracle Grid Infrastructure Upgrade

You can use the Console to perform a precheck prior to upgrading your Oracle Grid Infrastructure (GI) and to perform the GI upgrade operation.

Using the Console to Precheck Your VM Cluster Prior to Upgrading

  1. Open the navigation menu. Under Oracle Database, click Exadata Database Service on Cloud@Customer.
  2. Choose your Compartment.
    A list of VM Clusters is displayed for the chosen Compartment.
  3. In the list of cloud VM clusters, click the name of the cluster you want to patch to display the cluster details.
  4. Under Version, click the View Updates link beside the Updates Available field.
  5. Click View Updates to view the list of available patches and upgrades.
  6. Click the Actions icon (three dots) at the end of the row listing the Oracle Grid Infrastructure (GI) upgrade, then click Run Precheck.
  7. In the Confirm dialog, confirm you want to upgrade to begin the precheck operation.
Using the Console to Upgrade Oracle Grid Infrastructure of a VM Cluster

  1. Open the navigation menu. Under Oracle Database, click Exadata Database Service on Cloud@Customer.
  2. Choose your Compartment.
    A list of VM Clusters is displayed for the chosen Compartment.
  3. In the list of cloud VM clusters, click the name of the cluster you want to patch to display the cluster details.
  4. Under Version, click the View Updates link beside the Updates Available field.
  5. Click View Updates to view the list of available patches and upgrades.
  6. Click the Actions icon (three dots) at the end of the row listing the Oracle Grid Infrastructure (GI) upgrade, then click Upgrade Grid Infrastructure.
  7. In the Upgrade Grid Infrastructure dialog, confirm you want to upgrade the GI by clicking Upgrade Grid Infrastructure.

    If you haven't run a precheck, you have the option of clicking Run Precheck in this dialog to precheck your cloud VM cluster prior to the upgrade.

Using the API to Manage Oracle Grid Infrastructure Upgrade

Review the list of API calls to manage Oracle Grid Infrastructure upgrade.

For information about using the API and signing requests, see REST APIs and Security Credentials. For information about SDKs, see Software Development Kits and Command Line Interface.

Use these API operations to upgrade the Oracle Grid Infrastructure in a VM cluster and view the cluster's update history:
  • ListVmClusterUpdates
  • GetVmClusterUpdate
  • ListVmClusterUpdateHistoryEntries
  • GetVmClusterUpdateHistoryEntry
  • UpdateVmCluster

For the complete list of APIs, see Database Service API.

Upgrading Oracle Databases

Learn to upgrade Oracle Database 19c (Long Term Release) using the Console and the API.

The upgrade is accomplished by moving the Exadata database to a Database Home that uses the target software version. For Oracle Database release and software support timelines, see Release Schedule of Current Database Releases (Doc ID 742060.1).

Prerequisites to Upgrade Oracle Databases

Review the list of prerequsites to upgrade an Exadata Cloud@Customer Oracle Database instance.

  • The Exadata Cloud@Customer system software must use Oracle Linux 7 (OL7). See How to update the Exadata System Software (DomU) to 19 from 18 on the Exadata Cloud Service in OCI for instructions on manually updating the operating system.
  • The Oracle Grid Infrastructure must be version 19c. If patches are available for your Grid Infrastructure, Oracle recommends applying them prior to performing a database upgrade.
  • You must have an available Oracle Database Home that uses the two most recent versions of Oracle Database 19c available in Oracle Cloud Infrastructure. See Using the Console to Create Oracle Database Home on Exadata Cloud@Customer for information on creating a Database Home. You can use Oracle-published software images or a custom database software image based on your patching requirements to create Database Homes.
  • You must ensure that all pluggable databases in the container database that is being upgraded can be opened. Pluggable databases that cannot be opened by the system during the upgrade can cause an upgrade failure.
Your Oracle database must be configured with the following settings in order to upgrade:
  • The database must be in archive log mode.
  • The database must have flashback enabled.

See the Oracle Database documentation for your database's release version to learn more about these settings.

About Upgrading an Oracle Database

Before you upgrade the database, become familiar with the following procedures to prepare your database for upgrade.

  • Database upgrades involve database downtime. Keep this in mind when scheduling your upgrade.
  • Oracle recommends that you back up your database and test the new software version on a test system or a cloned version of your database before you upgrade a production database. See Creating an On-Demand Backup by Using the bkup_api Utility for information on creating an on-demand manual backup.
  • Oracle recommends running an upgrade precheck operation for your database prior to attempting an upgrade so that you can discover any issues that need mitigation prior to the time you plan to perform the upgrade. The precheck operation does not affect database availability and can be performed at any time that is convenient for you.
  • If your databases use Data Guard, you will need to disable or remove the Data Guard association prior to upgrading.
  • An upgrade operation cannot take place while an automatic backup operation is underway. Before upgrading, Oracle recommends disabling automatic backups and performing a manual backup. See Creating an On-Demand Backup by Using the bkup_api Utility and Customizing the Automatic Backup Configuration for more information.
  • After upgrading, you cannot use automatic backups taken prior to the upgrade to restore the database to an earlier point in time.
  • If you are upgrading a database that uses version 11.2 software, the resulting version 19c database will be a non-container database (non-CDB).
How the Upgrade Operation Is Performed by the Database Service

Familiarize yourself with what the Database service does during the upgrade process.

  • Executes an automatic precheck. This allows the system to identify issues needing mitigation and to stop the upgrade operation.
  • Sets a guaranteed restore point, enabling it to perform a flashback in the event of an upgrade failure.
  • Moves the database to a user-specified Oracle Database Home that uses the desired target software version.
  • Runs the Database Upgrade Assistant (DBUA) software to perform the upgrade.
Rolling Back an Oracle Database Unsuccessful Upgrade

If your upgrade does not complete successfully, then you have the option of performing a rollback.

Details about the failure are displayed on the Database Details page in the Console, allowing you to analyze and resolve the issues causing the failure.

A rollback resets your database to the state prior to the upgrade. All changes to the database made during and after the upgrade will be lost. The rollback option is provided in a banner message displayed on the database details page of a database following an unsuccessful upgrade operation. See Using the Console to Roll Back a Failed Database Upgrade for more information.

After Upgrading an Oracle Database

After a successful upgrade, note the following:

  • Check that automatic backups are enabled for the database if you disabled them prior to upgrading. See Customizing the Automatic Backup Configuration for more information.
  • Edit the Oracle Database
    COMPATIBLE
    parameter to reflect the new Oracle Database software version. See What Is Oracle Database Compatibility? for more information.
  • If your database uses a database_name.env file, ensure that the variables in the file have been updated to point to the 19c Database Home. These variables should be automatically updated during the upgrade process.
  • If you are upgrading a non-container database to Oracle Database version 19c, you can convert the database to a pluggable database after converting. See How to Convert Non-CDB to PDB (Doc ID 2288024.1) for instructions on converting your database to a pluggable database.
  • If your old Database Home is empty and will not be reused, you can remove it. See Using the Console to Delete an Oracle Database Home for more information.
Using the Console to Manage Oracle Database Upgrade

Oracle recommends that you use the precheck action to ensure that your database has met the requirements for the upgrade operation.

Using the Console to Run Oracle Database Upgrade Precheck or Perform Upgrade

  1. Open the navigation menu. Under Oracle Database, click Exadata Database Service on Cloud@Customer.
  2. Choose your Compartment.
    A list of VM Clusters is displayed for the chosen Compartment.
  3. In the list of VM clusters, click the name of the VM cluster that contains the database you want to upgrade.
  4. In the list of databases on the VM Cluster Details page, click the name of the database you want to upgrade to view the Database Details page.
  5. Click Upgrade.
  6. In the Upgrade Database dialog, select the following:
    • Oracle Database version: The drop-down selector lists only Oracle Database versions that are compatible with an upgrade from the current software version the database is using. The target software version must be higher than the database's current version.
    • Target Database Home: Select a Database Home for your database. The list of Database Homes is limited to those homes using the most recent versions of Oracle Database 19c software. Moving the database to the new Database Home results in the database being upgraded to the major release version and patching level of the new Database Home.

  7. Click one of the following:
    • Run Precheck: This option starts an upgrade precheck to identify any issues with your database that need mitigation before you perform an upgrade.
    • Upgrade Database: This option starts upgrade operation. Oracle recommends performing an upgrade only after you have performed a successful precheck on the database.
Using the Console to Roll Back a Failed Database Upgrade

  1. Open the navigation menu. Under Oracle Database, click Exadata Database Service on Cloud@Customer.
  2. Choose your Compartment.
    A list of VM Clusters is displayed for the chosen Compartment.
  3. In the list of VM clusters, click the name of the VM cluster that contains the database with the failed upgrade.
  4. Find the database that was unsuccessfully upgraded, and click its name to display details about it.
  5. The database must display a banner at the top of the details page that includes a Rollback button and details about what issues caused the upgrade failure.
  6. Click Rollback.
  7. In the Confirm rollback dialog, confirm that you want to initiate a rollback to the previous Oracle Database version.
Using the Console to View the Upgrade History of a Database

  1. Open the navigation menu. Under Oracle Database, click Exadata Database Service on Cloud@Customer.
  2. Choose your Compartment.
    A list of VM Clusters is displayed for the chosen Compartment.
  3. In the list of VM clusters, click the name of the VM cluster that contains the database you want to upgrade
  4. In the list of databases on the VM Cluster Details page, click the name of the database for which you want to view the upgrade history.
  5. On the Database Details page, under Database Version, click the View link that is displayed for databases that have been upgraded.

    This link does not appear for databases that have not been updated.

    The Updates History page is displayed. The table displayed on this page shows precheck and upgrade operations performed on the database.

Using the API to Upgrade Oracle Databases

Review the list of API calls to upgrade Oracle Databases.

For information about using the API and signing requests, see REST APIs and Security Credentials. For information about SDKs, see Software Development Kits and Command Line Interface.

Use the following APIs to manage database upgrades::
  • getDatabaseUpgradeHistoryEntry
  • ListDatabaseUpgradeHistoryEntries
  • UpgradeDatabase

For the complete list of APIs, see Database Service API.

Patching and Updating an Exadata Database Service on Cloud@Customer System Manually

This topic describes the procedures for patching and updating various components in Exadata Database Service on Cloud@Customer outside of the cloud automation. For information related to patching and updating with dbaascli, refer to "Patching Oracle Grid Infrastructure and Oracle Databases Using dbaascli".

Note

For more guidance on achieving continuous service during patching operations, see the Application Checklist for Continuous Service for MAA Solutions white paper.

Updating Software Manually

For daylight savings time and some routine or one-off patches it can be necessary for you to patch software manually.

To perform routine patching of Oracle Database and Oracle Grid Infrastructure software, Oracle recommends that you use the facilities provided by Oracle Exadata Database Service on Cloud@Customer. However, under some circumstances, it can be necessary for you to patch the Oracle Database or Oracle Grid Infrastructure software manually:
  • Daylight Savings Time (DST) Patching: Because they cannot be applied in a rolling fashion, patches for the Oracle Database DST definitions are not included in the routine patch sets for Exadata Database Service on Cloud@Customer. If you need to apply patches to the Oracle Database DST definitions, you must do so manually. See My Oracle Support Doc ID 412160.1.
  • Non-routine or One-off Patching: If you encounter a problem that requires a patch which is not included in any routine patch set, then work with Oracle Support Services to identify and apply the appropriate patch.

For general information about patching Oracle Database, refer to information about patch set updates and requirements in Oracle Database Upgrade Guide for your release.

Updating the Guest VM Operating System Manually

Learn about standard Exadata tools and techniques you can use to update the operating system components on the Exadata Database Service on Cloud@Customer Guest VMs.

You are responsible for managing patches and updates to the operating system environment on the Database Server Virtual Machines (VMs). For more information, read about updating Exadata Database Machine servers in Oracle Exadata Database Machine Maintenance Guide.

Preparing for an Operating System Update

To prepare for an operating system update for Oracle Exadata Database Service on Cloud@Customer, review this checklist of tasks.

Before you update your operating system, do each of these preparation tasks:

Determine the latest software update. Before you begin an update, to determine the latest software to use, review Exadata Cloud Service Software Versions in My Oracle Support note 2333222.1.

Updating the Operating System on All Virtual Machines of an Oracle Exadata Database Service on Cloud@Customer System

To update the operating system on the Database Server Virtual Machines (VMs), use the patchmgr tool.

Note

Customers who do not have My Oracle Support patch download privilege may obtain the Exadata patchmgr update utility and recent Exadata System Software releases using the Exadata Cloud@Customer Gen 2 utility exadata_updates.sh. For more information, see My Oracle Support Doc 2730739.1.

The patchmgr utility manages the entire update of one or more virtual machines remotely, including the pre-restart, restart, and post-restart steps of an Oracle Exadata Database Service on Cloud@Customer system.

You can run the utility either from one of your Oracle Exadata Database Service on Cloud@Customer virtual machines, or from another server running Oracle Linux. The server on which you run the utility is known as the driving system. You cannot use the driving system to update itself. Therefore, if the driving system is one of the virtual machines in a VM cluster that you are updating, then you must run the patchmgr utility more than once. The following scenarios describe typical ways of performing the updates:

  • Non-Exadata Driving System

    The simplest way to run the update the system is to use a separate Oracle Linux server to update all virtual machines in one operation.

  • Exadata Virtual Machine Driving System

    You can use one virtual machine to drive the updates for the rest of the virtual machines in the VM cluster. Then, you can use one of the updated nodes to drive the update on the original driving system. For example, consider updating a half rack system with four virtual machines; node1, node2, node3, and node4. You could first use node1 to drive the updates of node2, node3, and node4. Then, you could use node2 to drive the update of node1.

The driving system requires root user SSH access to each virtual machine being updated.

The following procedure is based on an example that assumes the following:

  • The system has two virtual machines, node1 and node2.
  • The target Exadata software version is 18.1.4.0.0.180125.3.
  • Each node is used as the driving system to update the other node.
  1. Gather the environment details.
    1. Using SSH, connect to node1 as the opc user.

      For detailed instructions, see Connecting to a Compute Node with SSH.

    2. Start a root user command shell.
      sudo su -
    3. Run the following command to determine the current Exadata software version.
      imageinfo -ver
      For example:
      # imageinfo -ver 19.2.13.0.0.200428
    4. Switch to the grid user, and identify all nodes in the cluster.
      su - grid
      olsnodes
      For example:
      olsnodes
      node1
      node2
  2. Configure the driving system.
    1. Switch back to the root user on node1 and check whether an SSH key pair (id_rsa and id_rsa.pub) exists. If not, then generate it.
      ls /root/.ssh/id_rsa*
      ls: cannot access /root/.ssh/id_rsa*: No such file or directory
      
      ssh-keygen -t rsa
      Generating public/private rsa key pair.
      Enter file in which to save the key (/root/.ssh/id_rsa):
      Enter passphrase (empty for no passphrase):
      Enter same passphrase again:
      Your identification has been saved in /root/.ssh/id_rsa.
      Your public key has been saved in /root/.ssh/id_rsa.pub.
      The key fingerprint is:
      93:47:b0:83:75:f2:3e:e6:23:b3:0a:06:ed:00:20:a5 root@node1.example.com
      The key's randomart image is:
      +--[ RSA 2048]----+
      |o..     + .      |
      |o.     o *       |
      |E     . o o      |
      | . .     =       |
      |  o .   S =      |
      |   +     = .     |
      |    +   o o      |
      |   . .   + .     |
      |      ...        |
      +-----------------+
    2. Distribute the public key to the target nodes, and verify this step. In the example, the only target node is node2.
      scp -i ~root/.ssh/id_rsa.pub opc@node2:/tmp/id_rsa.node1.pub
      ls -al /tmp/id_rsa.node1.pub
      -rw-r--r-- 1 opc opc 442 Feb 28 03:33 /tmp/id_rsa.node1.pub
      date
      Wed Feb 28 03:33:45 UTC 2018
    3. On the target node (node2 in the example), add the root public key of node1 to the root authorized_keys file.
      cat /tmp/id_rsa.node1.pub >> ~root/.ssh/authorized_keys
    4. Download patchmgr into /root/patch on the driving system (node1 in this example).

      You can download the patchmgr bundle from Oracle Support by using My Oracle Support Patch ID 21634633. Always obtain the latest available Exadata patchmgr update utility to install any Exadata System Software release.

      For further information, see also dbnodeupdate.sh and dbserver.patch.zip: Updating Exadata Database Server Software using the DBNodeUpdate Utility and patchmgr: My Oracle Support Doc ID 1553103.1.

    5. Unzip the patchmgr bundle.

      Depending on the version that you downloaded, the name of your ZIP file can differ.

      cd /root/patch/18.1.4.0.0.180125.3
      unzip dbserver.patch.zip
      Archive:  p21634633_181400_Linux-x86-64.zip   creating: dbserver_patch_5.180228.2/
      creating: dbserver_patch_5.180228.2/ibdiagtools/
      inflating: dbserver_patch_5.180228.2/ibdiagtools/cable_check.pl
      inflating: dbserver_patch_5.180228.2/ibdiagtools/setup-ssh
      inflating: dbserver_patch_5.180228.2/ibdiagtools/VERSION_FILE
      extracting: dbserver_patch_5.180228.2/ibdiagtools/xmonib.sh
      inflating: dbserver_patch_5.180228.2/ibdiagtools/monitord
      inflating: dbserver_patch_5.180228.2/ibdiagtools/checkbadlinks.pl
      creating: dbserver_patch_5.180228.2/ibdiagtools/topologies/
      inflating: dbserver_patch_5.180228.2/ibdiagtools/topologies/VerifyTopologyUtility.pm
      inflating: dbserver_patch_5.180228.2/ibdiagtools/topologies/verifylib.pm
      inflating: dbserver_patch_5.180228.2/ibdiagtools/topologies/Node.pm
      inflating: dbserver_patch_5.180228.2/ibdiagtools/topologies/Rack.pm
      inflating: dbserver_patch_5.180228.2/ibdiagtools/topologies/Group.pm
      inflating: dbserver_patch_5.180228.2/ibdiagtools/topologies/Switch.pm
      inflating: dbserver_patch_5.180228.2/ibdiagtools/topology-zfs
      inflating: dbserver_patch_5.180228.2/ibdiagtools/dcli
      creating: dbserver_patch_5.180228.2/ibdiagtools/netcheck/
      inflating: dbserver_patch_5.180228.2/ibdiagtools/netcheck/remoteScriptGenerator.pm
      inflating: dbserver_patch_5.180228.2/ibdiagtools/netcheck/CommonUtils.pm
      inflating: dbserver_patch_5.180228.2/ibdiagtools/netcheck/SolarisAdapter.pm
      inflating: dbserver_patch_5.180228.2/ibdiagtools/netcheck/LinuxAdapter.pm
      inflating: dbserver_patch_5.180228.2/ibdiagtools/netcheck/remoteLauncher.pm
      inflating: dbserver_patch_5.180228.2/ibdiagtools/netcheck/remoteConfig.pm
      inflating: dbserver_patch_5.180228.2/ibdiagtools/netcheck/spawnProc.pm
      inflating: dbserver_patch_5.180228.2/ibdiagtools/netcheck/runDiagnostics.pm
      inflating: dbserver_patch_5.180228.2/ibdiagtools/netcheck/OSAdapter.pm
      inflating: dbserver_patch_5.180228.2/ibdiagtools/SampleOutputs.txt
      inflating: dbserver_patch_5.180228.2/ibdiagtools/infinicheck
      inflating: dbserver_patch_5.180228.2/ibdiagtools/ibping_test
      inflating: dbserver_patch_5.180228.2/ibdiagtools/tar_ibdiagtools
      inflating: dbserver_patch_5.180228.2/ibdiagtools/verify-topology
      inflating: dbserver_patch_5.180228.2/installfw_exadata_ssh
      creating: dbserver_patch_5.180228.2/linux.db.rpms/
      inflating: dbserver_patch_5.180228.2/md5sum_files.lst
      inflating: dbserver_patch_5.180228.2/patchmgr
      inflating: dbserver_patch_5.180228.2/xcp
      inflating: dbserver_patch_5.180228.2/ExadataSendNotification.pm
      inflating: dbserver_patch_5.180228.2/ExadataImageNotification.pl
      inflating: dbserver_patch_5.180228.2/kernelupgrade_oldbios.sh
      inflating: dbserver_patch_5.180228.2/cellboot_usb_pci_path
      inflating: dbserver_patch_5.180228.2/exadata.img.env
      inflating: dbserver_patch_5.180228.2/README.txt
      inflating: dbserver_patch_5.180228.2/exadataLogger.pm
      inflating: dbserver_patch_5.180228.2/patch_bug_26678971
      inflating: dbserver_patch_5.180228.2/dcli
      inflating: dbserver_patch_5.180228.2/patchReport.py
      extracting: dbserver_patch_5.180228.2/dbnodeupdate.zip
      creating: dbserver_patch_5.180228.2/plugins/
      inflating: dbserver_patch_5.180228.2/plugins/010-check_17854520.sh
      inflating: dbserver_patch_5.180228.2/plugins/020-check_22468216.sh
      inflating: dbserver_patch_5.180228.2/plugins/040-check_22896791.sh
      inflating: dbserver_patch_5.180228.2/plugins/000-check_dummy_bash
      inflating: dbserver_patch_5.180228.2/plugins/050-check_22651315.sh
      inflating: dbserver_patch_5.180228.2/plugins/005-check_22909764.sh
      inflating: dbserver_patch_5.180228.2/plugins/000-check_dummy_perl
      inflating: dbserver_patch_5.180228.2/plugins/030-check_24625612.sh
      inflating: dbserver_patch_5.180228.2/patchmgr_functions
      inflating: dbserver_patch_5.180228.2/exadata.img.hw
      inflating: dbserver_patch_5.180228.2/libxcp.so.1
      inflating: dbserver_patch_5.180228.2/imageLogger
      inflating: dbserver_patch_5.180228.2/ExaXMLNode.pm
      inflating: dbserver_patch_5.180228.2/fwverify
    6. In the directory that contains the patchmgr utility, create the dbs_group file, which contains the list of virtual machines to update. Include the nodes listed after running the olsnodes command in step 1, except for the driving system. In this example, dbs_group only contains node2.
      cd /root/patch/18.1.4.0.0.180125.3/dbserver_patch_5.180228
      cat dbs_group
      node2
  3. Run a patching precheck operation.
    ./patchmgr -dbnodes dbs_group -precheck -iso_repo zipped_iso_file -target_version target-version -nomodify_at_prereq
    Note

    Run the precheck operation with the -nomodify_at_prereq option to prevent any changes to the system that could impact the backup you take in the next step. Otherwise, the backup might not be able to roll the system back to its original state, should it be necessary.

    The output should look similar to the following example:

    ./patchmgr -dbnodes dbs_group -precheck -iso_repo /root/patch/18.1.4.0.0.180125.3/exadata_ol6_18.1.4.0.0.180125.3_Linux-x86-64.zip -target_version 18.1.4.0.0.180125.3 -nomodify_at_prereq
     
    ************************************************************************************************************
    NOTE    patchmgr release: 5.180228 (always check MOS 1553103.1 for the latest release of dbserver.patch.zip)
    NOTE
    WARNING Do not interrupt the patchmgr session.
    WARNING Do not resize the screen. It may disturb the screen layout.
    WARNING Do not reboot database nodes during update or rollback.
    WARNING Do not open logfiles in write mode and do not try to alter them.
    ************************************************************************************************************
    2018-02-28 21:22:45 +0000        :Working: DO: Initiate precheck on 1 node(s)
    2018-02-28 21:24:57 +0000        :Working: DO: Check free space and verify SSH equivalence for the root user to node2
    2018-02-28 21:26:15 +0000        :SUCCESS: DONE: Check free space and verify SSH equivalence for the root user to node2
    2018-02-28 21:26:47 +0000        :Working: DO: dbnodeupdate.sh running a precheck on node(s).
    2018-02-28 21:28:23 +0000        :SUCCESS: DONE: Initiate precheck on node(s).
  4. Back up the current system.
    ./patchmgr -dbnodes dbs_group -backup -iso_repo zipped_iso_file -target_version target-version -allow_active_network_mounts
    Note

    Ensure that you take the backup at this point, before any modifications are made to the system.

    The output should look similar to the following example:

    ./patchmgr -dbnodes dbs_group -backup -iso_repo /root/patch/18.1.4.0.0.180125.3/exadata_ol6_18.1.4.0.0.180125.3_Linux-x86-64.zip -target_version 18.1.4.0.0.180125.3 -allow_active_network_mounts
     
    ************************************************************************************************************
    NOTE    patchmgr release: 5.180228 (always check MOS 1553103.1 for the latest release of dbserver.patch.zip)
    NOTE
    WARNING Do not interrupt the patchmgr session.
    WARNING Do not resize the screen. It may disturb the screen layout.
    WARNING Do not reboot database nodes during update or rollback.
    WARNING Do not open logfiles in write mode and do not try to alter them.
    ************************************************************************************************************
    2018-02-28 21:29:00 +0000        :Working: DO: Initiate backup on 1 node(s).
    2018-02-28 21:29:00 +0000        :Working: DO: Initiate backup on node(s)
    2018-02-28 21:29:01 +0000        :Working: DO: Check free space and verify SSH equivalence for the root user to node2
    2018-02-28 21:30:18 +0000        :SUCCESS: DONE: Check free space and verify SSH equivalence for the root user to node2
    2018-02-28 21:30:51 +0000        :Working: DO: dbnodeupdate.sh running a backup on node(s).
    2018-02-28 21:35:50 +0000        :SUCCESS: DONE: Initiate backup on node(s).
    2018-02-28 21:35:50 +0000        :SUCCESS: DONE: Initiate backup on 1 node(s).
  5. Remove all custom RPMs from the target virtual machines. Custom RPMs are reported in precheck results. They include RPMs that were manually installed after the system was provisioned.
    • If you are updating the system from version 12.1.2.3.4.170111, and the precheck results include krb5-workstation-1.10.3-57.el6.x86_64, then remove it. This item is considered a custom RPM for this version.
    • Do not remove exadata-sun-vm-computenode-exact or oracle-ofed-release-guest. These two RPMs are handled automatically during the update process.
  6. Perform the update. To ensure that the update process in not interrupted, use the command nohup. For example:
    nohup ./patchmgr -dbnodes dbs_group -upgrade -nobackup -iso_repo zipped_iso_file -target_version target-version -allow_active_network_mounts &

    The output should look similar to the following example:

    nohup ./patchmgr -dbnodes dbs_group -upgrade -nobackup -iso_repo /root/patch/18.1.4.0.0.180125.3/exadata_ol6_18.1.4.0.0.180125.3_Linux-x86-64.zip -target_version 18.1.4.0.0.180125.3 -allow_active_network_mounts &
     
    ************************************************************************************************************
    NOTE    patchmgr release: 5.180228 (always check MOS 1553103.1 for the latest release of dbserver.patch.zip)
    NOTE
    NOTE    Database nodes will reboot during the update process.
    NOTE
    WARNING Do not interrupt the patchmgr session.
    WARNING Do not resize the screen. It may disturb the screen layout.
    WARNING Do not reboot database nodes during update or rollback.
    WARNING Do not open logfiles in write mode and do not try to alter them.
    *********************************************************************************************************
    2018-02-28 21:36:26 +0000        :Working: DO: Initiate prepare steps on node(s).
    2018-02-28 21:36:26 +0000        :Working: DO: Check free space and verify SSH equivalence for the root user to node2
    2018-02-28 21:37:44 +0000        :SUCCESS: DONE: Check free space and verify SSH equivalence for the root user to node2
    2018-02-28 21:38:43 +0000        :SUCCESS: DONE: Initiate prepare steps on node(s).
    2018-02-28 21:38:43 +0000        :Working: DO: Initiate update on 1 node(s).
    2018-02-28 21:38:43 +0000        :Working: DO: Initiate update on node(s)
    2018-02-28 21:38:49 +0000        :Working: DO: Get information about any required OS upgrades from node(s).
    2018-02-28 21:38:59 +0000        :SUCCESS: DONE: Get information about any required OS upgrades from node(s).
    2018-02-28 21:38:59 +0000        :Working: DO: dbnodeupdate.sh running an update step on all nodes.
    2018-02-28 21:48:41 +0000        :INFO   : node2 is ready to reboot.
    2018-02-28 21:48:41 +0000        :SUCCESS: DONE: dbnodeupdate.sh running an update step on all nodes.
    2018-02-28 21:48:41 +0000        :Working: DO: Initiate reboot on node(s)
    2018-02-28 21:48:57 +0000        :SUCCESS: DONE: Initiate reboot on node(s)
    2018-02-28 21:48:57 +0000        :Working: DO: Waiting to ensure node2 is down before reboot.
    2018-02-28 21:56:18 +0000        :Working: DO: Initiate prepare steps on node(s).
    2018-02-28 21:56:19 +0000        :Working: DO: Check free space and verify SSH equivalence for the root user to node2
    2018-02-28 21:57:37 +0000        :SUCCESS: DONE: Check free space and verify SSH equivalence for the root user to node2
    2018-02-28 21:57:42 +0000        :SEEMS ALREADY UP TO DATE: node2
    2018-02-28 21:57:43 +0000        :SUCCESS: DONE: Initiate update on node(s)
  7. After the update operation completes, verify the version of the Exadata software on the virtual machine that was updated.
    imageinfo -ver
    18.1.4.0.0.180125.3
  8. Repeat steps 2 through 7 of this procedure using the updated virtual machine as the driving system to update the remaining virtual machine. In this example update, you would now use node2 to update node1.
  9. As root On each virtual machine, run the uptrack-install command to install the available ksplice updates.
    uptrack-install --all -y
    uptrack-install --all -y
Installing Additional Operating System Packages

Review these guidelines before you install additional operating system packages for Oracle Exadata Database Service on Cloud@Customer.

You are permitted to install and update operating system packages on Oracle Exadata Database Service on Cloud@Customer as long as you do not modify the kernel or InfiniBand-specific packages. However, Oracle technical support, including installation, testing, certification and error resolution, does not apply to any non-Oracle software that you install.

Also be aware that if you add or update packages separate from an Oracle Exadata software update, then these package additions or updates can introduce problems when you apply an Oracle Exadata software update. Problems can occur because additional software packages add new dependencies that can interrupt an Oracle Exadata update. For this reason, Oracle recommends that you minimize customization.

If you install additional packages, then Oracle recommends that you have scripts to automate the removal and reinstallation of those packages. After an Oracle Exadata update, if you install additional packages, then verify that the additional packages are still compatible, and that you still need these packages.

For more information, refer to Oracle Exadata Database Machine Maintenance Guide.