Scenario A: Automatically Resize VMs

This topic explains how to set up automatic resizing for virtual machines (VMs) that exceed memory.

Caution

Avoid entering confidential information when assigning descriptions, tags, or friendly names to your cloud resources through the Oracle Cloud Infrastructure Console, API, or CLI.

This scenario involves writing a function to resize VMs and creating an alarm  that sends a message to that function. When the alarm fires, the Notifications service sends the alarm message to the destination topic, which then fans out to the topic's subscriptions. In this scenario, the topic's subscriptions include the function as well as your email. The function is invoked on receipt of the alarm message.

Note

The Notifications service has no information about a function after it's invoked. For more details, see the troubleshooting information at Function not invoked or run.

This image shows Notifications in the context of a scenario that uses a function to resize VMs.

Required IAM Policy

To use Oracle Cloud Infrastructure, you must be granted security access in a policy  by an administrator. This access is required whether you're using the Console or the REST API with an SDK, CLI, or other tool. If you get a message that you don’t have permission or are unauthorized, verify with your administrator what type of access you have and which compartment  you should work in.

If you're a member of the Administrators group, you already have the required access to execute this scenario. Otherwise, you need access to Monitoring, Notifications, and Functions. You must have FN_INVOCATION permission against the function to be able to add the function as a subscription to a topic. To resize VMs, the function must be authorized to update Compute instances. To authorize your function for access to other Oracle Cloud Infrastructure resources, such as Compute instances, include the function in a dynamic group  and create a policy to grant the dynamic group access to those resources. For more information, see Accessing Other Oracle Cloud Infrastructure Resources from Running Functions.

Setting Up This Scenario

Once you create your function to resize VMs using your preferred SDK and authorize your function to access VMs (include the function in a dynamic group and grant that dynamic group access), setup is easy in the Console. Alternatively, you can use the Oracle Cloud Infrastructure CLI or API, which lets you execute the individual operations yourself.

For more information about authorizing functions to access other Oracle Cloud Infrastructure resources, see Accessing Other Oracle Cloud Infrastructure Resources from Running Functions.

Function code sample
Note

For this code sample, we recommend handling idempotency via a database.

The following code sample is for a function to resize VMs. For instructions on creating and deploying functions, see Creating and Deploying Functions.

import io
import json
import oci

from fdk import response

def increase_compute_shape(instance_id, alarm_msg_shape):
    signer = oci.auth.signers.get_resource_principals_signer()
    compute_client = oci.core.ComputeClient(config={}, signer=signer)
    current_shape = compute_client.get_instance(instance_id).data.shape
    print("INFO: current shape for Instance {0}: {1}".format(instance_id,current_shape), flush=True)
    if current_shape != alarm_msg_shape:
        return "The shape of Instance {} differs from the Alarm message".format(instance_id)
    # improve the logic below to handle more scenarios, make sure the shapes you select are available in the region and AD
    if  current_shape == "VM.Standard1.1":
        new_shape = "VM.Standard2.1"
    elif current_shape == "VM.Standard2.1":
        new_shape = "VM.Standard2.2"
    else:
        return "Instance {0} cannot get a bigger shape than its current shape {1}".format(instance_id,current_shape)
    print("INFO: new shape for Instance {0}: {1}".format(instance_id,new_shape), flush=True)
    try:
        update_instance_details = oci.core.models.UpdateInstanceDetails(shape=new_shape)
        resp = compute_client.update_instance(instance_id=instance_id, update_instance_details=update_instance_details)
        print(resp, flush=True)
    except Exception as ex:
        print('ERROR: cannot update instance {}'.format(instance_id), flush=True)
        raise
    return "The shape of Instance {} is updated, the instance is rebooting...".format(instance_id)

def handler(ctx, data: io.BytesIO=None):
    alarm_msg = {}
    message_id = func_response = ""
    try:
        headers = ctx.Headers()
        message_id = headers["x-oci-ns-messageid"]
    except Exception as ex:
        print('ERROR: Missing Message ID in the header', ex, flush=True)
        raise
    print("INFO: Message ID = ", message_id, flush=True)
    # the Message Id can be stored in a database and be used to check for duplicate messages
    try:
        alarm_msg = json.loads(data.getvalue())
        print("INFO: Alarm message: ")
        print(alarm_msg, flush=True)
    except (Exception, ValueError) as ex:
        print(str(ex), flush=True)

    if alarm_msg["type"] == "OK_TO_FIRING":
        if alarm_msg["alarmMetaData"][0]["dimensions"]:
            alarm_metric_dimension = alarm_msg["alarmMetaData"][0]["dimensions"][0]   #assuming the first dimension matches the instance to resize
            print("INFO: Instance to resize: ", alarm_metric_dimension["resourceId"], flush=True)
            func_response = increase_compute_shape(alarm_metric_dimension["resourceId"], alarm_metric_dimension["shape"])
            print("INFO: ", func_response, flush=True)
        else:
            print('ERROR: There is no metric dimension in this alarm message', flush=True)
            func_response = "There is no metric dimension in this alarm message"
    else:
        print('INFO: Nothing to do, alarm is not FIRING', flush=True)
        func_response = "Nothing to do, alarm is not FIRING"

    return response.Response(
        ctx, 
        response_data=func_response,
        headers={"Content-Type": "application/json"}
    )
Include your function in a dynamic group

Find and note your function OCID  (format is ocid1.fnfunc.oc1.iad.exampleuniqueID), then specify the following rule in the relevant dynamic group :

resource.id = '<function-ocid>'
Create a policy to grant the dynamic group access to VMs (Compute instances)

Add the following policy  :

allow dynamic-group <dynamic-group-name> to use instances in tenancy

Using the Console

This section walks through creating an alarm using the Console and then updating the topic created with the alarm. Your function must be deployed.

Note

Another workflow for this scenario involves creating your topic and subscriptions first, then selecting this topic when you create your alarm.

For help with troubleshooting, see Troubleshooting Notifications.

Task 1: Create the alarm

This example walks through using the Console to create an alarm that sends a message to run the function when high memory usage is detected. During this process, you'll create a topic that references your function. You can add the email subscription later.

  1. Open the navigation menu. Under Solutions and Platform, go to Monitoring and click Alarm Definitions.
  2. Click Create alarm.

  3. On the Create Alarm page, under Define alarm, set up your threshold: 

    • Metric description
      • Compartment: (select the compartment  that contains your VM)
      • Metric Namespaceoci_computeagent
      • Metric NameMemoryUtilization
      • Interval1m
      • StatisticMax
    • Trigger rule:

      • Operatorgreater than
      • Value: 90
      • Trigger Delay Minutes: 1
  4. Select your function under Notifications, Destinations:
    • Destination ServiceNotifications Service
    • Compartment: (select the compartment  where you want to create the topic and associated subscriptions)
    • Topic: Click Create a topic

      Note

      If you already created your topic and function subscripton, you can select that topic here instead of creating a new one.
      • Topic Name: Alarm Topic
      • Subscription Protocol: Function
      • Function Compartment: (select the compartment  that contains your function)
      • Function Application: (select the application that contains your function)
      • Function: (select your function)
  5. Click Save alarm.

Task 2: Add an email subscription (optional)

This example walks through using the Console to add an optional email subscription to the topic you created when you set up the alarm.

  1. Open the navigation menu. Under Solutions and Platform, go to Application Integration and click Notifications.
  2. Click the name of the topic that you want to add the subscription to.

    Example: "Alarm Topic" (assuming you used the suggested topic name when creating the topic in the alarm).

  3. On the topic detail page, click Create Subscription.
  4. In the Create Subscription dialog box, set up your email subscription:

    • Protocol: Select Email.
    • Email: Type an email address.
  5. Click Create.

    The email subscription has been created and a subscription confirmation URL is sent to the specified email address. The subscription remains in "Pending" status until it has been confirmed.

  6. To confirm your new email subscription, open your email and navigate to the confirmation URL.

Using the CLI

This section walks through creating the topic, subscriptions, and alarm using the CLI. Your function must be deployed.

For information about using the API and signing requests, see REST APIs and Security Credentials. For information about SDKs, see Software Development Kits and Command Line Interface.
  1. Create a topic: Open a command prompt and run the oci ons topic create command: 

    oci ons topic create --name "Alarm Topic" --compartment-id "<compartment-ocid>"
  2. To this topic, add subscriptions referencing your function OCID and an optional email address.
    • Create a function subscription: Open a command prompt and run the oci ons subscription create command:

      oci ons subscription create --compartment-id "<compartment-ocid>" --topic-id "<topic-ocid>" --protocol "ORACLE_FUNCTIONS" --subscription-endpoint "<function-ocid>"
    • Create an email subscription: Open a command prompt and run the oci ons subscription create command:

      oci ons subscription create --compartment-id "<compartment-ocid>" --topic-id "<topic-ocid>" --protocol "EMAIL" --subscription-endpoint "john.smith@example.com"
  3. Create an alarm that defines the memory threshold and references this topic as the destination: Open a command prompt and run the oci monitoring alarm create command: 

    oci monitoring alarm create --display-name "VM Memory Alarm" --compartment-id "<compartment-ocid>" --metric-compartment-id "<compartment-ocid>" --namespace "oci_computeagent" --query-text "MemoryUtilization[1m].max() > 90" --severity "CRITICAL" --destinations "<topic-ocid>" --is-enabled true

For help with troubleshooting, see Troubleshooting Notifications.

Using the API

This section walks through creating the topic, subscriptions, and alarm using the API. Your function must be deployed.

For information about using the API and signing requests, see REST APIs and Security Credentials. For information about SDKs, see Software Development Kits and Command Line Interface.

Use the following operations:

  1. CreateTopic: Create a topic.

    Example CreateTopic request
    POST /20181201/topics
                                Host: cp.notification.us-phoenix-1.oraclecloud.com
                                <authorization and other headers>
                                {
                                "name": "Alarm Topic",
                                "compartmentId": "<compartment_OCID>"
                                }
  2. CreateSubscription: To this topic, add subscriptions referencing your function OCID and an optional email address.

    Example CreateSubscription request: Function
    POST /20181201/subscriptions
                                Host: cp.notification.us-phoenix-1.oraclecloud.com
                                <authorization and other headers>
                                {
                                "topicId": "<topic_OCID>",
                                "compartmentId": "<compartment_OCID>",
                                "protocol": "ORACLE_FUNCTIONS",
                                "endpoint": "<function_OCID>"
                                }
    Example CreateSubscription request: Email
    POST /20181201/subscriptions
                                Host: cp.notification.us-phoenix-1.oraclecloud.com
                                <authorization and other headers>
                                {
                                "topicId": "<topic_OCID>",
                                "compartmentId": "<compartment_OCID>",
                                "protocol": "EMAIL",
                                "endpoint": "john.smith@example.com"
                                }
  3. CreateAlarm: Create an alarm that defines the memory threshold and references this topic.

    Example CreateAlarm request
    POST /20180401/alarms
                                Host: telemetry.us-phoenix-1.oraclecloud.com
                                <authorization and other headers>
                                {
                                "displayName": "VM Memory Alarm",
                                "compartmentId": "<compartment_OCID>",
                                "metricCompartmentId": "<compartment_OCID>",
                                "namespace": "oci_computeagent",
                                "query": "MemoryUtilization[1m].max() > 90",
                                "severity": "CRITICAL",
                                "destinations":
                                [
                                "<topic_OCID>"
                                ],
                                "isEnabled": true
                                }

For help with troubleshooting, see Troubleshooting Notifications.