View Cluster Metrics

You can monitor the health, capacity, and performance of your Oracle Big Data Service resources by using metrics, alarms, and notifications.

Resources: Big Data Service clusters and cluster nodes.

Required IAM Policy

To monitor resources, you must be given the required type of access in a policy written by an administrator, whether you're using the Cloud console or the REST API with an SDK, CLI, or other tool. The policy must give you access to the monitoring services as well as the resources being monitored. If you try to perform an action and get a message that you don’t have permission or are unauthorized, confirm with your administrator the type of access you've been granted and which compartment you should work in. For more information on user authorizations for monitoring, see the Authentication and Authorization section for the related service: Monitoring or Notifications.

Available Metrics: oci_big_data_service

The metrics listed in the following table are automatically available for any Big Data Service cluster that you create. You don't need to enable monitoring on the resource to get these metrics.

Big Data Service metrics include the following dimensions :

RESOURCEID

The Oracle Cloud ID (OCID) of the Big Data Service cluster (for cluster metrics)

The Oracle Cloud ID (OCID) of the Big Data Service node (for node metrics)

RESOURCETYPE

BigDataCluster (for cluster metrics)

BigDataClusterNode (for node metrics)

Metric Metric Display Name Unit Description Resource Type
HdfsSpaceUsed HDFS Space Used Bytes Total HDFS space used on the cluster Cluster
HdfsSpaceFree HDFS Space Free Bytes Total free HDFS space on the cluster Cluster
YarnJobsCompleted Yarn Jobs Completed Jobs/Min Number of YARN jobs completed on this cluster Cluster
SparkJobsCompleted Spark Jobs Completed Jobs/Min Number of Spark jobs completed on this cluster Cluster
DiskUtilization Disk Utilization Bytes Disk space utilized Node
MemoryUtilization Memory Utilization Bytes Total memory used Node
NetworkBytesIn Network Bytes In Bytes/Min Network bytes in per minute Node
NetworkBytesOut Network Bytes Out Bytes/Min Network bytes out per minute Node

View Default Metric Charts in the Console

Default metric charts use predefined service queries. You can select resources of interest and update the interval, statistic, and time range.

To view default metric charts for a cluster:

  1. Open the navigation menu. Under Data and AI, click Big Data.
  2. Choose the Compartment that contains the cluster you want to view, and click the name of the cluster.
  3. Under Resources, click Cluster Metrics. The Cluster Metrics page displays a default set of charts for the selected cluster.

To view default metric charts for a cluster node:

  1. Open the navigation menu. Under Data and AI, click Big Data.
  2. Choose the Compartment that contains the cluster with the node you want to view, click the name of the cluster, and then click the name of node.
  3. Under Resources, click Node Metrics. The Node Metrics page displays a default set of charts for the selected node.

To view default metric charts for all clusters in a compartment:

  1. Open the navigation menu. Under Solutions and Platform, go to Monitoring and click Service Metrics.
  2. For Compartment, select the compartment that contains the clusters that you're interested in.
  3. For Metric Namespace, select oci_big_data_service. The Service Metrics page dynamically updates the page to show charts for each metric that is emitted by the selected metric namespace.

Optionally, you can specify other dimensions to filter your displayed metrics. For more information, see To filter results and To select different resources in the Oracle Cloud Infrastructure documentation.

For more information about monitoring and notifications in Oracle Cloud Infrastructure, see Monitoring Overview and Notifications Overview, in the Oracle Cloud Infrastructure documentation.

Create Metric Queries in the Console

To create a query for a customized view of your metrics:

  1. Open the navigation menu. Under Solutions and Platform, go to Monitoring and click Metrics Explorer.

    The Metrics Explorer page displays an empty chart with fields to build a query.

  2. Fill in the fields for a new query.

    For more information, see Building Metric Queries in the Oracle Cloud Infrastructure documentation.

View Metrics By Using the API

For information about using the API and signing requests, see REST APIs and Security Credentials in the Oracle Cloud Infrastructure documentation. For information about SDKs, see Software Development Kits and Command Line Interface.

Use the following APIs for monitoring: