Oracle Cloud Infrastructure Documentation

Listing Streams and Stream Pools

This information describes how to list streams and stream pools.

Required IAM Policy

To use Oracle Cloud Infrastructure, you must be granted security access in a policy  by an administrator. This access is required whether you're using the Console or the REST API with an SDK, CLI, or other tool. If you get a message that you don’t have permission or are unauthorized, verify with your administrator what type of access you have and which compartment  to work in.

For administrators: The policy in Let streaming admins manage streaming resources lets the specified group do everything with streaming and related Streaming service resources.

If you're new to policies, see Getting Started with Policies and Common Policies. If you want to dig deeper into writing policies for the Streaming service, see Details for the Streaming service in the IAM policy reference and Accessing Streaming Resources Across Tenancies.

  • To Get a List of Streams

    • Open the navigation menu and click Analytics & AI. Under Messaging, click Streaming.

      A list of existing streams is displayed.

    To View Stream Details

    Stream details include the Messages endpoint and the stream OCID.

    1. Open the navigation menu and click Analytics & AI. Under Messaging, click Streaming.

      A list of existing streams is displayed.

    2. Click a stream to display the stream details page.

    To Get a List of Stream Pools

    1. Open the navigation menu and click Analytics & AI. Under Messaging, click Streaming.
    2. Click on Stream Pools on the left side of the screen.

      A list of existing stream pools is displayed.

    To View Stream Pool Details

    Stream pool details include encryption and Kafka Connect settings.

    1. Open the navigation menu and click Analytics & AI. Under Messaging, click Streaming.
    2. Click on Stream Pools on the left side of the screen.

      A list of existing stream pools is displayed.

    3. Click a stream pool to display the stream pool details page.
    4. Click on Kafka Connection Settings on the left side of the screen.

      The Kafka Connect configuration values are displayed.

  • For information about using the CLI, see Command Line Interface (CLI). For a complete list of flags and options available for CLI commands, see the Command Line Reference.

    Note

    The examples in this section use the full syntax for all parameters, for example --compartment-id. For some parameters, there are shortened versions that you can use instead, like -c. See the CLI online help for instances of a shortened parameter associated with a command.
    Tip

    Provide input for --custom-encryption-key-details, --private-endpoint-details, and --kafka-settings as valid formatted JSON. See Passing Complex Input and Using a JSON File for Complex Input for information about JSON formatting.

    To get a list of streams in a compartment

    oci streaming admin stream list --compartment-id <compartment_OCID>

    For example:

    oci streaming admin stream list --compartment-id ocid1.tenancy.oc1..exampleuniqueID
{
  "data": [
    {
      "compartment-id": "ocid1.tenancy.oc1..exampleuniqueID",
      "defined-tags": {},
      "freeform-tags": {},
      "id": "ocid1.stream.oc1.phx.exampleuniqueID",
      "lifecycle-state": "ACTIVE",
      "messages-endpoint": "https://cell-1.streaming.us-phoenix-1.oci.oraclecloud.com",
      "name": "example_stream_2",
      "partitions": 1,
      "stream-pool-id": "ocid1.streampool.oc1.phx.exampleuniqueID",
      "time-created": "2020-08-21T21:19:35.707000+00:00"
    },
    {
      "compartment-id": "ocid1.tenancy.oc1..exampleuniqueID",
      "defined-tags": {},
      "freeform-tags": {},
      "id": "ocid1.stream.oc1.phx.exampleuniqueID",
      "lifecycle-state": "DELETED",
      "messages-endpoint": "https://cell-1.streaming.us-phoenix-1.oci.oraclecloud.com",
      "name": "example_stream_1",
      "partitions": 5,
      "stream-pool-id": "ocid1.streampool.oc1.phx.exampleuniqueID",
      "time-created": "2020-07-16T20:59:32.904000+00:00"
    }
  ]
}

    By default, getting a list of streams returns up to the first 10 streams in the compartment.

    To get a list of streams in a stream pool

    oci streaming admin stream list --stream-pool-id <stream_pool_OCID>

    For example:

    oci streaming admin stream list --stream-pool-id ocid1.streampool.oc1.phx.exampleuniqueID
{
  "data": [
    {
      "compartment-id": "ocid1.tenancy.oc1..exampleuniqueID",
      "defined-tags": {},
      "freeform-tags": {},
      "id": "ocid1.stream.oc1.phx.exampleuniqueID",
      "lifecycle-state": "ACTIVE",
      "messages-endpoint": "https://cell-1.streaming.us-phoenix-1.oci.oraclecloud.com",
      "name": "example_stream_2",
      "partitions": 1,
      "stream-pool-id": "ocid1.streampool.oc1.phx.exampleuniqueID",
      "time-created": "2020-08-21T21:19:35.707000+00:00"
    },
    {
      "compartment-id": "ocid1.tenancy.oc1..exampleuniqueID",
      "defined-tags": {},
      "freeform-tags": {},
      "id": "ocid1.stream.oc1.phx.exampleuniqueID",
      "lifecycle-state": "DELETED",
      "messages-endpoint": "https://cell-1.streaming.us-phoenix-1.oci.oraclecloud.com",
      "name": "example_stream_1",
      "partitions": 5,
      "stream-pool-id": "ocid1.streampool.oc1.phx.exampleuniqueID",
      "time-created": "2020-07-16T20:59:32.904000+00:00"
    }
  ]
}

    By default, getting a list of streams returns up to the first 10 streams in the stream pool.

    To get a list of stream pools

    oci streaming admin stream-pool list --compartment-id <compartment_OCID>

    For example:

    oci streaming admin stream-pool list --compartment-id ocid1.tenancy.oc1..exampleuniqueID
{
  "data": [
    {
      "compartment-id": "ocid1.tenancy.oc1..exampleuniqueID",
      "defined-tags": {},
      "freeform-tags": {},
      "id": "ocid1.streampool.oc1.phx.exampleuniqueID",
      "is-private": false,
      "lifecycle-state": "ACTIVE",
      "name": "MyStreamPool",
      "time-created": "2020-10-09T22:21:16.931000+00:00"
    },
    {
      "compartment-id": "ocid1.tenancy.oc1..exampleuniqueID",
      "defined-tags": {},
      "freeform-tags": {},
      "id": "ocid1.streampool.oc1.phx.exampleuniqueID",
      "is-private": true,
      "lifecycle-state": "ACTIVE",
      "name": "example-stream-pool-02",
      "time-created": "2020-04-03T07:00:56.196000+00:00"
    },
    {
      "compartment-id": "ocid1.tenancy.oc1..exampleCompartmentOCID",
      "defined-tags": {},
      "freeform-tags": {},
      "id": "ocid1.streampool.oc1.phx.exampleStreamPoolOCID",
      "is-private": false,
      "lifecycle-state": "ACTIVE",
      "name": "example-stream-pool-01,
      "time-created": "2020-04-03T05:28:26.025000+00:00"
    }
  ]
}

    By default, getting a list of streams returns up to the first 10 streams in the compartment.

    To view stream details

    oci streaming admin stream get --stream-id <stream_OCID>

    For example:

    oci streaming admin stream get --stream-id ocid1.stream.oc1.phx.exampleuniqueID
{
  "data": {
    "compartment-id": "ocid1.tenancy.oc1..exampleuniqueID",
    "defined-tags": {},
    "freeform-tags": {},
    "id": "ocid1.stream.oc1.phx.exampleuniqueID",
    "lifecycle-state": "ACTIVE",
    "lifecycle-state-details": null,
    "messages-endpoint": "https://cell-1.streaming.us-phoenix-1.oci.oraclecloud.com",
    "name": "MyStream",
    "partitions": 5,
    "retention-in-hours": 24,
    "stream-pool-id": "ocid1.streampool.oc1.phx.exampleuniqueID",
    "time-created": "2020-11-02T19:12:22.385000+00:00"
  },
  "etag": "\"0613d634-86ab-4446-973f-268d175313d4-12e9725e-5574-4f6b-995b-7dcc80271666\""
}

    To view stream pool details

    oci streaming admin stream-pool get --stream-pool-id <stream_pool_OCID>

    For example:

    oci streaming admin stream-pool get --stream-pool-id ocid1.streampool.oc1.phx.exampleuniqueID
{
  "data": {
    "compartment-id": "ocid1.tenancy.oc1..exampleuniqueID",
    "custom-encryption-key": {
      "key-state": "NONE",
      "kms-key-id": null
    },
    "defined-tags": {},
    "endpoint-fqdn": "cell-1.streaming.us-phoenix-1.oci.oraclecloud.com",
    "freeform-tags": {},
    "id": "ocid1.streampool.oc1.phx.exampleuniqueID",
    "is-private": false,
    "kafka-settings": {
      "auto-create-topics-enable": false,
      "bootstrap-servers": "cell-1.streaming.us-phoenix-1.oci.oraclecloud.com:9092",
      "log-retention-hours": 24,
      "num-partitions": 1
    },
    "lifecycle-state": "ACTIVE",
    "lifecycle-state-details": null,
    "name": "MyStreamPool",
    "private-endpoint-settings": {
      "nsg-ids": null,
      "private-endpoint-ip": null,
      "subnet-id": null
    },
    "time-created": "2020-11-02T23:01:59.429000+00:00"
  },
  "etag": "\"6934531c-efaa-40ba-b083-94eb2350d737-a8b10bda-09cc-45e1-800b-b66b4bc29353\""
}

  • Use the following API operations to list streams and stream pools and retrieve details:

    For information about using the API and signing requests, see REST API documentation and Security Credentials. For information about SDKs, see SDKs and the CLI.

Using OCI SDKs

Use the listStreams method to return a list of streams for a given compartment or stream pool. To get details about a stream, use the getStream method and then examine the properties of the stream.

See the Developer Guide to Streaming for detailed SDK examples.

Using Resource Manager

Resource Manager is an Oracle Cloud Infrastructure (OCI) service that allows you to automate the process of provisioning your OCI resources. Using Terraform, Resource Manager helps you install, configure, and manage resources through the "infrastructure-as-code" model.

A Terraform configuration codifies your infrastructure in declarative configuration files. The configuration defines the resources you intend to provision, variables, and specific instructions for provisioning the resources

You can use Resource Manager or the Terraform CLI with the OCI Terraform provider to see how your streams and stream pools are represented in Terraform configuration files.

For more information about writing configurations for use with Resource Manager, see Terraform Configurations for Resource Manager and Terraform Configuration.

Streams

If you're using Resource Manager to manage streams in a stack, the streams are represented as oci_streaming_stream resources.

For example:

resource oci_streaming_stream export_example_stream {
  compartment_id = var.compartment_ocid
  defined_tags = {
  }
  freeform_tags = {
  }
  name               = "example_stream"
  partitions         = "1"
  retention_in_hours = "24"
  #stream_pool_id = <<Optional value not found in discovery>>
}

Listing Streams

You can use the oci_streaming_streams data source to retrieve the streams in your compartment if you're using the Terraform provider.

For example:

data "oci_streaming_streams" "test_streams" {

    #Optional
    compartment_id = var.compartment_id
    id = var.stream_id
    name = var.stream_name
    state = var.stream_state
    stream_pool_id = oci_streaming_stream_pool.test_stream_pool.id
}

Stream Pools

If you're using Resource Manager to manage streams in a stack, the streams are represented as oci_streaming_stream_pool resources.

For example:

resource oci_streaming_stream_pool export_test_stream_pool {
  compartment_id = var.compartment_ocid
  name           = "test_stream_pool"

  #Optional
  private_endpoint_settings {
    nsg_ids             = [oci_core_network_security_group.test_nsg.id]
    private_endpoint_ip = "10.0.0.5"
    subnet_id           = oci_core_subnet.test_subnet.id
  }

  kafka_settings {
    #Optional
    auto_create_topics_enable = true
    log_retention_hours       = 24
    num_partitions            = 1
  }
}

resource "oci_core_vcn" "test_vcn" {
  cidr_block     = "10.0.0.0/16"
  compartment_id = var.compartment_ocid
  display_name   = "testvcn"
  dns_label      = "dnslabel"
}

resource "oci_core_subnet" "test_subnet" {
  cidr_block     = "10.0.0.0/24"
  compartment_id = var.compartment_ocid
  vcn_id         = oci_core_vcn.test_vcn.id
}

resource "oci_core_network_security_group" "test_nsg" {
  compartment_id = var.compartment_ocid
  vcn_id         = oci_core_vcn.test_vcn.id
}

Stream Details

Terraform provider users can use the oci_streaming_stream data source to retrieve the details of a stream.

For example:

data "oci_streaming_stream" "test_stream" {
  #Required
  stream_id = oci_streaming_stream.stream.id
}

Stream Pool Details

Terraform provider users can use the oci_streaming_stream_pool data source to retrieve the details of a stream pool.

For example:

data "oci_streaming_stream_pool" "test_stream_pool" {
    #Required
    stream_pool_id = oci_streaming_stream_pool.test_stream_pool.id
}