Managing Stream Pools

This topic describes how to work with stream pools.

Required IAM Policy

To use Oracle Cloud Infrastructure, you must be given the required type of access in a policy  written by an administrator, whether you're using the Console or the REST API with an SDK, CLI, or other tool. If you try to perform an action and get a message that you don’t have permission or are unauthorized, confirm with your administrator the type of access you've been granted and which compartment  you should work in.

For administrators: The policy in Let streaming users manage streams lets the specified group do everything with streaming and related Streaming service resources.

To set up and use a stream archiver, you must have read access to the stream and write access to the Object Storage. For example:

allow service stream-processing to use stream-pull in tenancy
allow service stream-processing to manage objects in tenancy

To set up a private endpoint, you must have access to a VCN with a private subnet where DNS resolution is enabled. For general information about policies and permissions to do this, see IAM Policies for Networking. Specifically, you need use permissions for a VNIC, a network security group, if you specify one, and a subnet. For example:

allow user group ServiceWriters to use vnics in compartment ABC
allow user group ServiceWriters to use network-security-groups in compartment ABC
allow user group ServiceWriters to use subnets in compartment XYZ

To use your own encryption key, you must let the Streaming service use a Vault key to encrypt data in streams in this stream pool. For example:

allow service streaming to use keys in compartment ABC where target.key.id = '<key_OCID>'

The preceding policy also requires a companion policy to let Streaming use a key on behalf of a user group to create a stream pool that uses the key for cryptographic purposes. For example:

allow user group StreamWriters to use key-delegate in compartment ABC where target.key.id = '<key_OCID>'

If you're new to policies, see Getting Started with Policies and Common Policies. If you want to dig deeper into writing policies for the Streaming service, see Details for the Streaming service in the IAM policy reference.

Using the Console

To create a stream pool
  1. Open the navigation menu. Under Solutions and Platform, click Analytics, and then click Streaming.
  2. Click on Stream Pools on the left side of the screen. This will display the list of stream pools.
  3. Click Create Stream Pool to display the Create Stream Pool page.
  4. Enter a name for the stream pool in the Name text box.
  5. Add a description for the stream pool in the Description text box.
  6. To use the stream pool with Kafka, click the Auto-Create Topics checkbox and configure your stream settings:
    1. Add a number of hours for the retention period in Retention Period (hours) text box.
    2. Specify the Number of Partitions Per Topic.
  7. Select either a Public Endpoint or a Private Endpoint, depending on whether you want to restrict traffic to streams in this stream pool to a private endpoint that does not require traffic to traverse the internet. To create a private endpoint, you need access to a virtual cloud network (VCN) with a private subnet. Select a VCN with a private subnet where DNS resolution is also enabled, and then select the subnet. Optionally, if you want to assign a specific private IP address, you must choose one that belongs to the subnet's CIDR. By default, the Networking service assigns a random private IP address on your behalf and applies no security rules to the stream pool. For more information about VCNs and subnets, see VCNs and Subnets.
  8. Encryption Settings: Optionally, you can encrypt the data in the streams in this stream pool by using your own Vault encryption key. By default, Oracle handles encryption. To use the Vault service for your encryption needs, you need access to a vault and key and you need to allow the Streaming service to use the key. Then, select the Encrypt using customer-managed keys radio button. Click Vault to select the vault that contains the master encryption key you want to use, and then click Master Encryption Key and select a master encryption key. For more information about encryption with a Vault key that you manage, see Overview of Vault and Managing Keys.
  9. TagsOptionally, you can apply tags. If you have permissions to create a resource, you also have permissions to apply free-form tags to that resource. To apply a defined tag, you must have permissions to use the tag namespace. For more information about tagging, see Resource Tags. If you are not sure if you should apply tags, skip this option (you can apply tags later) or ask your administrator.
  10. Click Create Stream Group.
To edit a stream pool
  1. Open the navigation menu. Under Solutions and Platform, click Analytics, and then click Streaming.

  2. Click on Stream Pools on the left side of the screen. This will display the list of stream pools.
  3. Click the stream pool you want to delete to bring up the stream pool details page.
  4. Click Edit Settings.
  5. Click Save or Cancel when finished.
To delete a stream pool
  1. Open the navigation menu. Under Solutions and Platform, click Analytics, and then click Streaming.

  2. Click on Stream Pools on the left side of the screen. This will display the list of stream pools.
  3. Click the stream pool you want to delete to bring up the stream pool details page.
  4. Click Delete Stream Pool.
  5. Confirm when prompted.
To change or remove the master encryption key assigned to an existing stream pool
  1. Open the navigation menu. Under Solutions and Platform, click Analytics, and then click Streaming.

  2. Click Stream Pools.
  3. Click a stream pool to display the stream details page.
  4. In Stream Pool Information, next to Encryption Key, do one of the following:
    • To stop using an Oracle-managed key in favor of a Vault master encryption key that you manage, click Assign, select a vault and encryption key you have access to, and then click Assign.
    • To select a different Vault master encryption key that you manage, click Edit, select a vault and encryption key you have access to, and then click Update.
    • Click Unassign to remove the assigned Vault master encryption key and let Oracle manage the encryption key, and then click Unassign again to confirm the removal of the existing key assignment.
To move a stream pool to a different compartment
  1. Open the navigation menu. Under Solutions and Platform, click Analytics, and then click Streaming.

  2. Click on Stream Pools on the left side of the screen. This will display the list of stream pools.
  3. Click a stream pool to display the stream details page.
  4. Find the stream pool you want to move in the list, click the the Actions icon (three dots), and then click Move Resource.
  5. Choose the destination compartment from the list.
  6. Click Move Resource.