Oracle Cloud Infrastructure Documentation

Example Network Resource Configurations

Before you can use Container Engine for Kubernetes to create and deploy clusters in the regions in a tenancy:

  • The tenancy's root compartment must include a policy to allow Container Engine for Kubernetes to perform operations in the tenancy. See Create Required Policy for Container Engine for Kubernetes.
  • Within the tenancy, there must already be a compartment to contain the necessary network resources (such as a VCN, subnets, internet gateway, route table, security lists). If such a compartment does not exist already, you will have to create it. Note that the network resources can reside in the root compartment. However, if you expect multiple teams to create clusters, best practice is to create a separate compartment for each team.
  • Within the compartment, network resources (such as a VCN, subnets, internet gateway, route table, security lists) must be appropriately configured in each region in which you want to create and deploy clusters. When creating a new cluster, you can have Container Engine for Kubernetes automatically create and configure new network resources for a new 'quick cluster'. Alternatively, you can explicitly specify the existing network resources to use for a 'custom cluster'. If you specify existing network resources, you or somebody else must have already configured those resources appropriately. See Network Resource Configuration for Cluster Creation and Deployment.

This topic gives examples of how you might configure network resources for highly available 'custom cluster' creation and deployment in a region with three availability domains:

Note that all the examples in this topic include a service gateway to enable worker nodes to access other Oracle Cloud Infrastructure resources in the same region (such as Oracle Cloud Infrastructure Registry) without exposing data to the public internet. However, you might be expecting applications deployed on the cluster to require access to public endpoints or services not supported by a service gateway. For example, to download updates or patches. If so, configure additional network resources (such as a NAT gateway) to access the internet.

For an introductory tutorial, see Creating a Cluster with Oracle Cloud Infrastructure Container Engine for Kubernetes.

Example 1: Example Network Resource Configuration for a Highly Available Public Cluster in a Region with Three Availability Domains, Using AD-Specific Subnets

This example assumes you want worker nodes hosted in three public AD-specific subnets that can be accessed directly from the internet.

Example Network Resource Configuration

Resource Example
VCN

Created manually, and defined as follows:

  • Name: acme-dev-vcn
  • CIDR Block: 10.0.0.0/16
  • DNS Resolution: Selected
Internet Gateway

Created manually, and defined as follows:

  • Name: gateway-0
Service Gateway

Created manually, and defined as follows:

  • Name: service-gateway-0
  • Services: All <region> Services in Oracle Services Network
Route Table

Two route tables created manually, named, and defined as follows:

  • Name: routetable-0, with a route rule defined as follows:

    • Destination CIDR block: 0.0.0.0/0
    • Target Type: Internet Gateway
    • Target Internet Gateway: gateway-0
  • Name: routetable-1, with a route rule defined as follows:

    • Destination: All <region> Services in Oracle Services Network

    • Target Type: Service Gateway

    • Target: service-gateway-0

DHCP Options

Created automatically and defined as follows:

  • DNS Type set to Internet and VCN Resolver
Security Lists

Two created (in addition to the default security list) manually, named, and defined as follows:

  • Security List Name: workers
  • Security List Name: loadbalancers

For details of the ingress rules and egress rules defined for the workers security list and the loadbalancers security list, see Example Security List Configurations for a Highly Available Public Cluster Using AD-Specific Subnets.

Subnets

Three worker node AD-specific subnets created manually, named, and defined as follows:

  • Name: workers-1 with the following properties:

    • Availability Domain: AD1
    • CIDR Block: 10.0.10.0/24
    • Route Table: routetable-1

    • Subnet access: Public

    • DNS Resolution: Selected

    • DHCP Options: Default
    • Security List: workers
  • Name: workers-2 with the following properties:

    • Availability Domain: AD2
    • CIDR Block: 10.0.11.0/24
    • Route Table: routetable-1

    • Subnet access: Public

    • DNS Resolution: Selected
    • DHCP Options: Default

    • Security List: workers
  • Name: workers-3 with the following properties:

    • Availability Domain: AD3
    • CIDR Block: 10.0.12.0/24
    • Route Table: routetable-1

    • Subnet access: Public

    • DNS Resolution: Selected
    • DHCP Options: Default

    • Security List: workers

Two load balancer AD-specific subnets created, named, and defined as follows:

  • Name: loadbalancers-1 with the following properties:

    • Availability Domain: AD1
    • CIDR Block: 10.0.20.0/24
    • Route Table: routetable-0

    • Subnet access: Public

    • DNS Resolution: Selected
    • DHCP Options: Default

    • Security List: loadbalancers
  • Name: loadbalancers-2 with the following properties:

    • Availability Domain: AD2
    • CIDR Block: 10.0.21.0/24
    • Route Table: routetable-0

    • Subnet access: Public

    • DNS Resolution: Selected
    • DHCP Options: Default

    • Security List: loadbalancers

Example Security List Configurations for a Highly Available Public Cluster Using AD-Specific Subnets

In the example VCN, two security lists have been created (in addition to the default security list) to control access to and from the public worker node AD-specific subnets and the load balancer AD-specific subnets. The two security lists are named 'workers' and 'loadbalancers' respectively.

The workers security list has the following ingress and egress rules for the public worker node AD-specific subnets:

Example Ingress Rules in a Security List for Public Worker Node AD-Specific Subnets:
Example Egress Rules in a Security List for Public Worker Node AD-Specific Subnets:

The loadbalancers security list has the following ingress and egress rules for load balancer AD-specific subnets:

Example Ingress Rules in a Security List for Load Balancer AD-Specific Subnets:
Example Egress Rules in a Security List for Load Balancer AD-Specific Subnets:

Example 2: Example Network Resource Configuration for a Highly Available Private Cluster in a Region with Three Availability Domains, Using AD-Specific Subnets

This example assumes you want worker nodes hosted in three private AD-specific subnets that can only be accessed from within the VCN.

Example Network Resource Configuration

Resource Example
VCN

Created manually, and defined as follows:

  • Name: acme-dev-vcn
  • CIDR Block: 10.0.0.0/16
  • DNS Resolution: Selected
Internet Gateway

Created manually, and defined as follows:

  • Name: gateway-0
NAT Gateway

Created manually, and defined as follows:

  • Name: nat-gateway-0
Service Gateway

Created manually, and defined as follows:

  • Name: service-gateway-0
  • Services: All <region> Services in Oracle Services Network
Route Table

Two route tables created manually, named, and defined as follows:

  • Name: routetable-0, with a route rule defined as follows:

    • Destination CIDR block: 0.0.0.0/0
    • Target Type: Internet Gateway
    • Target Internet Gateway: gateway-0
  • Name: routetable-1, with two route rules defined as follows:

    • Rule 1:
      • Destination CIDR block: 0.0.0.0/0
      • Target Type: NAT Gateway
      • Target NAT Gateway: nat-gateway-0
    • Rule 2:
      • Destination: All <region> Services in Oracle Services Network

      • Target Type: Service Gateway

      • Target: service-gateway-0

DHCP Options

Created automatically and defined as follows:

  • DNS Type set to Internet and VCN Resolver
Security Lists

Two created (in addition to the default security list) manually, named, and defined as follows:

  • Security List Name: workers
  • Security List Name: loadbalancers

For details of the ingress rules and egress rules defined for these security lists, see Example Security List Configurations for a Highly Available Private Cluster Using AD-Specific Subnets.

Subnets

Three worker node AD-specific subnets created manually, named, and defined as follows:

  • Name: workers-1 with the following properties:

    • Availability Domain: AD1
    • CIDR Block: 10.0.10.0/24
    • Route Table: routetable-1

    • Subnet access: Private

    • DNS Resolution: Selected

    • DHCP Options: Default
    • Security List: workers
  • Name: workers-2 with the following properties:

    • Availability Domain: AD2
    • CIDR Block: 10.0.11.0/24
    • Route Table: routetable-1

    • Subnet access: Private

    • DNS Resolution: Selected
    • DHCP Options: Default

    • Security List: workers
  • Name: workers-3 with the following properties:

    • Availability Domain: AD3
    • CIDR Block: 10.0.12.0/24
    • Route Table: routetable-1

    • Subnet access: Private

    • DNS Resolution: Selected
    • DHCP Options: Default

    • Security List: workers

Two load balancer AD-specific subnets created, named, and defined as follows:

  • Name: loadbalancers-1 with the following properties:

    • Availability Domain: AD1
    • CIDR Block: 10.0.20.0/24
    • Route Table: routetable-0

    • Subnet access: Public

    • DNS Resolution: Selected
    • DHCP Options: Default

    • Security List: loadbalancers
  • Name: loadbalancers-2 with the following properties:

    • Availability Domain: AD2
    • CIDR Block: 10.0.21.0/24
    • Route Table: routetable-0

    • Subnet access: Public

    • DNS Resolution: Selected
    • DHCP Options: Default

    • Security List: loadbalancers

Example Security List Configurations for a Highly Available Private Cluster Using AD-Specific Subnets

In the example VCN, two security lists have been created (in addition to the default security list) to control access to and from the private worker node AD-specific subnets and the load balancer AD-specific subnets. The two security lists are named 'workers' and 'loadbalancers' respectively.

The workers security list has the following ingress and egress rules for private worker node subnets:

Example Ingress Rules in a Security List for Private Worker Node AD-Specific Subnets:
Example Egress Rules in a Security List for Private Worker Node AD-Specific Subnets:

The loadbalancers security list has the following ingress and egress rules for load balancer AD-specific subnets:

Example Ingress Rules in a Security List for a Load Balancer AD-Specific Subnet:
Example Egress Rules in a Security List for a Load Balancer AD-Specific Subnet:

Example 3: Example Network Resource Configuration for a Highly Available Public Cluster in a Region with Three Availability Domains, Using a Regional Subnet

This example assumes you want worker nodes hosted in a public regional subnet that can be accessed directly from the internet.

Example Network Resource Configuration

Resource Example
VCN

Created manually, and defined as follows:

  • Name: acme-dev-vcn
  • CIDR Block: 10.0.0.0/16
  • DNS Resolution: Selected
Internet Gateway

Created manually, and defined as follows:

  • Name: gateway-0
Service Gateway

Created manually, and defined as follows:

  • Name: service-gateway-0
  • Services: All <region> Services in Oracle Services Network
Route Table

Two route tables created manually, named, and defined as follows:

  • Name: routetable-0, with a route rule defined as follows:

    • Destination CIDR block: 0.0.0.0/0
    • Target Type: Internet Gateway
    • Target Internet Gateway: gateway-0
  • Name: routetable-1, with a route rule defined as follows:

    • Destination: All <region> Services in Oracle Services Network

    • Target Type: Service Gateway

    • Target: service-gateway-0

DHCP Options

Created automatically and defined as follows:

  • DNS Type set to Internet and VCN Resolver
Security Lists

Two created (in addition to the default security list) manually, named, and defined as follows:

  • Security List Name: workers
  • Security List Name: loadbalancers

For details of the ingress rules and egress rules defined for the workers security list and the loadbalancers security list, see Example Security List Configurations for a Highly Available Public Cluster Using AD-Specific Subnets.

Subnets

One worker node regional subnet created manually, named, and defined as follows:

  • Name: workers-rs with the following properties:

    • CIDR Block: 10.0.10.0/24
    • Route Table: routetable-1

    • Subnet access: Public

    • DNS Resolution: Selected

    • DHCP Options: Default
    • Security List: workers

One load balancer regional subnet created, named, and defined as follows:

  • Name: loadbalancers-rs with the following properties:

    • CIDR Block: 10.0.20.0/24
    • Route Table: routetable-0

    • Subnet access: Public

    • DNS Resolution: Selected
    • DHCP Options: Default

    • Security List: loadbalancers

Example Security List Configurations for a Highly Available Public Cluster using Regional Subnets

In the example VCN, two security lists have been created (in addition to the default security list) to control access to and from a public worker node regional subnet and a load balancer regional subnet. The two security lists are named 'workers' and 'loadbalancers' respectively.

The workers security list has the following ingress and egress rules for the public worker node regional subnet:

Example Ingress Rules in a Security List for a Public Worker Node Regional Subnet:
Example Egress Rules in a Security List for a Public Worker Node Regional Subnet:

The loadbalancers security list has the following ingress and egress rules for load balancer regional subnets:

Example Ingress Rules in a Security List for a Load Balancer Regional Subnet:
Example Egress Rules in a Security List for a Load Balancer Regional Subnet: