Example Network Resource Configurations

Before you can use Container Engine for Kubernetes to create and deploy clusters in the regions in a tenancy:

  • Within the tenancy, there must already be a compartment to contain the necessary network resources (such as a VCN, subnets, internet gateway, route table, security lists). If such a compartment does not exist already, you will have to create it. Note that the network resources can reside in the root compartment. However, if you expect multiple teams to create clusters, best practice is to create a separate compartment for each team.
  • Within the compartment, network resources (such as a VCN, subnets, internet gateway, route table, security lists) must be appropriately configured in each region in which you want to create and deploy clusters. When creating a new cluster, you can have Container Engine for Kubernetes automatically create and configure new network resources for a new 'quick cluster'. Alternatively, you can explicitly specify the existing network resources to use for a 'custom cluster'. If you specify existing network resources, you or somebody else must have already configured those resources appropriately. See Network Resource Configuration for Cluster Creation and Deployment.

This topic gives examples of how you might configure network resources for highly available 'custom cluster' creation and deployment in a region with three availability domains:

Note that all the examples in this topic include a service gateway to enable worker nodes to access other Oracle Cloud Infrastructure resources in the same region (such as Oracle Cloud Infrastructure Registry) without exposing data to the public internet. However, you might be expecting applications deployed on the cluster to require access to public endpoints or services not supported by a service gateway. For example, to download updates or patches. If so, configure additional network resources (such as a NAT gateway) to access the internet.

For an introductory tutorial, see Creating a Cluster with Oracle Cloud Infrastructure Container Engine for Kubernetes.

Example 1: Example Network Resource Configuration for a Highly Available Public Cluster in a Region with Three Availability Domains, Using AD-Specific Subnets

This example assumes you want worker nodes hosted in three public AD-specific subnets that can be accessed directly from the internet.

Example Network Resource Configuration

Resource Example
VCN

Created manually, and defined as follows:

  • Name: acme-dev-vcn
  • CIDR Block: 10.0.0.0/16
  • DNS Resolution: Selected
Internet Gateway

Created manually, and defined as follows:

  • Name: gateway-0
Service Gateway

Created manually, and defined as follows:

  • Name: service-gateway-0
  • Services: All <region> Services in Oracle Services Network
Route Table

Two route tables created manually, named, and defined as follows:

  • Name: routetable-0, with a route rule defined as follows:

    • Destination CIDR block: 0.0.0.0/0
    • Target Type: Internet Gateway
    • Target Internet Gateway: gateway-0
  • Name: routetable-1, with a route rule defined as follows:

    • Destination: All <region> Services in Oracle Services Network
    • Target Type: Service Gateway
    • Target: service-gateway-0

Note that to avoid the possibility of asymmetric routing, a route table for a public subnet cannot contain both a route rule that targets an internet gateway as well as a route rule that targets a service gateway (for more information, see Issues with access from Oracle services through a service gateway to your public instances).

DHCP Options

Created automatically and defined as follows:

  • DNS Type set to Internet and VCN Resolver
Security Lists

Two created (in addition to the default security list) manually, named, and defined as follows:

  • Security List Name: workers
  • Security List Name: loadbalancers

For details of the ingress rules and egress rules defined for the workers security list and the loadbalancers security list, see Example Security List Configurations for a Highly Available Public Cluster Using AD-Specific Subnets.

Subnets

Three worker node AD-specific subnets created manually, named, and defined as follows:

  • Name: workers-1 with the following properties:

    • Availability Domain: AD1
    • CIDR Block: 10.0.10.0/25
    • Route Table: routetable-1
    • Subnet access: Public
    • DNS Resolution: Selected
    • DHCP Options: Default
    • Security List: workers
  • Name: workers-2 with the following properties:

    • Availability Domain: AD2
    • CIDR Block: 10.0.11.0/25
    • Route Table: routetable-1
    • Subnet access: Public
    • DNS Resolution: Selected
    • DHCP Options: Default
    • Security List: workers
  • Name: workers-3 with the following properties:

    • Availability Domain: AD3
    • CIDR Block: 10.0.12.0/25
    • Route Table: routetable-1
    • Subnet access: Public
    • DNS Resolution: Selected
    • DHCP Options: Default
    • Security List: workers

Two load balancer AD-specific subnets created, named, and defined as follows:

  • Name: loadbalancers-1 with the following properties:

    • Availability Domain: AD1
    • CIDR Block: 10.0.20.0/24
    • Route Table: routetable-0
    • Subnet access: Public
    • DNS Resolution: Selected
    • DHCP Options: Default
    • Security List: loadbalancers
  • Name: loadbalancers-2 with the following properties:

    • Availability Domain: AD2
    • CIDR Block: 10.0.21.0/24
    • Route Table: routetable-0
    • Subnet access: Public
    • DNS Resolution: Selected
    • DHCP Options: Default
    • Security List: loadbalancers

Example Security List Configurations for a Highly Available Public Cluster Using AD-Specific Subnets

In the example VCN, two security lists have been created (in addition to the default security list) to control access to and from the public worker node AD-specific subnets and the load balancer AD-specific subnets. The two security lists are named 'workers' and 'loadbalancers' respectively.

Note that two sets of alternative ingress and egress rules are given for the 'loadbalancers' security list, showing how to allow either unrestricted access to/from the cluster, and how to restrict access to/from a specific CIDR range.

Example Ingress Rules in the 'workers' Security List for Public Worker Node AD-Specific Subnets:
#

State: Src. Type:

Src. CIDR

Protocol: Src. Port Range: Dest. Port Range:

Type: Code:

Allows: Description:

1

State:Stateless

Src. Type: CIDR

10.0.10.0/25

Protocol: All

Src. Port Range: n/a

Dest. Port Range: n/a

n/a

Allows: All traffic for all ports

Description: This optional rule enables intra-VCN traffic. Although optional, this rule is usually necessary to enable pods on one worker node to communicate with pods on other worker nodes. Only in the rare case where no inter-pod communication is expected, is this rule not required.

2 State: Stateless

Src. Type: CIDR

10.0.11.0/25

Protocol: All

Src. Port Range: n/a

Dest. Port Range: n/a

n/a

Allows: All traffic for all ports

Description: This optional rule enables intra-VCN traffic. Although optional, this rule is usually necessary to enable pods on one worker node to communicate with pods on other worker nodes. In the rare case where no inter-pod communication is expected, this rule is not required.

3 State: Stateless

Src. Type: CIDR

10.0.12.0/25

Protocol: All

Src. Port Range: n/a

Dest. Port Range: n/a

n/a

Allows: All traffic for all ports

Description: This optional rule enables intra-VCN traffic. Although optional, this rule is usually necessary to enable pods on one worker node to communicate with pods on other worker nodes. In the rare case where no inter-pod communication is expected, this rule is not required.

4 State: Stateful

Src. Type: CIDR

Src. CIDR: 0.0.0.0/0

0.0.0.0/0

Protocol: ICMP

Src. Port Range: n/a

Dest. Port Range: n/a

Type: 3

Code: 4

Allows: ICMP traffic for: 3, 4 Destination Unreachable: Fragmentation Needed and Don't Fragment was Set

Description: This rule enables worker nodes to receive Path MTU Discovery fragmentation messages.

5 State: Stateful

Src. Type: CIDR

0.0.0.0/0

Protocol: TCP

Src. Port Range: All

Dest. Port Range: 22

n/a

Allows: TCP traffic for ports: 22 SSH Remote Login Protocol

Description: This optional rule enables inbound SSH traffic from the internet on port 22 to access worker nodes.

6 State: Stateful

Src. Type: CIDR

0.0.0.0/0

Protocol: TCP

Src. Port Range: All

Dest. Port Range: 30000 - 32767

n/a

Allows: TCP traffic for ports: 30000 - 32767

Description: This optional rule enables inbound traffic to the worker nodes on the default NodePort range of 30000-32767 (see the Kubernetes documentation).

Example Egress Rules in the 'workers' Security List for Public Worker Node AD-Specific Subnets:

Consider these egress rules as a minimum requirement. If applications deployed on worker nodes need to communicate with destinations outside of the Oracle Service Network, add additional egress rules. For example, if applications use Helm charts from the Helm stable repository, or Docker images from docker.io.

# State: Dest. Type: Dest. CIDR or Dest. Service

Protocol: Src. Port Range: Dest. Port Range:

Type and Code Allows: and Description:
1 State: Stateless

Dest. Type: CIDR

10.0.10.0/25

Protocol: All

Src. Port Range: n/a

Dest. Port Range: n/a

n/a

Allows: All traffic for all ports

Description: This optional rule enables intra-VCN traffic. Although optional, this rule is usually necessary to enable pods on one worker node to communicate with pods on other worker nodes. Only in the rare case where no inter-pod communication is expected, is this rule not required.

2 State: Stateless

Dest. Type: CIDR

10.0.11.0/25

Protocol: All

Src. Port Range: n/a

Dest. Port Range: n/a

n/a

Allows: All traffic for all ports

Description: This optional rule enables intra-VCN traffic. Although optional, this rule is usually necessary to enable pods on one worker node to communicate with pods on other worker nodes. Only in the rare case where no inter-pod communication is expected, is this rule not required.

3 State: Stateless

Dest. Type: CIDR

10.0.12.0/25

Protocol: All

Src. Port Range: n/a

Dest. Port Range: n/a

n/a

Allows: All traffic for all ports

Description: This optional rule enables intra-VCN traffic. Although optional, this rule is usually necessary to enable pods on one worker node to communicate with pods on other worker nodes. Only in the rare case where no inter-pod communication is expected, is this rule not required.

4 State: Stateful

Dest. Type: Service

All <region> Services in Oracle Services Network

Protocol: ICMP

Src. Port Range: n/a

Dest. Port Range: n/a

All Allows: ICMP traffic for all ports.

Description: This rule enables outbound ICMP traffic.

5 State: Stateful

Dest. Type: Service

All <region> Services in Oracle Services Network

Protocol: TCP

Src. Port Range: 80

Dest. Port Range: 80

n/a Allows: TCP traffic for ports: 80.

Description: This rule enables worker nodes to communicate with Container Engine for Kubernetes to ensure correct start-up, and continued functioning.

6 State: Stateful

Dest. Type: Service

All <region> Services in Oracle Services Network

Protocol: TCP

Src. Port Range: 443

Dest. Port Range: 443

n/a Allows: TCP traffic for ports: 443

Description: This rule enables worker nodes to communicate with Container Engine for Kubernetes to ensure correct start-up, and continued functioning.

7 State: Stateful

Dest. Type: Service

All <region> Services in Oracle Services Network

Protocol: TCP

Src. Port Range: 6443

Dest. Port Range: 6443

n/a Allows: TCP traffic for ports: 6443

Description: This rule enables worker nodes to communicate with Container Engine for Kubernetes to ensure correct start-up, and continued functioning.

8 State: Stateful

Dest. Type: Service

All <region> Services in Oracle Services Network

Protocol: TCP

Src. Port Range: 12250

Dest. Port Range: 12250

n/a Allows: TCP traffic for ports: 12250

Description: This rule enables worker nodes to communicate with Container Engine for Kubernetes to ensure correct start-up, and continued functioning.

Example Ingress Rules in the 'loadbalancers' Security List for Load Balancer AD-Specific Subnets:

If you want to allow unrestricted incoming traffic through the load balancer from the public internet, set up the following security rule:

#

State: Src. Type:

Src. CIDR

Protocol: Src. Port Range: Dest. Port Range:

Type: Code:

Allows: Description:

1 State: Stateless

Src. Type: CIDR

0.0.0.0/0

Protocol: TCP

Src. Port Range: All

Dest. Port Range: All

n/a

Allows: TCP traffic for all ports: all

Description: This rule enables incoming public traffic to service load balancers.

Alternatively, if you want to restrict incoming traffic through the load balancer to just that coming from a particular public CIDR range (for example, 160.34.126.216/30), set up the following security rule:

#

State: Src. Type:

Src. CIDR

Protocol: Src. Port Range: Dest. Port Range:

Type: Code:

Allows: Description:

1 State: Stateful

Src. Type: CIDR

160.34.126.216/30

Protocol: TCP

Src. Port Range: All

Dest. Port Range: All

n/a

Allows: TCP traffic for all ports: all

Description: This rule only allows incoming public traffic to service load balancers from a particular public CIDR range.

Example Egress Rules in the 'loadbalancers' Security List for Load Balancer AD-Specific Subnets:

If you want to allow unrestricted outgoing response traffic through the load balancer to the public internet, set up the following security rules:

#

State: Dest. Type:

Dest. CIDR

Protocol: Src. Port Range: Dest. Port Range:

Type: Code:

Allows: Description:

1 State: Stateless

Dest. Type: CIDR

0.0.0.0/0

Protocol: TCP

Src. Port Range: All

Dest. Port Range: All

n/a

Allows: TCP traffic for ports: all

Description: This rule enables responses from a web application through the service load balancers.

Alternatively, if you want to restrict outgoing response traffic through the load balancer to just that going to worker node subnets in the cluster, set up the following security rules:

#

State: Dest. Type:

Dest. CIDR

Protocol: Src. Port Range: Dest. Port Range:

Type: Code:

Allows: Description:

1 State: Stateful

Dest. Type: CIDR

10.0.10.0/25

Protocol: TCP

Src. Port Range: All

Dest. Port Range: All

n/a

Allows: TCP traffic for ports: all

Description: This rule restricts responses of the web application to the workers-1 subnet.

2 State: Stateful

Dest. Type: CIDR

10.0.11.0/25 Protocol: TCP

Src. Port Range: All

Dest. Port Range: All

n/a Allows: TCP traffic for ports: all

Description: This rule restricts responses of the web application to the workers-2 subnet.

3 State: Stateful

Dest. Type: CIDR

10.0.12.0/25 Protocol: TCP

Src. Port Range: All

Dest. Port Range: All

n/a Allows: TCP traffic for ports: all

Description: This rule restricts responses of the web application to the workers-3 subnet.

Example 2: Example Network Resource Configuration for a Highly Available Private Cluster in a Region with Three Availability Domains, Using AD-Specific Subnets

This example assumes you want worker nodes hosted in three private AD-specific subnets that can only be accessed from within the VCN.

Example Network Resource Configuration

Resource Example
VCN

Created manually, and defined as follows:

  • Name: acme-dev-vcn
  • CIDR Block: 10.0.0.0/16
  • DNS Resolution: Selected
Internet Gateway

Created manually, and defined as follows:

  • Name: gateway-0
NAT Gateway

Created manually, and defined as follows:

  • Name: nat-gateway-0
Service Gateway

Created manually, and defined as follows:

  • Name: service-gateway-0
  • Services: All <region> Services in Oracle Services Network
Route Table

Two route tables created manually, named, and defined as follows:

  • Name: routetable-0, with a route rule defined as follows:

    • Destination CIDR block: 0.0.0.0/0
    • Target Type: Internet Gateway
    • Target Internet Gateway: gateway-0
  • Name: routetable-1, with two route rules defined as follows:

    • Rule 1:
      • Destination CIDR block: 0.0.0.0/0
      • Target Type: NAT Gateway
      • Target NAT Gateway: nat-gateway-0
    • Rule 2:
      • Destination: All <region> Services in Oracle Services Network
      • Target Type: Service Gateway
      • Target: service-gateway-0

Note that to avoid the possibility of asymmetric routing, a route table for a public subnet cannot contain both a route rule that targets an internet gateway as well as a route rule that targets a service gateway (for more information, see Issues with access from Oracle services through a service gateway to your public instances).

DHCP Options

Created automatically and defined as follows:

  • DNS Type set to Internet and VCN Resolver
Security Lists

Two created (in addition to the default security list) manually, named, and defined as follows:

  • Security List Name: workers
  • Security List Name: loadbalancers

For details of the ingress rules and egress rules defined for these security lists, see Example Security List Configurations for a Highly Available Private Cluster Using AD-Specific Subnets.

Subnets

Three worker node AD-specific subnets created manually, named, and defined as follows:

  • Name: workers-1 with the following properties:

    • Availability Domain: AD1
    • CIDR Block: 10.0.10.0/25
    • Route Table: routetable-1
    • Subnet access: Private
    • DNS Resolution: Selected
    • DHCP Options: Default
    • Security List: workers
  • Name: workers-2 with the following properties:

    • Availability Domain: AD2
    • CIDR Block: 10.0.11.0/25
    • Route Table: routetable-1
    • Subnet access: Private
    • DNS Resolution: Selected
    • DHCP Options: Default
    • Security List: workers
  • Name: workers-3 with the following properties:

    • Availability Domain: AD3
    • CIDR Block: 10.0.12.0/25
    • Route Table: routetable-1
    • Subnet access: Private
    • DNS Resolution: Selected
    • DHCP Options: Default
    • Security List: workers

Two load balancer AD-specific subnets created, named, and defined as follows:

  • Name: loadbalancers-1 with the following properties:

    • Availability Domain: AD1
    • CIDR Block: 10.0.20.0/24
    • Route Table: routetable-0
    • Subnet access: Public
    • DNS Resolution: Selected
    • DHCP Options: Default
    • Security List: loadbalancers
  • Name: loadbalancers-2 with the following properties:

    • Availability Domain: AD2
    • CIDR Block: 10.0.21.0/24
    • Route Table: routetable-0
    • Subnet access: Public
    • DNS Resolution: Selected
    • DHCP Options: Default
    • Security List: loadbalancers

Example Security List Configurations for a Highly Available Private Cluster Using AD-Specific Subnets

In the example VCN, two security lists have been created (in addition to the default security list) to control access to and from the private worker node AD-specific subnets and the load balancer AD-specific subnets. The two security lists are named 'workers' and 'loadbalancers' respectively.

Example Ingress Rules in the 'workers' Security List for Private Worker Node AD-Specific Subnets:
#

State: Src. Type:

Src. CIDR

Protocol: Src. Port Range: Dest. Port Range:

Type: Code:

Allows: Description:

1 State: Stateless

Src. Type: CIDR

10.0.10.0/25

Protocol: All

Src. Port Range: n/a

Dest. Port Range: n/a

n/a

Allows: All traffic for all ports

Description: This optional rule enables intra-VCN traffic. Although optional, this rule is usually necessary to enable pods on one worker node to communicate with pods on other worker nodes.Only in the rare case where no inter-pod communication is expected, is this rule not required.

2 State: Stateless

Src. Type: CIDR

10.0.11.0/25

Protocol: All

Src. Port Range: n/a

Dest. Port Range: n/a

n/a

Allows: All traffic for all ports

Description: This optional rule enables intra-VCN traffic. Although optional, this rule is usually necessary to enable pods on one worker node to communicate with pods on other worker nodes. In the rare case where no inter-pod communication is expected, this rule is not required.

3 State: Stateless

Src. Type: CIDR

10.0.12.0/25

Protocol: All

Src. Port Range: n/a

Dest. Port Range: n/a

n/a

Allows: All traffic for all ports

Description: This optional rule enables intra-VCN traffic. Although optional, this rule is usually necessary to enable pods on one worker node to communicate with pods on other worker nodes. In the rare case where no inter-pod communication is expected, this rule is not required.

4 State: Stateful

Src. Type: CIDR

10.0.0.0/16

Protocol: TCP

Src. Port Range: All

Dest. Port Range: 22

n/a

Allows: TCP traffic for ports: 22 SSH Remote Login Protocol

Description: This optional rule enables inbound SSH traffic from the VCN on port 22 to access worker nodes.

Example Egress Rules in the 'workers' Security List for Private Worker Node AD-Specific Subnets:

Consider these egress rules as a minimum requirement. If applications deployed on worker nodes need to communicate with destinations outside of the Oracle Service Network, add additional egress rules. For example, if applications use Helm charts from the Helm stable repository, or Docker images from docker.io.

#

State: Dest. Type:

Dest. CIDR or Dest. Service

Protocol: Src. Port Range: Dest. Port Range:

Type: Code:

Allows: Description:

1 State: Stateless

Dest. Type: CIDR

10.0.10.0/25

Protocol: All

Src. Port Range: n/a

Dest. Port Range: n/a

n/a

Allows: All traffic for all ports

Description: This optional rule enables intra-VCN traffic. Although optional, this rule is usually necessary to enable pods on one worker node to communicate with pods on other worker nodes. Only in the rare case where no inter-pod communication is expected, is this rule not required.

2 State: Stateless

Dest. Type: CIDR

10.0.11.0/25

Protocol: All

Src. Port Range: n/a

Dest. Port Range: n/a

n/a

Allows: All traffic for all ports

Description: This optional rule enables intra-VCN traffic. Although optional, this rule is usually necessary to enable pods on one worker node to communicate with pods on other worker nodes. Only in the rare case where no inter-pod communication is expected, is this rule not required.

3 State: Stateless

Dest. Type: CIDR

10.0.12.0/25 Protocol: All Src. Port Range: n/a Dest. Port Range: n/a n/a

Allows: All traffic for all ports

Description: This optional rule enables intra-VCN traffic. Although optional, this rule is usually necessary to enable pods on one worker node to communicate with pods on other worker nodes. Only in the rare case where no inter-pod communication is expected, is this rule not required.

4 State: Stateful

Dest. Type: Service

All <region> Services in Oracle Services Network Protocol: ICMP Src. Port Range: n/a Dest. Port Range: n/a All Allows: ICMP traffic for all ports.

Description: This rule enables outbound ICMP traffic.

5 State: Stateful

Dest. Type: Service

All <region> Services in Oracle Services Network

Protocol: TCP

Src. Port Range: 80

Dest. Port Range: 80

n/a Allows: TCP traffic for ports: 80.

Description: This rule enables worker nodes to communicate with Container Engine for Kubernetes to ensure correct start-up, and continued functioning.

6 State: Stateful

Dest. Type: Service

All <region> Services in Oracle Services Network

Protocol: TCP

Src. Port Range: 443

Dest. Port Range: 443

n/a Allows: TCP traffic for ports: 443

Description: This rule enables worker nodes to communicate with Container Engine for Kubernetes to ensure correct start-up, and continued functioning.

7 State: Stateful

Dest. Type: Service

All <region> Services in Oracle Services Network

Protocol: TCP

Src. Port Range: 6443

Dest. Port Range: 6443

n/a Allows: TCP traffic for ports: 6443

Description: This rule enables worker nodes to communicate with Container Engine for Kubernetes to ensure correct start-up, and continued functioning.

8 State: Stateful

Dest. Type: Service

All <region> Services in Oracle Services Network

Protocol: TCP

Src. Port Range: 12250

Dest. Port Range: 12250

n/a Allows: TCP traffic for ports: 12250

Description: This rule enables worker nodes to communicate with Container Engine for Kubernetes to ensure correct start-up, and continued functioning.

Example Ingress Rules in the 'loadbalancers' Security List for a Load Balancer AD-Specific Subnet:
#

State: Src. Type:

Src. CIDR

Protocol: Src. Port Range: Dest. Port Range:

Type: Code:

Allows: Description:

1 State: Stateless

Src. Type: CIDR

0.0.0.0/0

Protocol: TCP

Src. Port Range: All

Dest. Port Range: All

n/a

Allows: TCP traffic for all ports: all

Description: This rule enables incoming public traffic to service load balancers.

Example Egress Rules in the 'loadbalancers' Security List for a Load Balancer AD-Specific Subnet:
#

State: Dest. Type:

Dest. CIDR

Protocol: Src. Port Range: Dest. Port Range:

Type: Code:

Allows: Description:

1 State: Stateless

Dest. Type: CIDR

0.0.0.0/0 Protocol: TCP Src. Port Range: All Dest. Port Range: All n/a

Allows: TCP traffic for ports: all

Description: This rule enables responses from a web application through the service load balancers.

Example 3: Example Network Resource Configuration for a Highly Available Public Cluster in a Region with Three Availability Domains, Using a Regional Subnet

This example assumes you want worker nodes hosted in a public regional subnet that can be accessed directly from the internet.

Example Network Resource Configuration

Resource Example
VCN

Created manually, and defined as follows:

  • Name: acme-dev-vcn
  • CIDR Block: 10.0.0.0/16
  • DNS Resolution: Selected
Internet Gateway

Created manually, and defined as follows:

  • Name: gateway-0
Service Gateway

Created manually, and defined as follows:

  • Name: service-gateway-0
  • Services: All <region> Services in Oracle Services Network
Route Table

Two route tables created manually, named, and defined as follows:

  • Name: routetable-0, with a route rule defined as follows:

    • Destination CIDR block: 0.0.0.0/0
    • Target Type: Internet Gateway
    • Target Internet Gateway: gateway-0
  • Name: routetable-1, with a route rule defined as follows:

    • Destination: All <region> Services in Oracle Services Network
    • Target Type: Service Gateway
    • Target: service-gateway-0

Note that to avoid the possibility of asymmetric routing, a route table for a public subnet cannot contain both a route rule that targets an internet gateway as well as a route rule that targets a service gateway (for more information, see Issues with access from Oracle services through a service gateway to your public instances).

DHCP Options

Created automatically and defined as follows:

  • DNS Type set to Internet and VCN Resolver
Security Lists

Two created (in addition to the default security list) manually, named, and defined as follows:

  • Security List Name: workers
  • Security List Name: loadbalancers

For details of the ingress rules and egress rules defined for the workers security list and the loadbalancers security list, see Example Security List Configurations for a Highly Available Public Cluster using Regional Subnets.

Subnets

One worker node regional subnet created manually, named, and defined as follows:

  • Name: workers-rs with the following properties:

    • CIDR Block: 10.0.10.0/25
    • Route Table: routetable-1
    • Subnet access: Public
    • DNS Resolution: Selected
    • DHCP Options: Default
    • Security List: workers

One load balancer regional subnet created, named, and defined as follows:

  • Name: loadbalancers-rs with the following properties:

    • CIDR Block: 10.0.20.0/24
    • Route Table: routetable-0
    • Subnet access: Public
    • DNS Resolution: Selected
    • DHCP Options: Default
    • Security List: loadbalancers

Example Security List Configurations for a Highly Available Public Cluster using Regional Subnets

In the example VCN, two security lists have been created (in addition to the default security list) to control access to and from a public worker node regional subnet and a load balancer regional subnet. The two security lists are named 'workers' and 'loadbalancers' respectively.

Example Ingress Rules in the 'workers' Security List for a Public Worker Node Regional Subnet:
#

State: Src. Type:

Src. CIDR

Protocol: Src. Port Range: Dest. Port Range:

Type: Code:

Allows: Description:

1 State: Stateless

Src. Type: CIDR

10.0.10.0/25

Protocol: All

Src. Port Range: n/a

Dest. Port Range: n/a

n/a

Allows: All traffic for all ports

Description: This optional rule enables intra-VCN traffic. Although optional, this rule is usually necessary to enable pods on one worker node to communicate with pods on other worker nodes. Only in the rare case where no inter-pod communication is expected, is this rule not required.

2 State: Stateful

Src. Type: CIDR

0.0.0.0/0

Protocol: ICMP

Src. Port Range: n/a

Dest. Port Range: n/a

Type: 3

Code: 4

Allows: ICMP traffic for: 3, 4 Destination Unreachable: Fragmentation Needed and Don't Fragment was Set

Description: This rule enables worker nodes to receive Path MTU Discovery fragmentation messages.

3 State: Stateful

Src. Type: CIDR

0.0.0.0/0

Protocol: TCP

Src. Port Range: All

Dest. Port Range: 22

n/a

Allows: TCP traffic for ports: 22 SSH Remote Login Protocol

Description: This optional rule enables inbound SSH traffic from the internet on port 22 to access worker nodes.

4 State: Stateful

Src. Type: CIDR

0.0.0.0/0

Protocol: TCP

Src. Port Range: All

Dest. Port Range: 30000 - 32767

n/a

Allows: TCP traffic for ports: 30000 - 32767

Description: This optional rule enables inbound traffic to the worker nodes on the default NodePort range of 30000-32767 (see the Kubernetes documentation).

Example Egress Rules in the 'workers' Security List for a Public Worker Node Regional Subnet:

Consider these egress rules as a minimum requirement. If applications deployed on worker nodes need to communicate with destinations outside of the Oracle Service Network, add additional egress rules. For example, if applications use Helm charts from the Helm stable repository, or Docker images from docker.io.

#

State: Dest. Type:

Dest. CIDR or Dest. Service

Protocol: Src. Port Range: Dest. Port Range:

Type: Code:

Allows: Description:

1 State: Stateless

Dest. Type: CIDR

10.0.10.0/25

Protocol: All

Src. Port Range: n/a

Dest. Port Range: n/a

n/a

Allows: All traffic for all ports

Description: This optional rule enables intra-VCN traffic. Although optional, this rule is usually necessary to enable pods on one worker node to communicate with pods on other worker nodes. Only in the rare case where no inter-pod communication is expected, is this rule not required.

2 State: Stateful

Dest. Type: Service

All <region> Services in Oracle Services Network

Protocol: ICMP

Src. Port Range: n/a

Dest. Port Range: n/a

All Allows: ICMP traffic for all ports.

Description: This rule enables outbound ICMP traffic.

3 State: Stateful

Dest. Type: Service

All <region> Services in Oracle Services Network

Protocol: TCP

Src. Port Range: 80

Dest. Port Range: 80

n/a Allows: TCP traffic for ports: 80.

Description: This rule enables worker nodes to communicate with Container Engine for Kubernetes to ensure correct start-up, and continued functioning.

4 State: Stateful

Dest. Type: Service

All <region> Services in Oracle Services Network

Protocol: TCP

Src. Port Range: 443

Dest. Port Range: 443

n/a Allows: TCP traffic for ports: 443

Description: This rule enables worker nodes to communicate with Container Engine for Kubernetes to ensure correct start-up, and continued functioning.

5 State: Stateful

Dest. Type: Service

All <region> Services in Oracle Services Network

Protocol: TCP

Src. Port Range: 6443

Dest. Port Range: 6443

n/a Allows: TCP traffic for ports: 6443

Description: This rule enables worker nodes to communicate with Container Engine for Kubernetes to ensure correct start-up, and continued functioning.

6 State: Stateful

Dest. Type: Service

All <region> Services in Oracle Services Network

Protocol: TCP

Src. Port Range: 12250

Dest. Port Range: 12250

n/a Allows: TCP traffic for ports: 12250

Description: This rule enables worker nodes to communicate with Container Engine for Kubernetes to ensure correct start-up, and continued functioning.

Example Ingress Rules in the 'loadbalancers' Security List for a Load Balancer Regional Subnet:
#

State: Src. Type:

Src. CIDR

Protocol: Src. Port Range: Dest. Port Range:

Type: Code:

Allows: Description:

1 State: Stateless

Src. Type: CIDR

0.0.0.0/0

Protocol: TCP

Src. Port Range: All

Dest. Port Range: All

n/a

Allows: TCP traffic for all ports: all

Description: This rule enables incoming public traffic to service load balancers.

Example Egress Rules in the 'loadbalancers' Security List for a Load Balancer Regional Subnet:
#

State: Dest. Type:

Dest. CIDR

Protocol: Src. Port Range: Dest. Port Range:

Type: Code:

Allows: Description:

1 State: Stateless

Dest. Type: CIDR

0.0.0.0/0

Protocol: TCP

Src. Port Range: All

Dest. Port Range: All

n/a

Allows: TCP traffic for ports: all

Description: This rule enables responses from a web application through the service load balancers.