Oracle Cloud Infrastructure Documentation

Network Resource Configuration for Cluster Creation and Deployment

Before you can use Container Engine for Kubernetes to create and deploy clusters in the regions in a tenancy:

  • The tenancy's root compartment must include a policy to allow Container Engine for Kubernetes to perform operations in the tenancy. See Create Policy for Container Engine for Kubernetes (Required).
  • Within the tenancy, there must already be a compartment to contain the necessary network resources (VCN, subnets, internet gateway, route table, security lists). If such a compartment does not exist already, you will have to create it. Note that the network resources can reside in the root compartment. However, if you expect multiple teams to create clusters, best practice is to create a separate compartment for each team.
  • Within the compartment, network resources (VCN, subnets, internet gateway, route table, security lists) must be appropriately configured in each region in which you want to create and deploy clusters. When creating a new cluster, you can have Container Engine for Kubernetes automatically create and configure new network resources for the new cluster, or you can specify existing network resources. If you specify existing network resources, you or somebody else must have already configured those resources appropriately, as described in this topic.

This topic describes the necessary configuration for each network resource. To see details of a typical configuration, see Example Network Resource Configuration.

For an introductory tutorial, see Creating a Cluster with Oracle Cloud Infrastructure Container Engine for Kubernetes and Deploying a Sample App.

Root Compartment Configuration

You have to define a policy for the tenancy's root compartment to enable Container Engine for Kubernetes to perform operations on the tenancy. See Create Policy for Container Engine for Kubernetes (Required).

VCN Configuration

The VCN in which you want to create and deploy clusters must be configured as follows:

  • The VCN must have a CIDR block defined that is large enough for at least five subnets, in order to support the number of hosts and load balancers a cluster will usually have. A /16 CIDR block would be large enough for almost all use cases (10.0.0.0/16 for example). The CIDR block you specify for the VCN must not overlap with the CIDR block you specify for pods and for the Kubernetes services (see CIDR Blocks and Container Engine for Kubernetes).
  • The VCN must have an internet gateway defined. See Internet Gateway Configuration.
  • The VCN must have a route table defined that has a route rule specifying the internet gateway as the target for the destination CIDR block. See Route Table Configuration.
  • The VCN must usually have five subnets defined. See Subnet Configuration.

  • The VCN must have security lists defined for worker node subnets and load balancer subnets (if specified). See Security List Configuration.

In addition, Oracle recommends DNS Resolution is selected for the VCN.

See VCNs and Subnets and Example Network Resource Configuration.

Internet Gateway Configuration

The VCN in which you want to create and deploy clusters must have an internet gateway. The internet gateway must be specified as the target for the destination CIDR block 0.0.0.0/0 in a route rule in a route table.

See VCNs and Subnets and Example Network Resource Configuration.

Route Table Configuration

The VCN in which you want to create and deploy clusters must have a route table. The route table must have a route rule that specifies an internet gateway as the target for the destination CIDR block 0.0.0.0/0.

See Internet Gateway and Example Network Resource Configuration.

DHCP Options Configuration

The VCN in which you want to create and deploy clusters must have DHCP Options configured. The default value for DNS Type of Internet and VCN Resolver is acceptable.

See DHCP Options and Example Network Resource Configuration.

Security List Configuration

The VCN in which you want to create and deploy clusters must have security lists defined for worker node subnets and load balancer subnets (if specified). The security lists for worker node subnets and load balancer subnets must be different.

Worker nodes are created with public IP addresses and must be able to make outbound connections to the internet. Container Engine for Kubernetes must also be able to access worker nodes.

The security list for the worker node subnets must have:

  • stateless ingress and egress rules that allow all traffic between the different worker node subnets
  • stateless ingress and egress rules that allow all traffic between worker node subnets and load balancer subnets (if specified)
  • an egress rule that allows all outbound traffic to the internet
  • ingress rules to allow Container Engine for Kubernetes to access worker nodes on port 22 from the following source CIDR blocks:
    • 130.35.0.0/16
    • 138.1.0.0/17
    • 147.154.0.0/16
    • 192.29.0.0/16

Optionally, you can include ingress rules for worker node subnets to:

The security list for load balancer subnets must be unique and for their exclusive use.

See Security Lists and Example Network Resource Configuration.

Subnet Configuration

The VCN in which you want to create and deploy clusters must usually have five subnets defined as follows:

  • Three subnets in which to deploy worker nodes. Each worker node subnet must be in a different availability domain. Worker node subnets must have different security lists to load balancer subnets.
  • (Optionally, but usually) Two subnets to host load balancers. Each load balancer subnet must be in a different availability domain. Load balancer subnets must have different security lists to worker node subnets.

In addition, all subnets must have the following properties set as shown:

  • Route Table: The name of the route table that has a route rule specifying an internet gateway as the target for the destination CIDR block 0.0.0.0/0
  • Subnet access: Public
  • DHCP options: Default

The CIDR blocks you specify for worker node and load balancer subnets must not overlap with CIDR blocks you specify for pods running in the cluster (see CIDR Blocks and Container Engine for Kubernetes).

See VCNs and Subnets and Example Network Resource Configuration.