Using In-transit Encryption

In-transit encryption provides a way to secure your data between instances and mounted file systems using TLS v.1.2 (Transport Layer Security) encryption. Together with other methods of security such as Oracle Cloud Infrastructure Vault (KMS) and File Storage's encryption-at-rest, in-transit encryption provides for end-to-end security.

How In-transit Encryption is Enabled

In-transit encryption doesn't require any updates to your file system's mount target or export configuration. To enable in-transit encryption, you install a package called oci-fss-utils on your instance. The oci-fss-utils package creates a network namespace and virtual network interface on your instance and provides a local NFS endpoint. The oci-fss-utils package also runs a forwarder process in the background called oci-fss-fowarder.

The network namespace isolates the forwarder process from your instance’s networking environment. The virtual network interface provides the forwarder process a unique IP address. The local NFS endpoint provides NFS connection capability.

The file system is mounted using a special command that initiates encryption. After the file system is mounted, the oci-fss-forwarder process connects the local NFS client to the NFS endpoint. The process then receives requests from the NFS client, encrypts them and sends them to the mount target using a TLS tunnel.

Here are the general steps for setting up In-transit encryption:

  1. Download the oci-fss-utils package. For instructions, see Task 1: Download the OCI-FSS-UTILS package
  2. Install the oci-fss-utils package on the instance. For instructions, see Task 2: Install the OCI-FSS-UTILS package on Oracle Linux or CentOS
  3. Use the in-transit encryption command to mount the file system. For instructions, see Task 3: Mount the file system with the encryption command

Limitations and Considerations

  • The in-transit encryption installation package is distributed as an RPM for Oracle Linux and CentOS and can be downloaded at cloud-infrastructure-file-storage-downloads.html
  • You must install the oci-fss-utils package on every instance that requires encrypted access to a mount target.
  • The number of encrypted NFS/TLS connections for a single mount target is limited to 64. This limitation is caused by TLS memory requirements. Unlike NFS connections, TLS connections do not share memory buffers. So, once a TLS connection has been established, the allocated memory stays dedicated to it.

Setting up In-transit Encryption

Prerequisites

  • Add the following new rules to the security list for the mount target subnet. Alternatively, you can add the following rules to a Network Security Group (NSG) and then add the mount target to the NSG. For more information and instructions about adding security list rules for File Storage, see Configuring VCN Security Rules for File Storage.
    • A stateful ingress rule allowing TCP traffic to a Destination Port Range of 2051.
    • A stateful egress rule allowing TCP traffic from a Source Port Range of 2051.
    Important

    Standard (unencrypted) access to File Storage mount targets requires access to the following ports:

    • Stateful ingress to TCP ports 111, 2048, 2049, and 2050.
    • Stateful ingress to UDP ports 111 and 2048.
    • Stateful egress from TCP ports 111, 2048, 2049, and 2050.
    • Stateful egress from UDP port 111.

    If you have previously set up rules for standard access, and you want to enforce encrypted access only, then you can disable the standard access ports.

    Only the rules for TCP port 2051 are required for encrypted access.

Setup Tasks

Task 1: Download the OCI-FSS-UTILS package

Internet access is required to download the RPM installation package. If the destination instance doesn't have internet access, you can download the RPM to a staging instance on your network and then use the scp command to securely copy the RPM from the staging instance to the destination instance.

The scp command requires an SSH key pair to authenticate a remote user. If your instances are UNIX-style systems, you probably already have the ssh-keygen utility installed. To determine if it's installed, open a shell or terminal and type ssh-keygen on the command line. If it's not installed, you can obtain OpenSSH for UNIX from http://www.openssh.com/portable.html.

  1. (Optional) Create a directory for the RPM installation package on the destination instance. For example: 

    sudo mkdir -p /<rpm_directory_name>
  2. Download the oci-fss-utils package from cloud-infrastructure-file-storage-downloads.html to the directory on the destination instance or to a staging instance on your network.

    If you've downloaded the package directly to the destination instance, skip the next step and proceed directly to Task 2: Install the OCI-FSS-UTILS package on Oracle Linux or CentOS.

    If you've downloaded the package to a staging instance, proceed to the next step in these instructions.

  3. Open a terminal window on the staging instance, and use the scp command to securely copy the RPM from the staging instance to the destination instance. For example:

    scp -i <private_key> <path_to_.rpm> <username>@<destination-public-ip-address>:/<rpm_directory_name>

    After the RPM package is downloaded to the target instance, proceed to Task 2: Install the OCI-FSS-UTILS package on Oracle Linux or CentOS.

Task 2: Install the OCI-FSS-UTILS package on Oracle Linux or CentOS
  1. Open a terminal window on the destination instance.
  2. Install the package using the following command:

    sudo yum localinstall oci-fss-utils-<version>.rpm

The package creates a namespace called ns1 in your instance, which contains a default network interface for ethernet traffic. A network interface pair is created for each mount target.

After the package has finished installing, proceed to Task 3: Mount the file system with the encryption command.

Task 3: Mount the file system with the encryption command
  1. Open a terminal window in your instance.
  2. Create a mount point by typing the following, replacing yourmountpoint with the local directory from which you want to access your file system.

    sudo mkdir -p /mnt/yourmountpoint
    						
  3. Mount the file system using the following command:

    sudo mount -t oci-fss 10.x.x.x:/fs-export-path /mnt/yourmountpoint

    Replace 10.x.x.x: with the local subnet IP address assigned to your mount target, fs-export-path with the export path you specified when associating the file system with the mount target, and yourmountpoint with the path to the local mount point. The export path is the path to the file system (relative to the mount target IP address or hostname).

    Example output from the mount -t oci-fss command:

    Created symlink from /etc/systemd/system/multi-user.target.wants/oci-fss-2.service to /usr/lib/systemd/system/oci-fss-2.service.

    Each time you mount a file system using this command, a new oci-fss service is initiated with an incrementing sequence number between 2 and 255. For example, oci-fss-2.service, oci-fss-3.service, and so on.

    Tip

    You can use the resvport option to restrict the client to using a specific reserved port. For example:

    sudo mount -t oci-fss -o resvport=900 10.x.x.x:/fs-export-path /mnt/yourmountpoint

Managing In-transit Encryption

To auto-mount a file system

Auto-mount ensures that a file system is automatically re-mounted on an instance if it is rebooted.

  1. Open a terminal window on the instance. Mount the file system as described in .
  2. Open the /etc/fstab file for editing: 

    cd /etc
    						
    vi fstab
    						
  3. Add the following line to the fstab file:

    10.x.x.x:/fs-export-path /mnt/yourmountpoint oci-fss x-systemd.requires=oci-fss-init.service,defaults,nofail 0 0

    Replace 10.x.x.x: with the local subnet IP address assigned to your mount target, fs-export-path with the export path you specified when associating the file system with the mount target, and yourmountpoint with the path to the local mount point.

    Tip

    You can use the resvport option to restrict the client to using a specific reserved port. For example:

    10.x.x.x:/fs-export-path /mnt/yourmountpoint oci-fss x-systemd.requires=oci-fss-init.service,defaults,nofail,resvport=900 0 0
To unmount a file system

When you unmount a file system, you must use another oci-fss-utils command to ensure that the associated local network namespace is removed:

  1. Open a terminal window on the instance.
  2. Use the following command to unmount the file system:

    sudo umount -t oci-fss /mnt/yourmountpoint

    Replace yourmountpoint with the path to the local mount point.

To uninstall the OCI-FSS-UTILS package
  1. First, unmount all mounted file systems. For instructions, see To unmount a file system.
  2. Open a terminal window on the instance.
  3. Type the following command to uninstall the oci-fss package:

    sudo yum remove oci-fss-utils

Troubleshooting

If you experience issues with in-transit encryption, try the following techniques:

Verify that you have all the security list rules set up correctly for the mount target subnet.

In-transit encryption requires the following security list rules:

  • A stateful ingress rule allowing TCP traffic to a Destination Port Range of 2051.
  • A stateful egress rule allowing TCP traffic from a Source Port Range of 2051.

For more information and instructions, see Security Rules.

Verify that the oci-fss service is running for the mounted file system.

If it is not, restart the service.

To verify the service is running

When you mount a file system using the mount.oci-fss command, it creates a systemd-managed service called oci-fss<sequence_number>.service . <sequence_number> is an incrementing value between 2-255. An oci-fss service is created for every file system mounted using the command. The exact name of the service is displayed as output when the file system is mounted.

For example:

Created symlink from /etc/systemd/system/multi-user.target.wants/oci-fss-2.service to /usr/lib/systemd/system/oci-fss-2.service.
  1. Open a terminal window on the instance.
  2. Verify that the service is running using the following command:

    systemctl status oci-fss-<sequence_number>
To start the service
  1. Open a terminal window on the instance.
  2. Use the following command to start the service:

    systemctl start oci-fss-<sequence_number>

Verify that the namespace ns1 has been created and contains a network interface.

To verify the network namespace
  1. Open a terminal window on the instance.
  2. Use the following command to verify the namespace and see the network interface: 

    sudo ip netns exec ns1 ip link list

    You should see output displaying all the ethernet devices within namespace ns1. For example:

    1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    									link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    									3: v-peer1@if4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    								link/ether be:5b:35:2d:e9:54 brd ff:ff:ff:ff:ff:ff link-netnsid 0

Verify that IP forwarding is running on the instance.

Installing oci-fss-utils automatically turns on IP forwarding. However, you may have other processes running on the instance that disable it.

To verify that IP forwarding is running on the instance
  1. Open a terminal window on the instance.
  2. Use the following command to view the status of IP forwarding:

    # sysctl net.ipv4.ip_forward
    net.ipv4.ip_forward = 1

    An output value of 1 means that IP forwarding is enabled. This value is read from the /proc/sys/net/ipv4/ip_forward file.

    If the output value is 0, then IP forwarding is not enabled for the instance. Enable IP forwarding by following the instructions in To enable IP forwarding on the instance.

To enable IP forwarding on the instance

If IP forwarding is not currently enabled on the instance, you must enable it and make the change permanent.

  1. Open a terminal window on the instance.
  2. Type the following command to open the /etc/sysctl.conf file:

    sudo vi /etc/sysctl.conf
  3. Remove the # to uncomment this line: # net.ipv4.ip_forward=1. If the value is 0, change it to 1.
  4. Type :wq to save the file and exit the editor.

Use the tcpdump utility to analyze traffic between the oci-fss service and the NFS client.

To obtain information using TCPDUMP
  1. Open a terminal window on the instance.
  2. Type the following command:

    sudo ip netns exec ns1 tcpdump -i v-peer2 "port 2049"

Use the journalctl command to view any messages that may have been logged by systemd regarding the service.

To obtain information from the SYSTEMD journal
  1. Open a terminal window on the instance.
  2. Type the following command:

    journalctl -f -u oci-fss-<version>

-f displays the most recent journal entries, and prints new entries as they are appended to the journal.

-u specifies a specific systemd service unit. In this case, oci-fss-<sequence_number> is the specified unit. If no unit is specified, journalctl returns all systemd entries.