NEW! Mirantis Academy -   Learn confidently with expert guidance and On-demand content.   Learn More


Security update: Encrypted overlay networks in Moby and Mirantis Container Runtime


During routine work on the Moby project, Mirantis discovered a set of security issues affecting all versions of Docker Engine which include the Swarm mode feature. This includes Mirantis Container Runtime and other projects downstream of Moby.

While most users are not directly impacted, these issues could impact some Swarm mode users’ overlay network traffic that was meant to be encrypted. 

We have created the following CVEs to track and identify these vulnerabilities:

These CVEs have been patched via coordinated security updates released April 4, 2023.

CVE-2023-28840 and CVE-2023-28841 may affect users with encrypted Swarm overlay networks on Red Hat Enterprise Linux 8 or Red Hat Enterprise Linux 9 (or derivatives such as CentOS, Rocky, and Alma).

CVE-2023-28840 can additionally affect any platform depending on how the iptables rules are laid out. All firewalld configurations which allow Swarm overlay networks to function are known to be affected.

CVE-2023-28842 affects users using encrypted overlay networks with only a single node (machine) attached, which we expect to be rare in production deployments.

VXLAN ports and overlay networks
These CVEs are very limited in exploitability so long as the VXLAN port is not exposed to untrusted traffic, and Swarm overlay traffic does not transit an untrusted network. In addition to the CVEs, we have created a security advisory for a related documentation-only issue:
GHSA-vwm3-crmr-xfxw: The Swarm VXLAN port may be exposed to attack due to ambiguous documentation
All Swarm users should evaluate their configuration for any unintended exposure of the VXLAN port and potential security implications, regardless of whether encrypted overlay networks are used.

How can I mitigate these issues?

Patches for the three CVEs are available in Mirantis Container Runtime 20.10.16 and 23.0.3. All users should update, and should strongly prefer the patches over any temporary mitigations.

To determine whether Swarm overlay networks are in use, you can use the following command (which requires jq):

docker network inspect $(docker network ls -q) | jq -r '.[] | select(.Scope=="swarm" and .Driver=="overlay") | select(.Containers // {} | with_entries(select(.key | test("-sbox$") | not)) | length > 0) | .Name'

This will print the name of any overlay network in active use on the local node. This will include the default ingress network (and any user-defined ingress networks), but only if they have a container attached (as the -sbox suffixed ‘containers’ are in fact virtual).

To assess an entire cluster, the command should be run across every node on the cluster, and the results aggregated. While Swarm-scoped networks are replicated across the entire cluster, the ‘actively used’ status is local to each node, and should be queried individually.

If you know that you are using overlay networks, you should ensure the VXLAN port (UDP 4789 by default, and called “Data Path Port” in docker info) is not accessible to untrusted network traffic. If encrypted overlay networks are in use over an untrusted network, ensure that IPSec ESP (protocol 50) is allowed through the boundary firewall.

If a temporary mitigation is required, users of Red Hat Enterprise Linux 8 and 9 (and derivatives, such as CentOS, Rocky, and Alma) can take the following alternate steps:

  • Check if the xt_u32 kernel module is available (e.g. modprobe xt_u32).

  • If it is unavailable:

    • On Enterprise Linux 8, install kernel-modules-extra and restart dockerd.

    • On Enterprise Linux 9, discontinue use of overlay networks where Swarm-level encryption is necessary for security.

  • Update to MCR 20.10.16 or 23.0.3 as soon as practical.

Discovery and disclosure timeline

The timeline below details the steps Mirantis has taken to fix and disclose these vulnerabilities:

  • Issues first identified and confirmed March 8th, 2023

  • Upstream partners alerted March 9th, 2023

  • Patch is code-complete and reviewed on March 22nd, 2023

  • Patch completes internal validations on March 23rd, 2023

  • Timeline and content for disclosure finalized with partners on March 30th, 2023

  • Disclosed and patched today, April 4th, 2023

If you have questions about these issues, we encourage you to contact us.

Choose your cloud native journey.

Whatever your role, we’re here to help with open source tools and world-class support.


Subscribe to our bi-weekly newsletter for exclusive interviews, expert commentary, and thought leadership on topics shaping the cloud native world.