NEW! Dynamic Resource Balancer in Mirantis OpenStack for Kubernetes 24.2   |   Learn More


Understanding Kubernetes' new sidecar container feature

Eric Gregory - July 18, 2023

Last week, the Kubernetes project merged a new alpha feature enabling users to define “sidecar containers” in specifications. The new feature is intended to help define behavior for helper containers in multi-container pods that might assist in configuration, networking, log and metrics collection, and so on.

What is a sidecar container?

In theory, Kubernetes expects you to run one container per pod. In practice, many use cases call for multi-container pods—when you’re using certain service meshes, for example, you might have sidecars in almost all of your pods. 

Sometimes, helper containers are only used for initialization: configuring and managing secrets for the primary container, for example. Kubernetes has given users a way to define an initContainer for some time. This new feature ultimately provides more granularity to initContainer to reflect the particular requirements of sidecars, simplifying a common usage pattern and opening up some interesting design space for the future.

How does the sidecar container feature work?

In this new feature gate, sidecar containers are defined to…

  • Start up before other containers in a pod, since they probably need to initialize first. This is important for things like service meshes, where you want the sidecar to be ready to make networking connections for main container processes, as well as in logging, where you want the collector sidecar to be able to grab startup logs from the main container.

  • Keep running for the full lifespan of the pod, since they’re likely needed on an ongoing basis. In the cases of networking and metrics/logging, for example, you need the sidecar to run for as long as the main process does.

  • Never block the pod from being terminated, since these are only supporting the pod’s core functionality – without the new feature, a running sidecar container can stop a job from being completed even if a pod’s core task is done

On the arguably uglier side, in this alpha implementation you define a sidecar container by adding a restartPolicy field with the value Always to your initContainer spec. For example:

kind: Pod
  name: myapp-pod
  - name: init-myservice
    image: busybox:1.28
  - name: init-mydb
    image: busybox:1.28
  - name: istio-proxy
    image: istio/proxyv2:1.16.0
    args: ["proxy", "sidecar"]
    restartPolicy: Always
  - name: myapp-container
    image: busybox:1.28

In the spec above, init-myservice and init-mydb are standard issue initContainers, while the restartPolicy field set to Always makes istio-proxy a sidecar container. 

The Kubernetes Enhancement Proposal (KEP) for the new feature acknowledges this ostensible inelegance, noting that initContainer “is not a good fit for sidecar containers as they typically do more than initialization” and suggesting that “infrastructure containers” would be a better name that might be adopted in the future. The KEP explains the thinking behind the chosen structure by saying:

…it is important to have sidecar containers be defined among other init containers to be able to express the initialization order of containers.

A senior contributor added a little more flavor on Hacker News, noting:

The challenge with a separate attribute is that it is not forward compatible with new features we might add to pods around ordering and lifecycle. If we used a simple boolean, eventually we’d have to have it interact with other fields and deal with conflicting behaviors between what “sidecar” means and more flexibility. [...] We are leaving room for the possibility that init containers can fail the pod, and be parallelized, as well as regular containers having unique restartPolicies. Both of those would allow more control for workflow / job engines to break apart monolith containers and get better isolation.

In another comment, they added that the team wanted to...

…leave open more complex ordering of both init containers and sidecars (regular containers do not have a restart order). For instance, you might have a service mesh that needs a vault secret - those both might be sidecars, and you may need to ensure the vault sidecar starts first if both go down. Eventually we may want to add parallelism to that start order, and a separate field would prevent simple ordering from working now.

The KEP offers detailed insight on both the problem cases that prompted the feature as well as the interesting wider context that some organizations were running forks of Kubernetes in order to implement features similar to this. 

If you’re impatient to try out the new feature on a fresh test cluster, you’ll need to enable the SidecarContainers feature gate for the kubelet, kube-apiserver, kube-controller-manager, and kube-scheduler. The KEP offers useful details on default policies and implementation, and you can expect to see more discussion of the feature with the release of Kubernetes 1.28 in August.

Need Kubernetes support?
Whether you’re a Kubernetes beginner or cloud native pro, we’ve got your back. Our world-renowned Training can get your team up to speed quickly, while Mirantis Professional Services can help you design and stand up cloud native infrastructure, rapidly migrate legacy applications to containers, and more.

Choose your cloud native journey.

Whatever your role, we’re here to help with open source tools and world-class support.



Cloud Native & Coffee

Subscribe to our bi-weekly newsletter for exclusive interviews, expert commentary, and thought leadership on topics shaping the cloud native world.