Understanding Kubernetes' new sidecar container feature
Last week, the Kubernetes project merged a new alpha feature enabling users to define “sidecar containers” in specifications. The new feature is intended to help define behavior for helper containers in multi-container pods that might assist in configuration, networking, log and metrics collection, and so on.
What is a sidecar container?
In theory, Kubernetes expects you to run one container per pod. In practice, many use cases call for multi-container pods—when you’re using certain service meshes, for example, you might have sidecars in almost all of your pods.
Sometimes, helper containers are only used for initialization: configuring and managing secrets for the primary container, for example. Kubernetes has given users a way to define an initContainer for some time. This new feature ultimately provides more granularity to initContainer to reflect the particular requirements of sidecars, simplifying a common usage pattern and opening up some interesting design space for the future.
How does the sidecar container feature work?
In this new feature gate, sidecar containers are defined to…
Start up before other containers in a pod, since they probably need to initialize first. This is important for things like service meshes, where you want the sidecar to be ready to make networking connections for main container processes, as well as in logging, where you want the collector sidecar to be able to grab startup logs from the main container.
Keep running for the full lifespan of the pod, since they’re likely needed on an ongoing basis. In the cases of networking and metrics/logging, for example, you need the sidecar to run for as long as the main process does.
Never block the pod from being terminated, since these are only supporting the pod’s core functionality – without the new feature, a running sidecar container can stop a job from being completed even if a pod’s core task is done
On the arguably uglier side, in this alpha implementation you define a sidecar container by adding a
restartPolicy field with the value
Always to your
initContainer spec. For example:
kind: Pod metadata: name: myapp-pod spec: initContainers: - name: init-myservice image: busybox:1.28 - name: init-mydb image: busybox:1.28 - name: istio-proxy image: istio/proxyv2:1.16.0 args: ["proxy", "sidecar"] restartPolicy: Always containers: - name: myapp-container image: busybox:1.28
In the spec above, init-myservice and init-mydb are standard issue initContainers, while the
restartPolicy field set to
Always makes istio-proxy a sidecar container.
The Kubernetes Enhancement Proposal (KEP) for the new feature acknowledges this ostensible inelegance, noting that initContainer “is not a good fit for sidecar containers as they typically do more than initialization” and suggesting that “infrastructure containers” would be a better name that might be adopted in the future. The KEP explains the thinking behind the chosen structure by saying:
…it is important to have sidecar containers be defined among other init containers to be able to express the initialization order of containers.
A senior contributor added a little more flavor on Hacker News, noting:
The challenge with a separate attribute is that it is not forward compatible with new features we might add to pods around ordering and lifecycle. If we used a simple boolean, eventually we’d have to have it interact with other fields and deal with conflicting behaviors between what “sidecar” means and more flexibility. [...] We are leaving room for the possibility that init containers can fail the pod, and be parallelized, as well as regular containers having unique restartPolicies. Both of those would allow more control for workflow / job engines to break apart monolith containers and get better isolation.
In another comment, they added that the team wanted to...
…leave open more complex ordering of both init containers and sidecars (regular containers do not have a restart order). For instance, you might have a service mesh that needs a vault secret - those both might be sidecars, and you may need to ensure the vault sidecar starts first if both go down. Eventually we may want to add parallelism to that start order, and a separate field would prevent simple ordering from working now.
The KEP offers detailed insight on both the problem cases that prompted the feature as well as the interesting wider context that some organizations were running forks of Kubernetes in order to implement features similar to this.
If you’re impatient to try out the new feature on a fresh test cluster, you’ll need to enable the SidecarContainers feature gate for the kubelet, kube-apiserver, kube-controller-manager, and kube-scheduler. The KEP offers useful details on default policies and implementation, and you can expect to see more discussion of the feature with the release of Kubernetes 1.28 in August.
Need Kubernetes support?
Whether you’re a Kubernetes beginner or cloud native pro, we’ve got your back. Our world-renowned Training can get your team up to speed quickly, while Mirantis Professional Services can help you design and stand up cloud native infrastructure, rapidly migrate legacy applications to containers, and more.