Scaling with Kubernetes DaemonSets
We're used to thinking about scaling from the point of view of a deployment; we want it to scale up under different conditions, so it looks for appropriate nodes, and puts pods on them. DaemonSets, on the other hand, take a different tack: any time you have a node that belongs to the set, it runs the pods you specify. They are useful for running background services in a web or storage server, such as system resource monitoring or logging. DaemonSets are also crucial if you want to run a specific service before any other pods start.
For example, you might create a DaemonSet that requests Kubernetes run Nginx any time you create a node with the label app=webserver . Let's take a look at how that works.
Creating a Kubernetes DaemonSetLet's start by looking at a sample YAML file to define a Kubernetes DaemonSet:
apiVersion: extensions/v1beta1 kind: DaemonSet metadata: name: frontend spec: template: metadata: labels: app: frontend-webserver spec: nodeSelector: app: frontend-node containers: - name: webserver image: nginx ports: - containerPort: 80Here we're creating a DaemonSet called frontend. As with a ReplicationController, pods launched by the DaemonSet are given the label specified in the spec.template.metadata.labels property -- in this case, app=frontend-webserver.
The template.spec itself has two important parts: the nodeSelector and the containers. The containers are fairly self-evident (see our discussion of ReplicationControllers if you need a refresher) but the interesting part here is the nodeSelector.
The nodeSelector tells Kubernetes which nodes are part of the set and should run the specified containers. In other words, these pods are deployed automatically; there's no input at all from the scheduler, so schedulability of a node isn't taken into account. On the other hand, Daemon Sets are a great way to deploya collection of pods that need to be running before other objects.
Let's go ahead and create the Kubernetes DaemonSet. Create a configuration file called ds.yaml with the definition in it and run the command:
$ kubectl create -f ds.yaml daemonset "datastore" createdNow let's see how we can instruct a Kubernetes Daemon to start or remove pods automatically..
Scale Up or Scale Down a DaemonSetIf we check to see if the pods have been deployed, we'll see that they haven't:
$ kubectl get pods NAME READY STATUS RESTARTS AGEThat's because we don't yet have any nodes that are part of our DaemonSet. If we look at the nodes we do have ...
$ kubectl get nodes NAME STATUS AGE 10.0.10.5 Ready 75d 10.0.10.7 Ready 75dWe can go ahead and add at least one of them by adding the app=frontend-node label:
$kubectl label node 10.0.10.5 app=frontend-node node "10.0.10.5" labeledNow if we monitor the list of pods again...
$ kubectl get pods NAME READY STATUS RESTARTS AGE frontend-7nfxo 1/1 Running 0 19sWe can see that the pod was started without us taking any additional action.
Now we have a single webserver running. You now can, using the command kubectl, scale DaemonSet pods up by adding a new node as shown in the example:
$ kubectl label node 10.0.10.7 app=frontend-node node "10.0.10.7" labeledIf we check the list of pods again, we can see that a new one was automatically started:
$ kubectl get pods NAME READY STATUS RESTARTS AGE frontend-7nfxo 1/1 Running 0 1m frontend-rp9bu 1/1 Running 0 35sIf we remove a node from the DaemonSet, any related pods are automatically terminated:
$ kubectl label node 10.0.10.5 --overwrite app=backend node "10.0.10.5" labeled
$ kubectl get pods NAME READY STATUS RESTARTS AGE frontend-rp9bu 1/1 Running 0 1m
Updating Daemon Sets, and improvements in Kubernetes 1.6OK, so how do we update a running DaemonSet? Well, as of Kubernetes version 1.5, the answer is "you don't." Currently, it's possible to change the template of a DaemonSet, but it won't affect the pods that are already running.
Starting in Kubernetes 1.6, however, you will be able to do rolling updates with a Kubernetes DaemonSets. You'll have to set the updateStrategy, as in:
apiVersion: extensions/v1beta1 kind: DaemonSet metadata: name: frontend spec: updateStrategy: RollingUpdate maxUnavailable: 1 minReadySeconds: 0 template: metadata: labels: app: frontend-webserver spec: nodeSelector: app: frontend-node containers: - name: webserver image: nginx ports: - containerPort: 80Once you've done that, you can make changes and they'll propagate to the running pods. For example, you can change the image on which the containers are based. For example:
$kubectl set image ds/frontend webserver=httpdIf you want to make more substantive changes, you can edit or patch the Daemon Set:
kubectl edit ds/frontendor
kubectl patch ds/frontend -p=ds-changes.yaml(Obviously you would use your own DaemonSet names and files!)
So that's the basics of working with DaemonSets. What else would you like to learn about them? Let us know in the comments below.