Kustomize Tutorial, Part 1: Creating a Kubernetes app out of multiple pieces

What is Kustomize? It is one of the most useful tools in the Kubernetes ecosystem for simplifying deployments. Kustomize lets you lets you create an entire Kubernetes application out of individual pieces — without touching the YAML configuration filesfor the individual components.  For example, you can combine pieces from different sources, keep your customizations — or kustomizations, as the case may be — in source control, and create overlays for specific situations. And it is part of Kubernetes 1.14 or later.

Kustomize enables you to do that by creating a file that ties everything together, or optionally includes “overrides” for individual parameters.

Let’s see how it works in this mini Kustomize tutorial.

Installing Kustomize

The first step, of course, is to install Kustomize.  It’s easy if you’re on MacOS:

brew install kustomize

If not, you can install from go:

go get sigs.k8s.io/kustomize

Or you can install the latest from source:

opsys=linux  # or darwin, or windows
curl -s https://api.github.com/repos/kubernetes-sigs/kustomize/releases/latest |\
  grep browser_download |\
  grep $opsys |\
  cut -d '"' -f 4 |\
  xargs curl -O -L
mv kustomize_*_${opsys}_amd64 kustomize
chmod u+x kustomize

To make sure it’s installed, go ahead and check the version:

$ kustomize version
Version: {KustomizeVersion:v2.0.2 GitCommit:b67179e951ebe11d00125bdf3c2670e88dca8817 BuildDate:2019-02-25T21:36:48+00:00 GoOs:darwin GoArch:amd64}

OK, now we’re ready to go!

Combining specs into a single app

One of the most common uses for Kustomize is to take multiple objects and combine them into a single resource with common labels.  For this Kustomize example, let’s say you want to deploy WordPress, and you find two Kubernetes manifests on the web. Let’s start by creating a directory to serve as a base directory:

KUSTOM_HOME=$(mktemp -d)
BASE=$KUSTOM_HOME/base
mkdir $BASE
WORDPRESS_HOME=$BASE/wordpress
mkdir $WORDPRESS_HOME
cd $WORDPRESS_HOME

Now let’s look at the manifests.  One is for the deployment of WordPress itself. Let’s save that as $WORDPRESS_HOME/deployment.yaml.

apiVersion: apps/v1beta2 
kind: Deployment
metadata:
  name: wordpress
  labels:
    app: wordpress
spec:
  selector:
    matchLabels:
      app: wordpress
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: wordpress
    spec:
      containers:
      - image: wordpress:4.8-apache
        name: wordpress
        ports:
        - containerPort: 80
          name: wordpress
        volumeMounts:
        - name: wordpress-persistent-storage
          mountPath: /var/www/html
      volumes:
      - name: wordpress-persistent-storage
        emptyDir: {}

The second is a service to expose it. We’ll save it as $WORDPRESS_HOME/service.yaml.

apiVersion: v1
kind: Service
metadata:
  name: wordpress
  labels:
    app: wordpress
spec:
  ports:
    - port: 80
  selector:
    app: wordpress
  type: LoadBalancer

That all seems reasonable, and if we can put both of these files in a directory called wordpress and run:

$ kubectl apply -f $WORDPRESS_HOME
deployment.apps "wordpress" created
service "wordpress" created

Before we move on, let’s go ahead and clean that up:

$ kubectl delete -f $WORDPRESS_HOME

Now if you look at the definitions, you’ll notice that both resources show an app label of wordpress.  If you wanted to deploy them with a label of, say, app:my-wordpress, you’d either have to add parameters on the command line or edit the files — which does away with the advantage of reusing the files.

Instead, we can use Kustomize to combine them into a single file — including the desired app label — without changes to the originals.

We start by creating the file $WORDPRESS_HOME/kustomization.yaml and adding the following:

commonLabels:
  app: my-wordpress
resources:
- deployment.yaml
- service.yaml

This is a very simple file that just says that we want to add a common label — app: my-wordpress — to the resources defined in deployment.yaml and service.yaml.  Now we can use  the Kustomize build command to actually build the new YAML:

$ kustomize build $WORDPRESS_HOME
apiVersion: v1
kind: Service
metadata:
  labels:
    app: my-wordpress
  name: wordpress
spec:
  ports:
  - port: 80
  selector:
    app: my-wordpress
  type: LoadBalancer
---
apiVersion: apps/v1beta2
kind: Deployment
metadata:
  labels:
    app: my-wordpress
  name: wordpress
spec:
  selector:
    matchLabels:
      app: my-wordpress
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: my-wordpress
    spec:
      containers:
      - image: wordpress:4.8-apache
        name: wordpress
        ports:
        - containerPort: 80
          name: wordpress
        volumeMounts:
        - mountPath: /var/www/html
          name: wordpress-persistent-storage
      volumes:
      - emptyDir: {}
        name: wordpress-persistent-storage

The output is the concatenation of YAML documents for all of the resources we specified, with the common labels added.  You can output that to a file, or pipe it directly into kubectl. (We’ll cover that in Using Kustomize with kubectl.

How to manage Multiple directories

Now, that’s all great, but WordPress won’t run without access to a database.  Fortunately, we have a set of similar files for setting up MySQL. Unfortunately, they have the same names as the files we have for WordPress, and remember, we don’t want to have to alter any of the files, so we need a way to pull from multiple directories.  We can do that by creating multiple “bases”.

So we’ll start by creating a new directory:

MYSQL_HOME=$BASE/mysql
mkdir $MYSQL_HOME
cd $MYSQL_HOME

We’ll add three files to it. The first is a $MYSQL_HOME/deployment.yaml:

apiVersion: apps/v1beta2 
kind: Deployment
metadata:
  name: mysql
  labels:
    app: mysql
spec:
  selector:
    matchLabels:
      app: mysql
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: mysql
    spec:
      containers:
      - image: mysql:5.6
        name: mysql
        env:
        - name: MYSQL_ROOT_PASSWORD
          valueFrom:
            secretKeyRef:
              name: mysql-pass
              key: password
        ports:
        - containerPort: 3306
          name: mysql
        volumeMounts:
        - name: mysql-persistent-storage
          mountPath: /var/lib/mysql
      volumes:
      - name: mysql-persistent-storage
        emptyDir: {}

The second is $MYSQL_HOME/service.yaml:

apiVersion: v1
kind: Service
metadata:
  name: mysql
  labels:
    app: mysql
spec:
  ports:
    - port: 3306
  selector:
    app: mysql

And finally $MYSQL_HOME/secret.yaml to hold the database username and password:

apiVersion: v1
kind: Secret
metadata:
  name: mysql-pass
type: Opaque
data:
  # Default password is "admin".
  password: YWRtaW4=

And we’ll add a Kustomization file, $MYSQL_HOME/kustomization.yaml, them together:

resources:
- deployment.yaml
- service.yaml
- secret.yaml

Now we need to tie the two directories together.  First let’s clean up $WORDPRESS_HOME/kustomization.yaml to remove the labels so that it only references the resources:

resources:
- deployment.yaml
- service.yaml

Now we need to add a new kustomization file to the base directory at $BASE/kustomization.yaml:

commonLabels:
  app: my-wordpress
bases:
- ./wordpress
- ./mysql

So we’ve moved our labels declaration out to this main file and defined the two base directories we’re working with.  Now if we run the build ….

kustomize build $BASE

We can see that all of the files are gathered and the label is added to all of them:

apiVersion: v1
data:
  password: YWRtaW4=
kind: Secret
metadata:
  labels:
    app: my-wordpress
  name: mysql-pass
type: Opaque
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: my-wordpress
  name: mysql
spec:
  ports:
  - port: 3306
  selector:
    app: my-wordpress
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: my-wordpress
  name: wordpress
spec:
  ports:
  - port: 80
  selector:
    app: my-wordpress
  type: LoadBalancer
---
apiVersion: apps/v1beta2
kind: Deployment
metadata:
  labels:
    app: my-wordpress
  name: mysql
spec:
  selector:
    matchLabels:
      app: my-wordpress
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: my-wordpress
    spec:
      containers:
      - env:
        - name: MYSQL_ROOT_PASSWORD
          valueFrom:
            secretKeyRef:
              key: password
              name: mysql-pass
        image: mysql:5.6
        name: mysql
        ports:
        - containerPort: 3306
          name: mysql
        volumeMounts:
        - mountPath: /var/lib/mysql
          name: mysql-persistent-storage
      volumes:
      - emptyDir: {}
        name: mysql-persistent-storage
---
apiVersion: apps/v1beta2
kind: Deployment
metadata:
  labels:
    app: my-wordpress
  name: wordpress
spec:
  selector:
    matchLabels:
      app: my-wordpress
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: my-wordpress
    spec:
      containers:
      - image: wordpress:4.8-apache
        name: wordpress
        ports:
        - containerPort: 80
          name: wordpress
        volumeMounts:
        - mountPath: /var/www/html
          name: wordpress-persistent-storage
      volumes:
      - emptyDir: {}
       name: wordpress-persistent-storage

OK, so now we’ve gathered multiple components, but what happens if we need to change something?  Let’s look at that in part 2.