Kustomize Tutorial: Creating a Kubernetes app out of multiple pieces

Kustomize is one of the most useful tools in the Kubernetes ecosystem for simplifying deployments, allowing you to create an entire Kubernetes application out of individual pieces -- without touching the YAML configuration files for the individual components.

In this tutorial, we’ll set up kustomize and explore how it works with a sample WordPress deployment. Then we’ll take a look at using kustomize with kubectl, and consider the benefits of kustomize for deployments at scale.

Table of Contents

• What is Kustomize?
• Benefits of Using Kustomize
• Installing Kustomize
• Combining Specs
• Managing Multiple Directories
• Changing Parameters for a Component Using Kustomize Overlays
  • Creating a Kustomize Patch
• Using Kubectl with Kustomize
  • Example: Kustomize Secret Generator
• Conclusion

What is Kustomize?

Kustomize is a command-line configuration manager for Kubernetes objects. Integrated with kubectl since 1.14, it allows you to make declarative changes to your configurations without touching a template. For example, you can combine pieces from different sources, keep your customizations -- or kustomizations, as the case may be -- in source control, and create overlays for specific situations.

Kustomize enables you to do that by creating a file that ties everything together, or optionally includes "overrides" for individual parameters.

Benefits of Using Kustomize

By the end of this tutorial, you will understand the key benefits of kustomize, including:

• Declarative templating - Kustomize uses the same declarative approach to configuration as Kubernetes itself.
• Easy-to-track codebase consolidation - Making selective patches through a centralized file that can be placed in version control streamlines tracking and management.
• Integration with kubectl - Kustomize is now part of the core Kubernetes toolkit, so there are no additional dependencies or requirements to worry about.

Now, let’s see kustomize in action.

See how Mirantis Enterprise Kubernetes solutions can help deploy cloud-native applications at scale in any environment.

Installing Kustomize

Note: If you’re using the latest release of kubectl, you already have kustomize built-in. If you’re using a version of kubectl prior to 1.14 or simply wish to use the standalone kustomize tool, you can follow the instructions below. In some cases, commands will differ between the standalone and integrated versions -- we’ll be sure to call out those differences as they arise. And in the end, we’ll consider some of the benefits of using the kubectl-integrated version.

How to install kustomize on Linux

You can install standalone kustomize from go:

go get sigs.k8s.io/kustomize

Or you can install the latest from source:

opsys=linux  # or darwin, or windows
curl -s https://api.github.com/repos/kubernetes-sigs/kustomize/releases/latest |\
  grep browser_download |\
  grep $opsys |\
  cut -d '"' -f 4 |\
  xargs curl -O -L
mv kustomize_*_${opsys}_amd64 kustomize
chmod u+x kustomize

How to install kustomize on macOS

If you’re on macOS, you can use Homebrew to install the standalone kustomize tool:

brew install kustomize

How to install kustomize on Windows

Windows users can use the Chocolatey package manager:

choco install kustomize

Check your kustomize version

To make sure it's installed, go ahead and check the version:

$ kustomize version
Version: {KustomizeVersion:v2.0.2 GitCommit:b67179e951ebe11d00125bdf3c2670e88dca8817 BuildDate:2019-02-25T21:36:48+00:00 GoOs:darwin GoArch:amd64}

Note that this will only work for the standalone tool -- otherwise, you can simply use kubectl version to verify that you are using 1.14 or later.

Combining Specs

One of the most common uses for kustomize is to take multiple objects and combine them into a single resource with common labels. For this kustomize example, let's say you want to deploy WordPress, and you find two Kubernetes manifests on the web. Let's start by creating a directory to serve as a base directory:

KUSTOM_HOME=$(mktemp -d)
BASE=$KUSTOM_HOME/base
mkdir $BASE
WORDPRESS_HOME=$BASE/wordpress
mkdir $WORDPRESS_HOME
cd $WORDPRESS_HOME

Now let's look at the manifests. One is for the deployment of WordPress itself. Let's save that as $WORDPRESS_HOME/deployment.yaml.

apiVersion: apps/v1beta2 
kind: Deployment
metadata:
  name: wordpress
  labels:
    app: wordpress
spec:
  selector:
    matchLabels:
      app: wordpress
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: wordpress
    spec:
      containers:
      - image: wordpress:4.8-apache
        name: wordpress
        ports:
        - containerPort: 80
          name: wordpress
        volumeMounts:
        - name: wordpress-persistent-storage
          mountPath: /var/www/html
      volumes:
      - name: wordpress-persistent-storage
        emptyDir: {}

The second is a service to expose it. We'll save it as $WORDPRESS_HOME/service.yaml.

apiVersion: v1
kind: Service
metadata:
  name: wordpress
  labels:
    app: wordpress
spec:
  ports:
    - port: 80
  selector:
    app: wordpress
  type: LoadBalancer

That all seems reasonable, and if we can put both of these files in a directory called wordpress and run:

$ kubectl apply -f $WORDPRESS_HOME
deployment.apps "wordpress" created
service "wordpress" created

Before we move on, let's go ahead and clean that up:

$ kubectl delete -f $WORDPRESS_HOME

Now if you look at the definitions, you'll notice that both resources show an app label of wordpress. If you wanted to deploy them with a label of, say, app:my-wordpress, you'd either have to add parameters on the command line or edit the files -- which does away with the advantage of reusing the files.

Instead, we can use kustomize to combine them into a single file -- including the desired app label -- without changes to the originals.

We start by creating the file $WORDPRESS_HOME/kustomization.yaml and adding the following:

commonLabels:
  app: my-wordpress
resources:
- deployment.yaml
- service.yaml

This is a very simple file that just says that we want to add a common label -- app: my-wordpress -- to the resources defined in deployment.yaml and service.yaml. Now we can use the kustomize build command to actually build the new YAML. If you’re using standalone kustomize, you’ll run:

$ kustomize build $WORDPRESS_HOME

If you’re using kubectl kustomize, you’ll instead run:

$ kubectl kustomize $WORDPRESS_HOME

The output is the concatenation of YAML documents for all of the resources we specified, with the common labels added:

apiVersion: v1
kind: Service
metadata:
  labels:
    app: my-wordpress
  name: wordpress
spec:
  ports:
  - port: 80
  selector:
    app: my-wordpress
  type: LoadBalancer
---
apiVersion: apps/v1beta2
kind: Deployment
metadata:
  labels:
    app: my-wordpress
  name: wordpress
spec:
  selector:
    matchLabels:
      app: my-wordpress
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: my-wordpress
    spec:
      containers:
      - image: wordpress:4.8-apache
        name: wordpress
        ports:
        - containerPort: 80
          name: wordpress
        volumeMounts:
        - mountPath: /var/www/html
          name: wordpress-persistent-storage
      volumes:
      - emptyDir: {}
        name: wordpress-persistent-storage

You can output this to a file, or pipe it directly into kubectl -- we'll cover both methods in a moment.

Managing Multiple Directories

Now, that's all great, but WordPress won't run without access to a database. Fortunately, we have a set of similar files for setting up MySQL. Unfortunately, they have the same names as the files we have for WordPress, and remember, we don't want to have to alter any of the files, so we need a way to pull from multiple directories. We can do that by creating multiple "bases."

So we'll start by creating a new directory:

MYSQL_HOME=$BASE/mysql
mkdir $MYSQL_HOME
cd $MYSQL_HOME

We'll add three files to it. The first is a $MYSQL_HOME/deployment.yaml:

apiVersion: apps/v1beta2 
kind: Deployment
metadata:
  name: mysql
  labels:
    app: mysql
spec:
  selector:
    matchLabels:
      app: mysql
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: mysql
    spec:
      containers:
      - image: mysql:5.6
        name: mysql
        env:
        - name: MYSQL_ROOT_PASSWORD
          valueFrom:
            secretKeyRef:
              name: mysql-pass
              key: password
        ports:
        - containerPort: 3306
          name: mysql
        volumeMounts:
        - name: mysql-persistent-storage
          mountPath: /var/lib/mysql
      volumes:
      - name: mysql-persistent-storage
        emptyDir: {}

The second is $MYSQL_HOME/service.yaml:

apiVersion: v1
kind: Service
metadata:
  name: mysql
  labels:
    app: mysql
spec:
  ports:
    - port: 3306
  selector:
    app: mysql

And finally $MYSQL_HOME/secret.yaml to hold the database username and password:

apiVersion: v1
kind: Secret
metadata:
  name: mysql-pass
type: Opaque
data:
  # Default password is "admin".
  password: YWRtaW4=

And we'll add a kustomization file, $MYSQL_HOME/kustomization.yaml, then together:

resources:
- deployment.yaml
- service.yaml
- secret.yaml

Now we need to tie the two directories together. First let's clean up $WORDPRESS_HOME/kustomization.yaml to remove the labels so that it only references the resources:

resources:
- deployment.yaml
- service.yaml

Now we need to add a new kustomization file to the base directory at $BASE/kustomization.yaml:

commonLabels:
  app: my-wordpress
bases:
- ./wordpress
- ./mysql

So we've moved our labels declaration out to this main file and defined the two base directories we're working with. Now if we run the build...

kubectl kustomize $BASE

Or the standalone command...

kustomize build $BASE

We can see that all of the files are gathered and the label is added to all of them:

apiVersion: v1
data:
  password: YWRtaW4=
kind: Secret
metadata:
  labels:
    app: my-wordpress
  name: mysql-pass
type: Opaque
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: my-wordpress
  name: mysql
spec:
  ports:
  - port: 3306
  selector:
    app: my-wordpress
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: my-wordpress
  name: wordpress
spec:
  ports:
  - port: 80
  selector:
    app: my-wordpress
  type: LoadBalancer
---
apiVersion: apps/v1beta2
kind: Deployment
metadata:
  labels:
    app: my-wordpress
  name: mysql
spec:
  selector:
    matchLabels:
      app: my-wordpress
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: my-wordpress
    spec:
      containers:
      - env:
        - name: MYSQL_ROOT_PASSWORD
          valueFrom:
            secretKeyRef:
              key: password
              name: mysql-pass
        image: mysql:5.6
        name: mysql
        ports:
        - containerPort: 3306
          name: mysql
        volumeMounts:
        - mountPath: /var/lib/mysql
          name: mysql-persistent-storage
      volumes:
      - emptyDir: {}
        name: mysql-persistent-storage
---
apiVersion: apps/v1beta2
kind: Deployment
metadata:
  labels:
    app: my-wordpress
  name: wordpress
spec:
  selector:
    matchLabels:
      app: my-wordpress
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: my-wordpress
    spec:
      containers:
      - image: wordpress:4.8-apache
        name: wordpress
        ports:
        - containerPort: 80
          name: wordpress
        volumeMounts:
        - mountPath: /var/www/html
          name: wordpress-persistent-storage
      volumes:
      - emptyDir: {}
       name: wordpress-persistent-storage

OK, so now we've gathered multiple components...but what happens if we need to change something?

Changing Parameters for a Component Using Kustomize Overlays

Now, we're almost ready, but we do have one more problem. While we're deploying our production system to a cloud provider that supports LoadBalancer, we're developing on our laptop so we need our services to be type: NodePort. Fortunately, we can solve this problem with overlays.

Overlays enable us to take the base YAML and selectively change pieces of it. For example, we're going to create an overlay that includes a patch to change the Services to NodePort type services.

Creating a kustomize patch

It's important that the overlay isn't in the same directory as the base files, so we'll create it in an adjacent directory, then add a dev subdirectory.

OVERLAY_HOME=$BASE/../overlays
mkdir $OVERLAY_HOME
DEV_HOME=$OVERLAY_HOME/dev
mkdir $DEV_HOME
cd $DEV_HOME

Next we want to create the patch file, $DEV_HOME/localserv.yaml:

apiVersion: v1
kind: Service
metadata:
  name: wordpress
spec:
  type: NodePort
---
apiVersion: v1
kind: Service
metadata:
  name: mysql
spec:
  type: NodePort

Notice that we've included the bare minimum of information here; just enough to identify each service we want to change, and then specify the change that we want to make -- in this case, the type.

Now we need to create the $DEV_HOME/kustomization.yaml file to tie all of this together:

bases:
- ../../base
patchesStrategicMerge:
- localserv.yaml

Notice that this is really very simple; we're pointing at our original base directory, and specifying the patch(es) that we want to add.

Now we can go ahead and build the original $BASE, and see that it's untouched.

We still have LoadBalancer services:

...
spec:
  ports:
  - port: 3306
  selector:
    app: my-wordpress
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: my-wordpress
  name: wordpress
spec:
  ports:
  - port: 80
  selector:
    app: my-wordpress
  type: LoadBalancer
---
apiVersion: apps/v1beta2
kind: Deployment
metadata:
  labels:
...

But if we build the overlay instead, we should now have NodePort services.

kubectl kustomize $DEV_HOME

Or the standalone command…

$ kustomize build $DEV_HOME

Notice that everything is unchanged by the kustomize patch except the type:

...
  name: mysql-pass
type: Opaque
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: my-wordpress
  name: mysql
spec:
  ports:
  - port: 3306
  selector:
    app: my-wordpress
  type: NodePort
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: my-wordpress
  name: wordpress
spec:
  ports:
  - port: 80
  selector:
    app: my-wordpress
  type: NodePort
---
apiVersion: apps/v1beta2
kind: Deployment
metadata:
...

Now let's see how we can make use of these objects in kubectl.

Using Kustomize with Kubectl

Now, all of this is great, but saving it to a file then running the file seems like a little bit of overkill. Fortunately there are two ways we can feed this in directly. One is to simply pipe it in, as you would do with any other Linux program:

kustomize build $DEV_HOME | kubectl apply -f -

Or if you're using Kubernetes 1.14 or above, you can simply use the -k parameter:

kubectl apply -k $DEV_HOME
secret "mysql-pass" created
service "mysql" created
service "wordpress" created
deployment.apps "mysql" created
deployment.apps "wordpress" created

Example: Kustomize Secret Generator

This shortened process may not seem like a big deal at first, but consider an example from the documentation, showing the old way of doing things:

kubectl create secret docker-registry myregistrykey --docker-server=DOCKER_REGISTRY_SERVER --docker-username=DOCKER_USER --docker-password=DOCKER_PASSWORD --docker-email=DOCKER_EMAIL
secret/myregistrykey created.

Versus the new way, where we create a kustomization.yaml file:

secretGenerator:
- name: myregistrykey
 type: docker-registry
 literals:
 - docker-server=DOCKER_REGISTRY_SERVER
 - docker-username=DOCKER_USER
 - docker-password=DOCKER_PASSWORD
 - docker-email=DOCKER_EMAIL
EOF

Then simply reference it using the -k parameter:

$ kubectl apply -k .
secret/myregistrykey-66h7d4d986 created

Conclusion

Considering that kustomization.yaml files can be stored in repos and subject to version control, where they can be tracked and more easily managed, this provides a much cleaner way to manage your infrastructure as code.

When you’re working across a variety of environments with distinct requirements, kustomize can help your team share the same base files with only the necessary changes overlaid simply and selectively. Remember our example patch, swapping the dev environment’s NodePort type for a production-ready LoadBalancer. We can continue our work on that app with confidence that our configs remain the same across environments -- and if we need to debug a problem, we won’t have to comb through separate codebases.

Pair this with kustomize’s integration into kubectl and declarative nature -- all perfectly suited to a K8s toolkit -- and it’s clear that kustomize can serve as a powerful tool for deploying our Kubernetes infrastructure more quickly and cleanly.