How to deploy Spinnaker on Kubernetes: a quicker and dirtier guide

Earlier this year we gave you a quick and dirty guide to deploying Spinnaker on Kubernetes; today we’re going to give you an even easier way.  Two easier ways, in fact. One is incredibly easy but a little harder to control, the other is almost as easy but gives you all the control you could possibly need.

The “point and shoot” method: Spinnaker quick install

The easiest, quickest, dirtiest way to deploy Spinnaker to Kubernetes is to use the “quick install” Kubernetes manifest maintained by the Spinnaker community.  For example:

kubectl apply -f https://spinnaker.io/downloads/kubernetes/quick-install.yml

The advantages of using this method include the fact that it gives you a serviceable install without having to do … well, much of anything except having a working Kubernetes cluster. The disadvantage is that you have to work a bit harder if you want to do any customization, such as changing the storage or adding repositories or additional cloud providers.

To customize this Spinnaker install, you have two choices:

  • Download and edit your own copy of the manifest, or
  • Find and log into the deployed halyard pod and perform your edits from there, then redeploy Spinnaker.

Fortunately, there’s an easier way.

The “professional camera” method: deployment by Spinnaker Helm chart

If you want to have any significant control over your Spinnaker deployment, you’re better off using a method that gives you easy access to the parameters that control it, such as a Kubernetes Helm chart.

In this article, I’m going to show you how to deploy Spinnaker using Helm, including how to use an external values.yaml file to customize the parameters used for deployment. Simply put, there are three steps:

  1. Install and initialize Helm.
  2. helm repo add mirantisworkloads https://mirantisworkloads.storage.googleapis.com
  3. helm install mirantisworkloads/halyard

And if you’re willing to take the defaults, that’s all you need to do. Let me explain.

What’s actually going on

Helm is a native Kubernetes software packager; in this case, we’re using it to deploy the basic infrastructure you need to effectively use Spinnaker, such as Minio for storage and Jenkins for continuous integration. Helm will also install Halyard, which will then go on to install Spinnaker.

As part of this process, the Helm chart gives Halyard a default configuration, and once it’s up, calls

hal deploy apply

to actually deploy Spinnaker for you.

Fortunately, you have the option of overriding those default values so that Spinnaker gets deployed the way you want it. You can also use the chart to add supplemental files such as a kubeconfig to the container as it gets spawned.

Halyard chart configuration and overrides

If you want to look at the default values used by Helm (and subsequently by Halyard) you can find the Halyard chart source files on Github, here:

https://github.com/Mirantis/k8s-apps/tree/master/charts/halyard

If you decide you want to change any of those values, create a new file called values.yaml and add the appropriate YAML to it.

For example, by default the chart assumes that your Kubernetes cluster has RBAC enabled , and that it supports the LoadBalancer service type. You can disable RBAC and change the type for the Spinnaker Gate and Deck services by overriding these chart values, like so:

rbac:
   enabled: false

service: ui:
   type: NodePort
api:
   type: NodePort

Minio and Jenkins deployment is also optional. Both are the subcharts of the Halyard chart, and you can disable them by setting:

minio:
  deployChart: false

jenkins:
  deployChart: false

Of course, if you disable Minio, you’ll have to configure some storage provider instead. You can do that by configuring Halyard itself.

Adding the halconfig

Now we’ve arrived at the halyard configuration itself. You can provide a halconfig and any number of arbitrary files by specifying their contents in the values.yaml file.

You can add the contents of the halconfig file you’d like to change. Here’s an example of halyard configuration with Kubernetes v2 provider enabled:

halconfig: |
  currentDeployment: default
  deploymentConfigurations:
  - name: default
    version: 1.8.5
    providers:
      kubernetes:
        enabled: true
        accounts:
        - name: local
          requiredGroupMembership: []
          providerVersion: V2
          serviceAccount: true
          configureImagePullSecrets: true
          namespaces: []
          omitNamespaces: []
          oauthScopes: []
          oAuthScopes: []
        primaryAccount: local
    deploymentEnvironment:
      size: SMALL
      type: Distributed
      accountName: local
      updateVersions: true
    persistentStorage:
      s3:
        bucket: spin-bucket
    features:
      auth: false
      fiat: false
      chaos: false
      entityTags: false
      jobs: true
      pipelineTemplates: true
      artifacts: true
    timezone: America/Los_Angeles

You can also add files directly to the container by adding their path and content to values.yaml:

files:
 - path: /home/spinnaker/.hal/default/service-settings/deck.yml
   content: |
     env:
       API_HOST: "http://spin-gate:8084"
 - path: /home/spinnaker/.hal/default/service-settings/gate.yml
   content: |
     overrideBaseUrl: "/gate"
 - path: /home/spinnaker/.hal/default/profiles/front50-local.yml
   content: |
     spinnaker.s3.versioning: false

Helm will create configmaps from these files and mount them into the halyard container.

Working with kubeconfig

Of course if you’re deploying to Kubernetes you’ll need to have the appropriate kubeconfig file. The Helm chart assumes you’re deploying to the Kubernetes cluster Helm is running in, but you might want to deploy Spinnaker in another Kubernetes cluster. In that case, you won’t need Helm to prepare a kubeconfig for the current cluster and you could disable that:

prepareKubeconfig:
  enabled: false

Note that you’ll also want to provide the new kubeconfig as a file, as we did in the previous section and point the halconfig towards it.

No matter what Kubernetes you are planning to use, I strongly suggest you set a separate namespace for the Spinnaker deployment if you are using the Kubernetes v1 provider, so you can delete Spinnaker without deleting everything else along with it.

prepareKubeconfig:
  namespace: my-spinnaker-ns

If you don’t set this value, Halyard will deploy Spinnaker in the same namespace in which Halyard itself is running.

A few more handy tasks

Another thing provided by the chart that might be handy is Spinnaker service accounts creation.

Just list the accounts and they will be created after Spinnaker installation:

serviceAccounts:
  - name: example
    memberOf: ["example"]

Finally, there are also two variables in values.yaml you should know about:

daemon: false
command: "hal deploy apply"

Leaving them with defaults will lead to a k8s job creation that will configure halyard, deploy the Spinnaker instance and shut itself down. This is useful when you have already prepared a halconfig file and want to use a helm release as storage for the halyard configuration. The halyard job is registered as a “post-install,post-upgrade” hook in Helm and will be rescheduled on each upgrade of the release. That means that Spinnaker configuration will be reapplied on each upgrade as well.

Of course, you don’t always have a prepared halconfig. You might want to run a halyard daemon, enter several “hal” commands, and then manually start the Spinnaker deployment. In this situation you’ll find the following configuration useful:

daemon: true
command: "sleep infinity"

In this case, Helm creates the Halyard daemon as a Kubernetes Deployment. You’ll be able to get into the halyard container with kubectl exec, experiment with configuration, deploy with hal deploy apply, fix the errors, deploy again… And you can later get the halconfig you’ve prepared inside the halyard container and use it in your future deployment.

Cleanup

When you’re finished and you want to clean up your Spinnaker deployment, nothing could be easier.  Simply run

helm delete <halyard_release_name>

Which will remove all objects associated with the release, as well as the deployed Spinnaker instance, by running hal deploy clean in the pre-delete hook.  (You can get the <halyard_release_name> by running helm list.)

Again, remember that if you are using Kubernetes v1 provider to deploy a Spinnaker instance, hal deploy clean will delete the entire Spinnaker namespace, even if you have other things in it. If you didn’t create a namespace just for Spinnaker, you can prevent this by running

helm delete <halyard_release_name> --no-hooks

The –no-hooks flag prevents triggering of the pre-delete hook that is responsible for Spinnaker deployment cleanup.  (But remember, you won’t have Spinnaker deleted. Remember to set a namespace or use Kubernetes v2 provider for deployment.)

Moving on from here

So that’s a quicker, dirtier, and most of all easier way to deploy Spinnaker to Kubernetes.

Once you’ve got it deployed, there’s obviously a ton of things you can do with it, though some of them aren’t always obvious. To learn more, please go ahead and sign up for our new Spinnaker Fundamentals course.

Latest Tweets

WEBINAR
Mirantis and Ericsson on Edge Computing