How to deploy Spinnaker on Kubernetes: a quicker and dirtier guide
The "point and shoot" method: Spinnaker quick installThe easiest, quickest, dirtiest way to deploy Spinnaker to Kubernetes is to use the "quick install" Kubernetes manifest maintained by the Spinnaker community. For example:
kubectl apply -f https://spinnaker.io/downloads/kubernetes/quick-install.ymlThe advantages of using this method include the fact that it gives you a serviceable install without having to do ... well, much of anything except having a working Kubernetes cluster. The disadvantage is that you have to work a bit harder if you want to do any customization, such as changing the storage or adding repositories or additional cloud providers.
To customize this Spinnaker install, you have two choices:
- Download and edit your own copy of the manifest, or
- Find and log into the deployed halyard pod and perform your edits from there, then redeploy Spinnaker.
The "professional camera" method: deployment by Spinnaker Helm chartIf you want to have any significant control over your Spinnaker deployment, you're better off using a method that gives you easy access to the parameters that control it, such as a Kubernetes Helm chart.
In this article, I'm going to show you how to deploy Spinnaker using Helm, including how to use an external values.yaml file to customize the parameters used for deployment. Simply put, there are three steps:
- Install and initialize Helm.
helm repo add mirantisworkloads https://mirantisworkloads.storage.googleapis.com
helm install mirantisworkloads/halyard
What's actually going onHelm is a native Kubernetes software packager; in this case, we're using it to deploy the basic infrastructure you need to effectively use Spinnaker, such as Minio for storage and Jenkins for continuous integration. Helm will also install Halyard, which will then go on to install Spinnaker.
As part of this process, the Helm chart gives Halyard a default configuration, and once it's up, calls
hal deploy applyto actually deploy Spinnaker for you.
Fortunately, you have the option of overriding those default values so that Spinnaker gets deployed the way you want it. You can also use the chart to add supplemental files such as a kubeconfig to the container as it gets spawned.
Halyard chart configuration and overridesIf you want to look at the default values used by Helm (and subsequently by Halyard) you can find the Halyard chart source files on Github, here:
If you decide you want to change any of those values, create a new file called values.yaml and add the appropriate YAML to it.
For example, by default the chart assumes that your Kubernetes cluster has RBAC enabled , and that it supports the LoadBalancer service type. You can disable RBAC and change the type for the Spinnaker Gate and Deck services by overriding these chart values, like so:
rbac:Minio and Jenkins deployment is also optional. Both are the subcharts of the Halyard chart, and you can disable them by setting:
minio:Of course, if you disable Minio, you’ll have to configure some storage provider instead. You can do that by configuring Halyard itself.
Adding the halconfigNow we've arrived at the halyard configuration itself. You can provide a halconfig and any number of arbitrary files by specifying their contents in the values.yaml file.
You can add the contents of the halconfig file you'd like to change. Here’s an example of halyard configuration with Kubernetes v2 provider enabled:
halconfig: |You can also add files directly to the container by adding their path and content to values.yaml:
- name: default
- name: local
files:Helm will create configmaps from these files and mount them into the halyard container.
- path: /home/spinnaker/.hal/default/service-settings/deck.yml
- path: /home/spinnaker/.hal/default/service-settings/gate.yml
- path: /home/spinnaker/.hal/default/profiles/front50-local.yml
Working with kubeconfigOf course if you're deploying to Kubernetes you'll need to have the appropriate kubeconfig file. The Helm chart assumes you're deploying to the Kubernetes cluster Helm is running in, but you might want to deploy Spinnaker in another Kubernetes cluster. In that case, you won't need Helm to prepare a kubeconfig for the current cluster and you could disable that:
prepareKubeconfig:Note that you'll also want to provide the new kubeconfig as a file, as we did in the previous section and point the halconfig towards it.
No matter what Kubernetes you are planning to use, I strongly suggest you set a separate namespace for the Spinnaker deployment if you are using the Kubernetes v1 provider, so you can delete Spinnaker without deleting everything else along with it.
prepareKubeconfig: namespace: my-spinnaker-nsIf you don't set this value, Halyard will deploy Spinnaker in the same namespace in which Halyard itself is running.
A few more handy tasksAnother thing provided by the chart that might be handy is Spinnaker service accounts creation.
Just list the accounts and they will be created after Spinnaker installation:
serviceAccounts:Finally, there are also two variables in values.yaml you should know about:
- name: example
daemon: false command: "hal deploy apply"Leaving them with defaults will lead to a k8s job creation that will configure halyard, deploy the Spinnaker instance and shut itself down. This is useful when you have already prepared a halconfig file and want to use a helm release as storage for the halyard configuration. The halyard job is registered as a “post-install,post-upgrade” hook in Helm and will be rescheduled on each upgrade of the release. That means that Spinnaker configuration will be reapplied on each upgrade as well.
Of course, you don't always have a prepared halconfig. You might want to run a halyard daemon, enter several “hal” commands, and then manually start the Spinnaker deployment. In this situation you'll find the following configuration useful:
daemon: true command: "sleep infinity"In this case, Helm creates the Halyard daemon as a Kubernetes Deployment. You’ll be able to get into the halyard container with kubectl exec, experiment with configuration, deploy with hal deploy apply, fix the errors, deploy again… And you can later get the halconfig you’ve prepared inside the halyard container and use it in your future deployment.
CleanupWhen you're finished and you want to clean up your Spinnaker deployment, nothing could be easier. Simply run
helm delete <halyard_release_name>Which will remove all objects associated with the release, as well as the deployed Spinnaker instance, by running hal deploy clean in the pre-delete hook. (You can get the <halyard_release_name> by running helm list.)
Again, remember that if you are using Kubernetes v1 provider to deploy a Spinnaker instance, hal deploy clean will delete the entire Spinnaker namespace, even if you have other things in it. If you didn't create a namespace just for Spinnaker, you can prevent this by running
helm delete <halyard_release_name> --no-hooksThe --no-hooks flag prevents triggering of the pre-delete hook that is responsible for Spinnaker deployment cleanup. (But remember, you won't have Spinnaker deleted. Remember to set a namespace or use Kubernetes v2 provider for deployment.)
Moving on from hereSo that's a quicker, dirtier, and most of all easier way to deploy Spinnaker to Kubernetes.
Once you've got it deployed, there's obviously a ton of things you can do with it, though some of them aren't always obvious. To learn more, please go ahead and sign up for our new Spinnaker Fundamentals course.