Mirantis writing a blog about CloudStack? WTFBBQ? No, it’s not April Fool’s day. As much as we love OpenStack here at Mirantis, I wanted to see for myself what the CloudStack platform offers, things it does better and worse, what works and what doesn’t, how it stacks (pun intended) up against OpenStack, and give CloudStack credit where it’s due.
The idea came to me from a Mirantis Bootcamp OpenStack training session we delivered where a number of attendees asked about CloudStack and how it compares to OpenStack. I spent some time researching and didn’t find many technical reviews, let alone comparisons. I decided the best way to learn more and better prepare for this question was to deploy CloudStack in a test environment and try it myself. The following are my impressions of CloudStack along with comparison notes to OpenStack when applicable.
CloudStack Installation and Configuration
An exercise in prerequisites
If someone told me a week ago that I couldn’t install CloudStack on top of the latest Ubuntu 12.04 LTS release, I would have been shocked and assumed they got their facts mixed up. However, after attempting a basic two-node CloudStack installation, I’m convinced that it’s easier to use the two-year-old version of Ubuntu (listed in their requirements) and install the missing hardware drivers manually than it is to use Ubuntu 12.04 LTS and deal with script hacking and modification to get CloudStack running.
Attempt #1: First, I proceeded to ignore the install guide and used Ubuntu 12.04 LTS instead of Ubuntu 10.04 LTS because I did not want to install my hardware drivers manually. I expected to have minor problems with Ubuntu 12.04 with easy workarounds. Instead, I was met with error after error when running the install scripts. Each time I found a workaround through online forums for one script, I’d run into a new error on the next script. I quickly concluded it was a dead end and gave up.
Attempt #2: Eventually, I gave in and downgraded to Ubuntu 10.04 (almost three years old) and manually installed the drivers for my hardware. I was able to get further in the installation process but ran in to some hiccups when attempting add a host and provision my instances, more details to follow.
I did not try Red Hat or Centos with CloudStack but found similar complaints here: CloudStack OS Versions.
CloudStack setup and configuration wizard
Now, a shining moment for CloudStack. The wizard for setting up the infrastructure is really slick and easy to use. It reminds of the wizards they use for routers. You can click through the screens I captured below to get an idea. They also included mouse over hints on the parameters in case you have trouble filling out the data. These guys really nailed this part of the experience.
“Unable to add CloudStack host”
As you may have noted from the slide deck above, it ended with “Add Host”. This is because I was unable to add a host and continue the wizard. There were two reason why the system kept failing with “unable to add host”.
- SSH is not installed on Ubuntu 10.04 LTS but it’s required in order for a CloudStack host to connect to Cloud Management Server. While attempting to add a host, the wizard complained with “unable to add host” which couldn’t be less helpful. I found the root cause after looking at /var/log/cloud/management/management-log and saw an error about not being able to connect to the agent on port 22.
- After getting past the first error by installing SSH, I was still “unable to add host” due to missing network configurations. The cause: Ubuntu has a GUI for configuring networking which is the default way a user would set up the networking but isn’t usable CloudStack. CloudStack requires networking to be configured using /etc/network/interfaces, otherwise it will think it has no network. See here for help Ubuntu Network Configuration
Neither of these requirements are a big deal to resolve; the problem was that they weren’t documented.
Zones, Pods, Clusters and Hosts
CloudStack uses Organizational Units (OU) to group hosts within a CloudStack deployment. Zones, the largest OU, usually corresponds to a datacenter and consists of at least one pod and a ‘Secondary’ storage shared by all pods in a zone. Pods represent a single rack and consist of at least one cluster. Clusters in the same pod use the same subnet. All hosts in a cluster are homogeneous in that they use the same hardware, hypervisor, subnet, and access the same shared storage (‘Primary’ Storage). Hosts are single nodes running hypervisor software that provide computing resources to run guest VMs.
CloudStack offers two types of storage for use by the platform, which they call ‘Primary’ and ‘Secondary’. Primary storage is configured at the cluster level and stores all disk volumes for VMs in a cluster. These volumes are persistent, and will remain after you terminate the VM. Local disk, iSCSI, and NFS can be used for Primary storage. Secondary storage is configured at the Zone level, and is used to store templates, ISOs, and snapshots. All zones need at least one Secondary storage. The documentation says that either NFS and OpenStack Swift can be used as Secondary storage.
CloudStack Wizards and Monitoring (“Eye Candy”)
There’s no denying that the CloudStack GUI is beautiful and easy to use. I was seduced by the pictures. But when I dug deeper, I found some basic functionality that didn’t work though it appeared to on the surface. For example, after attaching a volume to an instance to use as persistent storage, the UI showed no errors. However, when I logged in to the VM instance, the volume was not there (I checked dmesg and /proc/partitions ). It was only after scouring the internet that I found a suggestion to reboot the VM. No problem, since I remembered seeing the VM reboot functionality in the GUI, but I was again let down. I tried rebooting the instance several times and each time the UI promised that it had succeeded, but the VM never rebooted and showed an uptime of several hours. Fortunately, I was able to ssh in to the VM to reboot manually. This is potentially a big issue for people who don’t have ssh access to the instance and can only reboot via the GUI.
That said, the GUI for provisioning instances is really user-friendly and will impress new users. See video below:
CloudStack also does a good job of using the GUI to monitor resources. See the pictures below; managers will eat this stuff up.
CloudStack System Virtual Machines
CloudStack uses two system virtual machines for each zone. Console Proxy VM provides console access to VM instances and SSVM(Secondary Storage VM) manages and downloads images. During my testing, I came across a third system VM, a virtual router that provides networking to the tenants. While this System VM approach may be useful for isolating major services, I think it could be inefficient for resource utilization. These system VMs need their own IPs, CPU, RAM, and disk space that can’t be shared. For example, if SSVM is not doing any work, the reserved CPU and RAM can’t be used for something else. Also, I wonder how well these system VMs will scale and support HA.
CloudStack Templates and ISOs
In CloudStack, a template or ISO is required to provision an instance. CloudStack creates a Secondary Storage VM (SSVM) to manage and download ISOs and templates. If the SSVM can’t connect to the internet after installation, you won’t be able to add a template or ISO. In addition, the default template needs to be downloaded before a VM can be provisioned. To fix the internet access for SSVM, I had to ssh in to the SSVM and add a gateway to /etc/network/interfaces. There’s no information in the documentation on SSVM, how to ssh to it, what it’s used for, or how to troubleshoot it. I found the links below helpful in getting SSVM working.
The Snapshot functionality did not work for me in CloudStack. After some research, I found that the problem has to do with the older version of Ubuntu.
Some concluding observations
I was a bit let down by how much trouble I had getting CloudStack to a state where it could provision virtual machines. The prime cause of this frustration came from shortcomings in the documentation and lack of support for newer operating systems. Once you are able get over the humps during deployment and configuration, there are some areas like user interface, setup wizards and resource metrics where it shines. Another feature which I really liked (but didn’t try because I only had two nodes) is Live Migration – that’s something OpenStack can’t do yet via its dashboard.
Based on what I learned from this experience, I sense that the biggest disadvantage with the CloudStack platform is where it lacks flexibility. I also think that the breadth of support in the open source community is shallower, considering the outdated OS and how much I had to dig around for answers to questions – much more digging than I experienced in deploying OpenStack.
Disagree? Had similar experience? Please post your comments below!