Cloud certification with Rally

Testing OpenStack cloud should go beyond just making sure “everything works,” which is really just the tip of the iceberg for meeting customer expectations. The cloud must also scale in the expected timeframe and meet other high levels of performance.

At 3:40 on Tuesday at the OpenStack Summit in Vancouver, Jesse Keating (BlueBox) and I will be presenting Using Rally for OpenStack certification at scale! We’ve analyzed testing levels for cloud certification, and we’ll demonstrate in depth why Rally is the right tool for certifying your cloud at scale.

Rally is a benchmarking tool that tells you how OpenStack performs, especially under load at scale. You can use it to validate, performance test, and benchmark your OpenStack deployment using various pluggable Rally benchmark scenarios.

Now let’s establish what cloud certification means.

Testing levels for cloud certification

We start here with the five testing levels for cloud certification as seen in Figure 1.

Fig. 1

1. As noted, you first need to check that “everything works” at a basic level.

2. “Expected performance” ascertains not only performance, but that everything in the environment works in a timely way.

3. Testing “under expected load” confirms the environment works on time with a specific load, e.g, 100 VMs in parallel, while meeting expected performance.

4. “Under expected scale” means testing how many VMS you can run in total and how the cloud works as you add more. For example, if you’re running 10,000 VMs, what happens to performance when you add 100 more?

5. Testing “high availability” means ensuring the cloud functions if something goes wrong, like a controller going down. Does the cloud continue its work at expected scale with the expected load and performance?

Rally tests these steps, easily creating graphical test results, including pie charts, performance charts, and histograms, so you can create reports that are easy to understand, with information that viewers can quickly digest.  See more details.