Mirantis OpenStack

  • Download

    Mirantis OpenStack is the zero lock-in distro that makes deploying your cloud easier, and more flexible, and more reliable.

  • On-Demand

    Mirantis OpenStack Express is on demand Private-Cloud-as-a-Service. Fire up your own cloud and deploy your workloads immediately.

Solutions Engineering

Services offerings for all phases of the OpenStack lifecycle, from green-field to migration to scale-out optimization, including Migration, Self-service IT as a Service (ITaaS), CI/CD. Learn More

Deployment and Operations

The deep bench of OpenStack infrrastructure experts has the proven experience across scores of deployments and uses cases, to ensure you get OpenStack running fast and delivering continuous ROI.

Driver Testing and Certification

Mirantis provides coding, testing and maintenance for OpenStack drivers to help infrastructure companies integrate with OpenStack and deliver innovation to cloud customers and operators. Learn More

Certification Exam

Know OpenStack? Prove it. An IT professional who has earned the Mirantis® Certificate of Expertise in OpenStack has demonstrated the skills, knowledge, and abilities needed to create, configure, and manage OpenStack environments.

OpenStack Bootcamp

New to OpenStack and need the skills to run an OpenStack cluster yourself? Our bestselling 3 day course gives you the hands-on knowledge you need.

OpenStack: Now

Your one stop for the latest news and technical updates from across OpenStack ecosystem and marketplace, for all the information you need stay on top of rapid the pace innovation.

Read the Latest

The #1 Pure Play OpenStack Company

Some vendors choose to “improve” OpenStack by salting it with their own exclusive technology. At Mirantis, we’re totally committed to keeping production open source clouds free of proprietary hooks or opaque packaging. When you choose to work with us, you stay in full control of your infrastructure roadmap.

Learn about Our Philosophy

OpenStack Benchmarking on SoftLayer with Rally

on December 10, 2013

As we recently posted, Mirantis and IBM SoftLayer collaborated to benchmark OpenStack. A little while ago, right before the Icehouse Summit, Mirantis released a project called Rally, a tool for OpenStack benchmarking. Mirantis has been collaborating with SoftLayer for the last three to four months to benchmark OpenStack on a larger scale using Rally. As a result of this collaboration, we’ve prepared a solution that allows us to specify real-world workloads and can be run against existing OpenStack clouds.

This article takes a look at the setup for this benchmarking, as well as specifics of the actual use case.

IBM SoftLayer is a global hosting leader and has a large number of servers that can be used for different kinds of workloads. As you know, Mirantis has been around for a while, concentrating primarily on OpenStack deployment and operations for the last three years.

Who is Rally for?

Rally is a solution designed to benchmark and profile OpenStack-based clouds and it’s made for both developers and operators. When it comes to developers, the tools allow them to specify some kind of synthetic workload to stress-test OpenStack clouds and get the low-level profiling results. What Rally does in that case is collect all of the nitty-gritty details about running specific things, like provisioning a thousand VMs, for example, and see how on average a cloud performs in that environment.

On the other hand, if you think of operators, they typically don’t run the workloads, like provision and immediately destroy VMs. So, their workloads are more complicated, and we’ve created an engine that allows us to specify real-life workloads and runs on existing OpenStack clouds. The results that we generate from this kind of benchmark are more high level but allow you to identify bottlenecks on your cloud. For example, if something is wrong with the database, and it also saves some sort of historical data, you can see if you’re getting better or worse with the changes that you’ve applied to your clouds.

The setup

The setup for the benchmark is very simple. We’ve been using DevStack to deploy the latest from OpenStack trunk as well as Mirantis Fuel for larger deployment in an HA cloud. IBM SoftLayer has provided us with 1500 baremetal servers. We created several pools of these servers and deployed several clouds to see how OpenStack behaves on different scales. The biggest cloud was almost one thousand nodes.

Before I dive into the demo we did (which you can check out here), it’s important to note something about SoftLayer. Typically, when you deploy OpenStack clouds, you can perform baremetal provisioning through a tool like Cobbler or something else. What we needed to do is integrate the SoftLayer API into our solution called Fuel so we could provision hundreds and thousands of servers on demand through SoftLayer.

A real-world use case

In the real world, companies have several applications that they want to deploy within several different usage patterns. Suppose you have a complex web application with a web layer, cache, load balancer, and a database. If you have multiple concurrent teams developing this application, they’d probably be deploying different versions of this application on the cloud several times a day. And, if this is primarily for Dev/QA, this environment would not live for a long time: you deploy it, use it for several minutes or hours, you get the results, and you get this environment destroyed.

Therefore, if you have a large environment, then you have several teams around a bunch of standard stack applications, and each application contains a lot of VMs. To translate these requirements and workload into OpenStack, you would need a number of tenants, users per tenant, and a number of VMs you’d be provisioning concurrently. In terms of steps, this use case goes through provisioning, using, and destroying VMs. If you want to benchmark OpenStack, it only participates in the first and the last step. For those two steps, you need to identify how long on average it takes to provision as well as the success rate.

Technically, this becomes the baseline for your SLA. However, OpenStack is not a monolithic structure. It’s a distributed system that contains a bunch of daemons talking to each other. In our case, provisioning a VM requires for the request to go through the nova API, then probably go through Keystone for authentication, then to the nova database, and so on.

You want to know where you spend most of the time during this provisioning and you want to build a solution that would allow you to collect that data. When you run this for the first time, it gives you a baseline of how your cloud performs. As soon as you get the baseline, you want to be able to provide some historical data. For example, if you run the same benchmark after you’ve changed something on the cloud–such as database configuration or Glance caching option–you want to be able to see if your changes are helping you get the overall time below your SLA. That brings us to our demo.

The application

Our application is the front end for Rally deployed on a local server with an access to SoftLayer servers. We’ve deployed several environments. Each of the environments contains information about the benchmarks that have been run there. As the demo shows, these benchmarks have been specified to two workloads called “Creative Testing” and “Report Generation.”

The workflow

If you want to do some testing on different things in parallel, you need to provision your entire web application and you need to specify the workflow saying you’d like to provision it, use it for some time, and destroy it. For report generation, you probably need to run something several times a day, but it would require you to deploy and run a large cluster on the results, run the workload, and destroy everything.

Every company has their own workloads, and the uniqueness can come from either the concurrency pattern, or the application itself, or the steps that you take. We made a little more sophisticated workload called Dev/QA. It begins by provisioning the application, and then assumes you use this for a while, let’s say 20 minutes. Then, if you’ve made some changes in the database, it takes a snapshot for further verification. And, then, it assumes you want to use it for a while, for 5 more minutes, just to collect some data, and after that, it destroys the VMs.

The wizard

Therefore, you can specify most of the steps that are available on OpenStack through the Rally wizard. As soon as you specify the workflow, you can switch to the usage pattern and say that the cloud will be used by, say, 5 tenants, that each tenant will have at least 3 users, and that these users will use this 2 times per hour. After that, you would go to the usage topology.

Initially, we wanted to rely on Heat application templates but decided to allow users to specify a very simple application topology by defining roles of servers. As soon as you specify the workflow, you can create the workload in the Rally wizard. After that, you can go to the clusters, select a cluster and run a benchmark on your workflow (for our example, it could be a baseline for Dev/QA). Technically, a workflow takes about 40 minutes, but in the real world, you’d want to repeat this step multiple times. So, then you would specify how much time you want to spend on each step and run the benchmark.

The benchmark

If you go in the details for each benchmark in Rally, you can see how much time you spend on each of the OpenStack components so you can make adjustments and rerun the benchmark for comparison. A future version of Rally will include SLA-related information about how your cloud should perform on average and also calculate the maximum time.

Want to join us?

Are you an OpenStacker intrigued by Rally? Then, we want you to join our team! Check out the following resources and post your comments. We’d love to hear your feedback.

Resources:

  1. Georges

    Hi! I want to thank you for this post. It is very hepful.In fact ,I am a software engineer and a new OpenStacker à B-com France. But Now I have to benchmark our openstack cloud with a complex workload. I would have a deep understanding of the Rally framework in order to achieve this task and to contribute to the development of rally. Please could you help me.
    Regards!

    April 30, 2014 01:04
    • Boris Pavlovic

      Georges,

      It will be nice to get new users/contributors=)

      Could you contact me:

      IRC freenode #openstack-rally (boris-42)

      or via email bpavlovic@mirantis.com

      And I’ll help you to start use and contribute to it.

      April 30, 2014 14:26

Some HTML is OK


or, reply to this post via trackback.