Mirantis OpenStack

  • Download

    Mirantis OpenStack is the zero lock-in distro that makes deploying your cloud easier, and more flexible, and more reliable.

  • On-Demand

    Mirantis OpenStack Express is on demand Private-Cloud-as-a-Service. Fire up your own cloud and deploy your workloads immediately.

Solutions Engineering

Services offerings for all phases of the OpenStack lifecycle, from green-field to migration to scale-out optimization, including Migration, Self-service IT as a Service (ITaaS), CI/CD. Learn More

Deployment and Operations

The deep bench of OpenStack infrrastructure experts has the proven experience across scores of deployments and uses cases, to ensure you get OpenStack running fast and delivering continuous ROI.

Driver Testing and Certification

Mirantis provides coding, testing and maintenance for OpenStack drivers to help infrastructure companies integrate with OpenStack and deliver innovation to cloud customers and operators. Learn More

Certification Exam

Know OpenStack? Prove it. An IT professional who has earned the Mirantis® Certificate of Expertise in OpenStack has demonstrated the skills, knowledge, and abilities needed to create, configure, and manage OpenStack environments.

OpenStack Bootcamp

New to OpenStack and need the skills to run an OpenStack cluster yourself? Our bestselling 3 day course gives you the hands-on knowledge you need.

OpenStack: Now

Your one stop for the latest news and technical updates from across OpenStack ecosystem and marketplace, for all the information you need stay on top of rapid the pace innovation.

Read the Latest

The #1 Pure Play OpenStack Company

Some vendors choose to “improve” OpenStack by salting it with their own exclusive technology. At Mirantis, we’re totally committed to keeping production open source clouds free of proprietary hooks or opaque packaging. When you choose to work with us, you stay in full control of your infrastructure roadmap.

Learn about Our Philosophy

Rally: OpenStack Tempest Testing Made Simple(r)

on May 29, 2014

Cloud users today often also need to be cloud innovators: upgrading components, integrating new hardware, scaling up, and making lots of configuration changes. But how can you be sure the changes you make don’t break anything?

Testing using Tempest is (part of) the answer. Tempest, OpenStack’s official test suite, is a powerful tool for running a set of functional tests against an OpenStack cluster. Tempest automatically runs against every patch in every project of OpenStack, which lets us avoid merging changes that break things.

Historically, however, there have been a few roadblocks to using Tempest. Accessing its power reminds me of being back at University, learning Assembler by coding inline with a high-level language IDE like Turbo Pascal. It’s sure great to have the IDE there to run the system assembler for you, link the code, and report errors. But, all in all, you’d prefer to be writing Pascal or another high level language.

Tempest complexity can make it difficult and time-consuming to use. Tempest also lacks certain features useful for benchmarking, like the ability to generate simulated loads and present results in graph form.

The solution for these problems is Rally, an OpenStack project to create a framework for validating, performance testing and benchmarking OpenStack at scale with Tempest. Rally automatically installs and configures Tempest, and automates running Tempest tests. In contrast to Tempest, which is installed and executed individually on each cluster, Rally can verify a huge number of clouds — just add clouds as deployments to Rally, then easily switch between them. Rally’s benchmarking engine can automatically perform tests under simulated user loads. Results of these tests and benchmarks are saved in Rally’s database. You can review them immediately or schedule many tests over time to see how configuration changes are affecting your cloud.

In this article, we’re going to look at how to use Tempest and Rally, but before we get to the fun part, here are the main reasons why we’ve integrated Rally with Tempest:

  1. Most OpenStack projects use CI/CD systems that employ Tempest as a ‘gate’: for automated testing of committed changes. So it makes sense for us to help developers use Tempest more widely and with greater ease.

  2. Benchmarking with Tempest tests is efficient:

    1. Tempest already implements a multiplicity of tests, so there’s no need to recreate the same tests in Rally. Less code duplication => better world.

    2. Verification should not be limited to testing with a single user. Rally can simulate loading for Tempest. In other words, Rally Benchmark can launch Tempest tests with a variable number of (simulated) active users.

  3. Debugging tempest & OpenStack. If there are race conditions in gates developers will be able to run the same test (or set of test) under load on their environment => which will simplify detecting source of race condition.

Sounds reasonable. But you’ll never know how good Rally is until you try it. So let’s go hands-on!

Install Rally

First of all, you should download Rally from the github repository and run the script “install_rally.sh” to install it.

Add your Cloud to Rally

Rally lets you add clouds in two ways. You can source the openrc credentials of your cloud and create the rally deployment from your environment, like this:

Or you can describe your cloud in json (see link for instructions and details) and offer that to ‘rally deployment create’ using the –file argument:

After adding your deployment, you can use `rally deployment list` to display a list of all known deployments.

Check your Cloud

Use the command `rally deployment check` to see if Rally has been deployed correctly, and to determine which Rally services are available:

Prepare Tempest for your Cloud

As mentioned earlier, Rally can customize Tempest for a huge number of clouds. To decrease the time required for cloning the Tempest repository, Rally clones it once, and then uses a local repository to reproduce it on each cloud.

The command `rally-manage tempest install` takes care of cloning the repository, generates the configuration file for Tempest and installs the virtual environment with all dependencies.

Run Tempest

The command `rally verify start` launches auto-verification. This command expects only one argument: the test set name. If this argument is not specified, smoke tests (the default) will be executed. Output of this command is similar to the Tempest output.

Valid test set names include: full, smoke, baremetal, compute, data_processing, identity, image, network, object_storage, orchestration, telemetry, and volume.

List all Tempest Results

Results of verification tests are stored in Rally’s database, so you can view and compare them. Lists of verifications can be displayed with the command `rally verify list`.

Show Detailed Information for One Execution

Detailed information for one execution can be derived with two commands: `rally verify show` and `rally verify detailed`. The latter displays tracebacks for failed tests.

Benchmarking with Tempest Tests

In addition to installing, configuring and running Tempest, Rally will also store and display results for multiple tests, under simulated user loads.

Let’s run benchmarking for the test scenario  tempest.api.identity.admin.v3.test_domains.DomainsTestXML.test_create_update_delete_domain, with simulated loads ranging from 2 to 10 users. Once the test is finished, we’ll generate an HTML file of the result, including graphs. Information about the JSON/YAML file used to define this task can be found locally in rally/doc/samples/tasks, or on the OpenStack wiki.

For Two Active Users:

For Six Active Users:

For Ten Active Users:

Conclusion

When you deploy and run a cloud, you must cope with constant change, so how do you verify each change? Tempest is a good solution but is cumbersome to use. Rally makes this simple.

Rally provides benchmarking for OpenStack, and now, thanks to Tempest integration, the scenario library is huge. Will your patch make some operations faster? Is your deployment as fast as your competitor’s? Why not find out with Rally?

Some HTML is OK


or, reply to this post via trackback.