Icehouse Community Investment Report

The momentum behind OpenStack is continuing to build, and we are seeing more and more traditional players embrace OpenStack as the cloud platform of choice. During this most recent semi-annual release cycle, called Icehouse, we saw Oracle put its weight behind OpenStack and Ericsson announced that OpenStack (packaged and supported by Mirantis) will be the cloud platform driving its main offering for telcos.

Learn more about What’s New in OpenStack Icehouse during Mirantis’ technical overview webcast this Thursday, April 24th.

As a significant player in the OpenStack space, Mirantis is aggressively investing effort and resources in OpenStack improvements and innovations directly in the community. Our focus area, as driven by our experience with customers, has been on making OpenStack easier to adopt for production use; this release cycle is no exception. In this post, I would like to offer a progress report on our community investment, for the benefit of those those readers interested in OpenStack, yet not familiar with developments inside the ecosystem.

Let’s start with a look at the numbers. According to stackalytics.com, which shows contributions by Mirantis and others to OpenStack projects, Mirantis ranked as:

  • Top-5 contributor to OpenStack by number of commits

  • a Top-3 contributor in terms of number of drafted and completed Blueprints

  • the number one contributor on Stackforge, the largest OpenStack innovation sandbox.

Within the range of activities across a variety of projects, there are several that bear highlighting; I’ll delve into that below.

Project Fuel and OpenStack Deployment

It’s widely recognized that one of the larger challenges facing any OpenStack implementation is in choosing how to deploy OpenStack in the face of a myriad of potential configurations across a multitude of hardware choices. Project Fuel, pioneered and open sourced by Mirantis, has been downloaded thousands of times. We are constantly learning of new users who are using Fuel as part of the community version of Mirantis OpenStack, alongside our commercial customers who use it.

In the Icehouse cycle, our engineers participated in the TripleO mini-summit that took place in March. At this forum, we brought the knowledge we’ve gained in implementing complex HA architectures, along many other areas where our Fuel team has acquired expertise, and shared it with engineers from HP, RedHat and others involved in the project.  Mirantis is committed to the TripleO program; we’re currently the number three contributor.  We have also committed code to Ironic, the bare metal provisioning project, and have made a significant investment there as well.

OpenStack Data Processing Program

One of the popular and growing use cases for OpenStack adoption is its ability to run clusters for complex data processing tasks. One such use case includes the ability to execute Hadoop tasks atop OpenStack. In the spring of 2013, during the Havana release cycle, a Mirantis-led initiative called Savanna was approved for incubation. We are happy to report that after more than a year of collaboration with our colleagues from Hortonworks, Red Hat and others through the Icehouse cycle, the Data Processing initiative, now renamed Sahara, has successfully graduated as an integrated project for the Juno cycle.  Using Sahara allows customers to automatically deploy and manage various Hadoop distributions on top of OpenStack clusters at far less cost and effort than standing them up on bare metal.

Streamlining the process for vendors to make their drivers OpenStack compatible

One of the most quoted news items of the Icehouse release cycle is the move by OpenStack to streamline how vendors integrate their driver code with OpenStack. After all, the ability to integrate and abstract device drivers is core to OpenStack’s value proposition. An open, self-service approach to testing and reporting driver compatibility with OpenStack can help vendors put their products to work in OpenStack, and help OpenStack take advantage of vendor product innovations. No less importantly, it provides customers a real-time dashboard that tells them which vendor products work with each stable release and what level of testing has been passed.

Mirantis has been driving this effort from two perspectives. First, Mirantis architect Jay Pipes has been leading the initiative from the community side, facilitating online engineering discussions, authoring instructions for vendors and hosting weekly IRC meetings. The three blog posts he published detail the inner workings of OpenStack Continuous Integration/Continuous Deployment systems, and show how to set up an external OpenStack Testing System. Second, Mirantis engineers are also developing a single, open repository for keeping track of all vendor plugins and a dashboard to enable OpenStack adopters to easily access that information, and get the latest data on what works with OpenStack. Here’s how this dashboard looks:

 

At the OpenStack Foundation Board, Mirantis co-founder Boris Renski (with great support and collaboration from other Board members) has been working with the foundation staff and the DefCore committee to discuss the branding direction that the compliant vendors will be able to leverage for participating in the program. While still a work in progress, we expect to be able to announce it very soon, so please stay tuned.

Application Orchestration

As more and more OpenStack use cases are moving into production workloads, we see more and more customers looking at how to efficiently manage workloads on top of OpenStack. Mirantis has been investing in this area for quite a while now through its Murano and Mistral projects, open-sourced on Stackforge. Murano is a service that runs on top of Heat, allowing applications to register with OpenStack through an application catalog service. Mistral standardizes the workflow-as-a-service functionality.

In the Icehouse timeframe, we have seen the conversations around application catalog and workflow functionality move into the mainstream. We have been working very closely with community colleagues from Heat (Mirantis is a Top-5 contributor), Glance (where Mirantis is in the Top-3) and Solum. I expect that there will be a lot of news coming from this area in Juno release.

Scalability and OpenStack Performance at Scale

Every customer that we engage with asks us at least one of the following two questions: “How well will my OpenStack cloud scale when I add nodes and/or VMs?”; and “What will happen to my workload performance if I introduce a patch, an upgrade or a new feature to my cloud?”

To provide a systematic approach to answering this question, Mirantis’ Boris Pavlovic started Rally, an OpenSource project on Stackforge designed to provide functional and performance testing of various workloads at scale. It was introduced at the Hong Kong OpenStack Summit. Rally is designed to run a mix of Deploy/Verify/Benchmark/Analyze sequences,  and to provide a standardized set of benchmarks for major OpenStack functionality that can be used by developers and operators alike.

For an initiative that was started just a short while ago, Rally has enjoyed tremendous attention and participation. Rackspace, Numergy, Redhat, Intel, IBM, Yahoo (16 companies in total) have been contributing code into Rally. At this point, Rally is integrating closely with Tempest (where Mirantis is also a Top-5 contributor) and more and more Rally is being used in the standard gate checks for major components of OpenStack itself — i.e., in support of development, testing, and collaboration tools & systems used in the day to day operation of the OpenStack project as a whole. More and more companies we see are using Rally as a part of their internal effort to guarantee their internal SLA commitments. Mirantis also used Rally as the main engine for its well-publicized effort to benchmark the running of 75,000 virtual machines on top of IBM Softlayer bare metal servers.

Scalable Data Stores

The continuous expansion of the data universe is everywhere; OpenStack is certainly part of the ever-increasing appetite for scalable data stores. A new Open Source Database as a Service project called Trove premiered this cycle; its mission, to provide scalable and reliable Cloud Database as a Service provisioning functionality for both relational and non-relational database engines, and to continue to improve its fully-featured and extensible open source framework. Started in the Diablo Release by Rackspace and HP more than 2 years ago, Trove has officially graduated to an Integrated Release in Juno. Mirantis is a top-5 contributor to Trove, where we’ve focused on enabling the usage of scalable no-sql databases such as MongoDB and Cassandra.

Along the same lines, Mirantis has started a new project called MagnetoDB, which is an analog for AWS’ DynamoDB, a fast, fully managed NoSQL database key-value service that makes it simple and cost-effective to store and retrieve any amount of data, and serve any level of request traffic. With MagnetoDB, you can offload the administrative burden of operating and scaling a highly available distributed database cluster.

Unlike Trove, MagnetoDB is an actual implementation of higher level database functionality (think of the Swift vs. Cinder implementation approach), but we fully expect that Trove will be able to deploy and manage MagnetoDB in the near future. The efforts behind MagentoDB are led by Mirantis in partnership with Symantec.

Looking Ahead

In Juno, Mirantis will continue to drive significant upstream innovation. Our major focus remains unchanged: helping OpenStack to become more robust, more scalable and easier to adopt. The following areas will see Mirantis engineers investing their time and talent:

  • Making core OpenStack more stable — Bug fixes in Nova, Neutron, work in Oslo, etc. More test coverage in Tempest. Helping to drive towards parity in Neutron Networking. Adding Neutron support in Heat.

  • Getting OpenStack to scale better — Improving Nova Scheduler. Integrating Rally benchmarking for all core projects, as well as adding it to OpenStack Infra. Adding more services and features to Neutron (such as LBaaS, etc.)

  • Improving OpenStack deployment — Automation of Ironic migrating from Nova-Baremetal. Contributions to Ironic. Adding HA capabilities for TripleO elements.

  • More complete, transparent driver compatibility — Working with more vendors to embrace the DriverLog framework.

  • Application on-boarding — More contributions in Murano, Mistral, Sahara and additional related OpenStack projects.

Latest Tweets

Suggested Content

WEBINAR
Edge Computing: A Unified Infrastructure for All the Different Pieces