Mirantis OpenStack 7.0 vs Red Hat Enterprise Linux OpenStack Platform 7
|Review 1||Deployment (“Day 1” of OpenStack lifecycle management)|
|Review 2||Ongoing operations (“Day 2” of OpenStack lifecycle management)|
|Review 3||Reliability at scale|
|Review 4||Developer productivity|
|Review 5||Open source approach & community participation|
|Review 8||Completeness of offering|
We used only public documentation and available software. No scripts, professional services, or corporate support have been used. For the software comparison we are looking at the OpenStack distributions and not the host OS, hypervisor, or guest OS. Finally, Fuel is now an official Big Tent OpenStack project, but it was not the case when these distros were released. For this reason, we are not treating Fuel as an OpenStack project in this blog to ensure that Mirantis doesn’t get an unfair advantage.
Review #1: Lifecycle management (Day 1)
Level set - InfrastructureFor the purposes of this comparison, we are focused on splitting it into Day 1 and Day 2, where Day 1 involves the ease of installation and deployment, and Day 2 involves ongoing operations like scaling the environment up and down, patches, and upgrades. Using the available documentation, we designed equivalent physical environments and used the exact same hardware for each distribution deployment.
Installation ProceduresThis review consists of the overall process for getting both distributions from bare metal and an ISO to an up-and-running OpenStack environment capable of launching instances and workloads.
- Guided vs manual installation steps
- Time required for installation and deployment
- Frequency and types of tripping points/issues
- Troubleshooting required
MirantisWe start the review with Mirantis. As shown in the Installation and Deployment Steps diagram, from start to finish Mirantis OpenStack took 12 steps, of which more than 80% were guided procedures, either by a wizard or a GUI. The only point that caused any significant amount of troubleshooting was caused by user error, where I failed to “Apply” my network changes on each page of the opening installation GUI of the Fuel Master. Rather, I made all the changes and then clicked “Apply,” losing the changes I made on previous pages. But, as a testament to Fuel’s network verification check, I was able to quickly diagnose and fix the issue.
Red HatAs for getting Red Hat Enterprise Linux OpenStack Platform 7 off the ground, that was significantly more challenging. This is both the first version of their TripleO deployment model and the first version of their Director installer; both have several tripping points. From start to finish, the preponderance of steps are pure CLI commands with a heavy dependence on the documentation. It is also the first version release of the documentation for TripleO and Director, and it suffers from several syntax typographical errors, and—in some areas—missing steps altogether. At the time of this article’s writing, there were 67 outstanding errata filed against the installation documentation. Our experience with these errors resulted in many showstoppers, such as failed installations and deployments with cryptic logs. It took hours of research to turn up the correct context and syntax for many CLI commands. The Red Hat OpenStack distribution lacks any active pre-deployment checks, so several issues were discovered only after attempting deployment.
Adding the comparison of errors found in Red Hat’s documentation, at 67 known errata, to Mirantis’ 6 outstanding documentation errata and the number of issues that came up during the install and deployment, Mirantis clearly has a better installation method.
Review #1 Scores:Mirantis OpenStack 7.0 with Fuel
The user experience to install from scratch and get to a working environment was easy considering the vast majority of the steps were wizard driven, requiring only that I enter prompted information; the rest was basically automatic. There were very few issues along the path of installation.
Total Steps: 12 mostly guided steps (includes some defaults, sub-steps, and 2 optional health verifications)
Total Time: 1 hour 20 minutes.
Red Hat OpenStack Platform 7 with Director
The experience here was significantly difficult. The majority of steps were numerous CLI commands to get things off the ground, and with them came several syntactical errors, both in the documentation and in user-error. The higher number of user-error issues are to be expected when dealing with a large set of manual CLI, context, and syntax related commands. However, the many errors in the documentation were unexpected. Coupling the two together resulted in an extended and frustrating experience to get a minimally viable environment. In addition, the extra steps required after deploying OpenStack to get to the point of being able to launch instances was another letdown.
Total Steps: 48 mostly manual CLI (includes some default values, sub-steps, and one optional health verification)
Total Time: 3 hours 5 minutes (not including troubleshooting time)
Review #2: Lifecycle management (Day 2)Continuing operations for OpenStack is a different challenge to meet than the initial configuration and deployment, so we have broken it out separately. In this category, we look at the abilities for each distribution to scale up and down, patch an existing environment, and eventually upgrade it to the next major revision.
- Ease and automation of scaling the physical infrastructure up/down
- Patching methods and documentation
- Method and documentation for updating
- Managing multiple environments
Scaling the physical infrastructureMirantis - Using Fuel, existing environments are easily scaled both up and down, with automated discovery of PXE bootstrapped nodes and GUI or CLI based scaling of nodes.
Red Hat - Red Hat’s scaling, while functional, is lackluster. While it performs as expected, there aren’t any shining features that benefit administrators in making their jobs easier or less involved. One aspect that is appreciated is the use of Ironic for bare metal node registration. While Ironic is still young and without numerous features, and requires some manual data collection and entry (for example, Node MAC addresses), it has the potential to grow into an essential component. But with the added steps of data collection and the added complexity of the IPMI-to-Management networking requirement countering the benefit of Ironic registration, it’s a wash for scoring. Additionally, Red Hat documentation is limited to adding and removing nodes through manual CLI operations rather than through their Director GUI.
PatchingPatching for bugs and security are a commonplace event in operating systems and software. Both distributions have documented processes to apply patches, and both use similar methods for distributing patches, including customer notifications and normal system level commands such as yum and apt-get.
UpgradesUpgrading OpenStack is a long-standing pain point in operations. The community and distribution vendors continue to make improvements to the process, but it still remains a manual, though practiced, process. The procedures exist in both of these distributions to allow for minimal downtime, albeit a relatively long and time-consuming process, or if more downtime is available within a change window, a shorter bulk style process can be undertaken.
Mirantis has well documented manual processes for each procedure type, and focused upgrade scripts in their Octane tooling for Fuel to help automate where possible.
Red Hat does have documentation on the processes required for upgrading their environment. Currently, there are hundreds of individual CLI commands required to achieve an upgrade; they have not yet provided any automation in scripting.
Management of multiple environmentsThe deployment engine built into Mirantis OpenStack, Fuel, can maintain management of multiple OpenStack environments, each at a different version. This flexibility enables organizations to concurrently run test and development environments that have a more dynamic versioning structure, alongside production environments with more rigid versioning cadences. This has the further benefit of allowing for gradual, granular staging of upgrades.
Red Hat's OpenStack distribution only supports a single overcloud environment per undercloud and Director seed node. (This is a current limitation of the TripleO deployment model.)
Review #2 ScoresMirantis
Scaling - Mirantis has the edge here, especially in the scaling department. Fuel management of scaling in multiple environments and a completely automated discovery mechanism is clearly a strong benefit for administrators.
Patching - The patching aspect is on par with Red Hat.
Upgrade - Full upgrades continue to be challenging, but Mirantis does have a slight advantage with the inclusion of their Octane tooling for Fuel.
Multiple management - Mirantis gets points here allowing for numerous environments at different versions with centralized management.
Scaling - Using Director and Ironic for node management and registration/discovery is more involved. Ironic registration of nodes requires some manual data collection that, especially at scale, can be troublesome.
Patching - The patching is on par with Mirantis in process and method.
Upgrade - Upgrades are documented but slightly more involved than Mirantis due to the lack of tooling similar to Fuel Octane. The requirement of a significantly greater number of CLI commands and lack of scripting offered are a disadvantage.
Multiple management - Red Hat loses points here with the inability to manage multiple environments with a single installation. The difficult installation process has to be repeated for any additional environments.
Review #3: Downstream Testing & Stability ImprovementsIn this review we compare the stability enhancements of the two distributions. We could try to use metrics such as failures at real customers or number of critical bugs filed by customers, but generally this data is hard to come by or proprietary (or both). So instead, we will score reliability at scale based on these four factors:
- Validation testing
- Reference architectures
Validation TestingBoth Mirantis & Red Hat perform testing over and above what the community does to ensure their respective distros are resilient at scale.
The respective statistics are:
|Test Suites||Bugs fixed in OpenStack Kilo||Additional bugs fixed in Distro|
|Mirantis OpenStack 7.0||8 test suites over & above community testing: ||922||328|
|Red Hat Enterprise Linux OSP7||Unknown number of test suites. The only one we were able to confirm is Scale testing.||794||Unknown|
While we believe this to be a relevant aspect of overall product distribution, given that we do not have visibility into Red Hat’s validation testing, this comparison is clearly inconclusive, so we are omitting this section from the scoring until we can gather more concrete data. That said, we chose not to completely remove this section from the blog because we think it’s an important aspect that should be considered by potential customers. When coupled with the installation and usage experience in reviews #1 and #2, questions are raised regarding the validation and stability (as they should be, with all vendors). (And if Red Hat wants to provide the information, we're happy to include this metric in the scoring.)
PackagingThere are three levels at which we look at packaging:
The differences look like this -- first, which projects are included. Next, what middleware is packaged. Finally, which patches are included from trunk.
Both distributions include core projects, i.e. Nova, Swift, Neutron, Keystone, Cinder, and Glance. They also include Horizon, Ceilometer, Heat, and Sahara. Where the two distros deviate is that Mirantis OpenStack 7.0 includes Fuel for deployment and ongoing management and as well as Murano application catalog; RHEL OSP7 includes Director for deployment and ongoing management and Ironic for bare-metal provisioning. OSP7 also has a few projects in technology preview e.g., Trove, Designate, and Manila. However, because these are in the state of technology preview and not supported for production environments, we can't weight them heavily.
As for middleware, both Mirantis and RH use MariaDB, Galera MySQL, MongoDB, Pacemaker and Corosync in an HA configuration.
Finally, the specific patches utilized or rejected from trunk is proprietary information on both sides, so no scoring differences are made on this point, but as mentioned before, it can still be an important aspect of the customer’s decision and brought up with each vendor.
With basically equivalent pros and cons for both distros, the packaging comparison results in a draw.
Reference ArchitecturesReference architectures are a critical piece of a distro because these documents state what configurations and combinations are recommended and assure that they have been tested together. A distro can have its own reference architecture document or one created in conjunction with a third-party technology.
Both distributions offer documentation explaining the lower-level architectures involving the communication paths of services and the best practices for implementing them in suggested design scenarios.
This section of comparison results in a draw.
Review #3 ScoreMirantis
Mirantis scores reasonably well due to the completeness around a variety of factors related to resilience at scale. More OpenStack projects could be included.
Review #4: Developer productivityOpenStack, or any private cloud, is not an end in itself. Most companies build it to enable workloads like developer productivity tools, big data, NFV, etc. We see unlocking developer productivity as the most important use case, and therefore we made it a comparison review.
- PaaS support
- CMP support
- Container framework support
- App Catalog support
- the developer space is very fast-moving, and partnering with best-in-class vendors gives users maximum choice and flexibility; and
- less opinionated is better.
- OpenShift based on Kubernetes is the right choice for PaaS; and
- a prescriptive approach is better.
|PaaS Support||Approach: partnerships |
Basic Cloud Foundry
Pivotal Cloud Foundry
|CMP Support||Approach: partnerships |
|Container framework Support||Approach: partnerships |
|App Catalog||Approach: community |
We are going to declare this review as a draw. Even though the Mirantis view is that a pure-play, user-makes-the-choice, and fluid approach in a fast-moving space make more sense, a user may disagree and prefer a fully integrated stack from one vendor. So we are going to set our views aside. We are giving both distros a 7 since there is much work to be done on the developer front.
Review #4 Score
Review #5: Open Source & CommunityUsers get creative when trying to figure out which distro to buy. One way they figure this out is by studying the contributions of the vendor and to see how well they play in the community. The arguments for open source are well known; the ability to ‘look under the hood’, so to speak, and freely change and customize the code to fit your requirements. But not all players in the open source community are created equal. We look at the following aspects as to how Mirantis and Red Hat play in the OpenStack community:
- Open-source approach
- Contributions by commits
- Contributions by bug fixes
- Thought leadership
Open-Source ApproachMirantis uses open source for all their projects. All Mirantis projects are developed 100% through a meritocracy-based community approach. Mirantis also develops all their code, bug fixes, etc. upstream-first.
Red Hat also uses open source for all their projects. However, not all projects are developed in a community. For example, aspects of Director are not being developed in the community.
As a testament to Mirantis’ efforts to be upstream, Mirantis pioneered Fuel as a project upstream that has become part of the OpenStack Big Tent and is included in the distribution. In addition, Mirantis has but one version of the distribution (Mirantis OpenStack) that can be downloaded and used for free by the community. It is the same distribution for commercial purposes with the addition of support services. All fixes to Mirantis OpenStack distributions are upstream; there’s no conflict in versioning. Red Hat, on the other hand, has RDO (community edition) and RHEL-OSP 7 (commercial version), each with their respective repositories, packages, and patches.
Contributions by Commits (Kilo)We are using the Kilo release of OpenStack as a comparison point because both distros are based on that release. Both Red Hat and Mirantis are top 3 contributors to OpenStack, where both have give and take in different projects. For example, Red Hat has an advantage in the number of overall commits performed with over 5,200 to Mirantis’ over 3,400.
Contributions by Bugs Resolved (Kilo)While Mirantis has an advantage in overall bug fixes with 922 to Red Hat’s 794.
Thought LeadershipThe Mirantis-initiated Murano orchestrator/app catalog and the Fuel deployer are both official OpenStack projects. Outside of core projects, but still included in the distro, Mirantis leads on Murano, Sahara, Heat, Fuel, and Rally.
Outside of core projects, but still included in the distro, Red Hat leads on Ironic and have several projects included in their technology preview, like Designate and Trove.
Review #5 ScoreMirantis
Mirantis scores reasonably well by leading on the open-source approach, contributions by bugs resolved, and earns points in thought leadership.
WIth Red Hat’s scores higher in overall contributions by commits and their long standing, strong community presence, they gain the advantage in this review.
Review #6: ExtensibilityThis review is about the ability for users to customize and extend their OpenStack solution. This involves the ability to choose from lists of third-party plugins for their specific use case and the capability of developing their own unique plugin for deploying and managing their environments.
- Catalog of plugins
- Availability of a documented framework for DIY plugins
Red Hat also has a list of plugins available for their distribution; they do get good points for that. However, the deployment of the plugins is completely manual; CLI commands are required on each node that the plugin affects, which can be an administrative nightmare in large-scale deployments. Also, we could not find a publicly available open-source framework for creating your own plugins for their distribution. (Again, we're happy to revisit this review if such a framework is indeed available.)
Review #6 Score
Review #7: InteroperabilityOpenStack requires numerous integrations “above” the stack in terms of PaaS, containers, CMP, and other middleware, and “below” the stack in terms of hypervisors, storage, networking/SDN, and so on. This is one section that we couldn’t get hard evidence supporting the numbers of partners for Red Hat’s distribution across the technology silos, so we are focusing on the transparency of the validation process.
Review #7 ScoreMirantis has a wide array of published partners across the technology silos offering pure-play options for end users in every aspect of their solution. Mirantis’ ‘Unlocked’ program provides transparent procedures, templates, scripts and tools for validating hardware; OpenStack drivers and infrastructure-level solutions; and application-level and/or OpenStack API-integrated solutions. Mirantis offers additional validation guidance for Fuel plugins (which automates deployment of drivers and complex products with Mirantis OpenStack) and Murano app catalog packages (which orchestrate application deployments on a running OpenStack cluster). Mirantis insists that partners revalidate their products with each major release (six-month cadence). They also incorporate many community-approved open-source technology options (e.g., Ceph) in their regular distributions, in robust reference configurations, configured and deployed automatically with Fuel.
Because Mirantis OpenStack remains extremely close to OpenStack trunk, moreover, it is likely that non-validated OpenStack drivers compatible with a given OpenStack release (e.g., Kilo) will work with the corresponding Mirantis OpenStack release (e.g., MOS 7.0). Mirantis led the effort to create DriverLog, a community-maintained index of available OpenStack drivers, compatible versions, automated test results, and driver-builder contact information.
Red Hat develops their own goods in the PaaS, CMP, and Containers. We didn’t have direct access to their partner counts in many of the categories, and as a disclaimer, they are grandfathering in the hardware servers from their RHEL operating system partner program. While this would make the number of partners looks high for Red Hat, they are not actively revalidating the partner solutions specifically with their distro. These may or may not be functional with each specific OSP release.
We score the two distros highly as both do offer additional functionality through other offerings. However, since our data is incomplete with regards to Red Hat specifics, and considering their presence across the technology fields, we score them equally here. Again, if Red Hat wants to provide more information we'll be happy to review scoring for this review.
Review #8: Completeness of offeringIn comparing the completeness of the distribution offering we look beyond the software itself at aspects surrounding the use of the product: available training, professional services, and support.
- Professional services and support offerings
Mirantis offers a broad catalog of nearly 20 productized services specific to OpenStack, from design assessments to fully managed environments. Mirantis has received a long standing customer support satisfaction score of 9.2 out of 10 with their multiple tiered offerings.
Red Hat also offers OpenStack training, although it is specific to their distribution only. We don’t have the quantities of people trained and certified, but given Red Hat has a strong training corp around their operating system, we’ll give them the benefit of the doubt for their OpenStack training and score them equally.
Red Hat also has a professional services organization offering several consulting and delivery services. We don’t have their overall customer satisfaction numbers for their services and support, so giving them the benefit of the doubt again, we’ll score them on par.
Review #8 ScoresMirantis gains a slight advantage here for their training offerings in both the pure-play community and distribution specific OpenStack, whereas Red Hat only offers their distribution specific training. Their support and pro-serve offerings are on par.
SummaryMirantis has the overall advantage with a score of 62 to 53. There is a heavy advantage in the complete solution of Mirantis’ OpenStack distribution. From direct user experience benefits to community direction and participation, Mirantis offers more than the other vendor, Red Hat.
For more information and to download Mirantis OpenStack 7.0 to try it for yourself, visit www.mirantis.com.