Remember the rise of OpenStack? First there was Amazon and cloud. And then VMware said that cloud can also be private. And then Eucalyptus and CloudStack said that private cloud should be open. And then came Rackspace with OpenStack and said that private cloud should be ever pluggable and flexible. And all the vendors cheered! (And yes, that includes Mirantis). All hail OpenStack, protector of DIY private cloud and conqueror of Amazon!
But in the end few were able to find refuge from AWS through OpenStack. So everybody ran to find new cover. Today that new cover is starting to look a lot like Containers-as-a-Service (aka unstructured PaaS) propelled forward by Google and Kubernetes. We are effectively observing Cloud_Opinion’s law coming in effect: “Every vendor that can’t compete in Cloud chooses hybrid as their strategy.”
Multi-cloud is the new private. Kubernetes is the new OpenStack. But is there an opportunity to learn from the past and do it better this time around? So far, at least some of the parallels are concerning. Let’s examine them…
Before OpenStack there was Eucalyptus and CloudStack. Both were opinionated implementations of a private cloud reference architecture. Their opinionated nature stifled broad customer adoption, but things were chugging along. Then came OpenStack, which played the unopinionated DIY card and the following happened:
Fast forward to today. There is Cloud Foundry and there is OpenShift. Like Eucalyptus and CloudStack, both are opinionated. So things are chugging along, but neither is a runaway success. Both are swimming against the strong current of enterprises’ desire to DIY.
Along comes a Kubernetes-fueled wave of multi-cloud CaaS and, sure enough, unopinionated DIY wins:
Kubernetes CaaS won. There is no denying it. Mesosphere is now a Kubernetes CaaS. Super opinionated Pivotal Cloud Foundry PaaS is now a Kubernetes CaaS. And even the very conservative Gartner in its May report threw out some not so conservative statements: “Platform-as-a-Service vendors… are pivoting to offer CaaS solutions… Such platforms can ultimately make the multicloud promise…a reality.”
I think there are two ways to look at it. The optimist in me cheers because the unopinionated multi-cloud CaaS is finally getting that bottoms up developer adoption that structured PaaS could never enjoy. The pessimist in me ponders how, once again, the industry caves in to thirst for DIY, choosing short term speed of adoption over long term operational sustainability. We are heading towards a composable, multi-cloud CaaS that is a plethora of best of breed building blocks – Docker, Kubernetes, Helm, Istio, Spinnaker etc. – developed by a variety of loosely coupled interests; each with its own release cycle. So how will we operate all of this stuff?
Operational challenges are exactly what stifled private cloud and dragged OpenStack down with it. So as we move from structured PaaS to composable CaaS are we not marching down the same exact path again?
Opinionated solutions delivered as software can’t win against opinionated solutions delivered as cloud. So the only way to move the infrastructure market with software is to play the DIY card, which after a while makes operational challenges more acute and, consequently, the cloud delivery model more attractive. Is it a spiral of doom? To get adoption for private IaaS we made it DIY friendly with OpenStack. But then we stumbled with operations and surrendered to public cloud. Now moved to private PaaS software. To get adoption for private PaaS software we are now making it DIY friendly by moving to CaaS. You can guess what will happen next.
Boris,
This is an interesting and thought provoking post.
First, the implication that OpenStack is dead is likely to get the post lots of attention, and play to a certain crowd, but we both know that’s far from the truth. I’ll just leave it at that and move on to your central point.
I think that if you look at the most successful open source stack ever created that drove a massive industry, the LAMP stack, you’ll see a pattern that involved multiple open source projects, each with their own communities. This was not one community creating a monolith, and that’s why it worked. Each community was able to focus on what they were good at, which wasn’t feasible in just one community. Although it was somewhat opinionated as a concept, that did get the operational burden you allude to under control, and yet it was still modular enough so that each layer could be swapped out. They seemed to have struck the right balance.
We have every opportunity to replicate this success in the open infrastructure / open cloud area, but only if we work across communities and listen to the users. It’s clear from every user I talk to that they are in fact struggling with pulling all of the pieces together, particularly as it relates to operating the beast once assembled, but that’s what they want, we just aren’t there yet. Within OpenStack itself, we have started to cull options and projects to essentially make it more opinionated, which is helping.
Our opportunity, which is far from trivial, is to rise to the occasion across communities to actually serve the needs of the users, rather than succumbing to ego driven development. If we can’t put aside the egos, AWS will absolutely eat everyone’s lunch. In every industry. Period. We can meet at Wholefoods to discuss it.
To give a concrete example, eBay is perhaps the largest OpenStack user in the world AND, I believe one of the largest deployment of Kubernetes. They deploy them together, and it’s very powerful! What works well about that opinionated combination, and what doesn’t? How are the participants in each community doing when it comes to listening to what eBay actually needs and wants out of the combination? It will take more than Kubernetes+OpenStack to form the lamp stack of the cloud, but I believe they’re a great place to start.
We’re meeting today, in San Francisco, with members of many open source communities looking at how to build the right stack for edge computing, including leaders from eBay doing it in the real world with kubernetes+openstack. This is one concrete step we can take to build something that is both diverse and open, while also being opinionated enough to actually operate at scale.
Join us. http://www.opendevconf.com
Mark Collier
COO, OpenStack Foundation
@sparkycollier
Mark – thanks for the comment. First, I am not saying that OpenStack is dead. That would be a violation of my fiduciaries as a board member =), not to mention not good for my business. My point is that it didn’t replace AWS, which is what many vendors jumping onboard hoped it would do…. I think we can all agree there.
Re: your comparison with LAMP stack, I think it’s a good one. Some of the building blocks of the new lamp stack for the cloud looks like Kubernetes+Istio+Spinnaker. There are still critical pieces in this cloud-LAMP stack that are missing and I believe there may be opportunities to leverage the power of the OpenStack community to fill in some of these gaps. Moreover neither Istio or Spinnaker are attached to any foundation still so maybe there is a play for OpenStack there? There are blank spaces too that can be filled out within that stack by new projects in the service management space as well. Pretty convinced thought that projects like Nova and Neutron won’t be the critical building blocks of the cloud lamp stack…. happy to stand corrected.
>> It will take more than Kubernetes+OpenStack to form the lamp stack of the cloud, but I believe they’re a great place to start
I fully agree. I also believe that this goes beyond just infrastructure components.
One area where companies will need a lot of help is — how you deliver and manage applications/services on top. While Spinnaker, which Boris mentioned, is great for managing individual CD pipelines, it’s far from being a complete answer to all requirements from an enterprise.
We are actually building something that will help solve this particular problem. Allowing Dev and Ops to think in terms of applications and services, not just containers and infrastructure primitives. Will be available in the open-source in Oct – http://aptomi.io/
I’d love to learn more, Roman. Please drop me a note: mark@openstack.org