Long Live Enterprise Clouds

One active arena of cloud blog battles lately was about open vs. enterprise clouds. Randy Bias of Cloudscaling is on the forefront of open cloud advocacy with a clear stance thatanything that is not an open cloud – is a legacy cloud. Massimo, in his recent blog did a great job categorizing different types of clouds. He, too, gently implies that as organizations’ IT strategies mature, they will be making a shift away from “orchestrated clouds” to “design for fail clouds.” Much of the “cool cloud thinking” today is based on the notion that a) open clouds are good; b) writing applications for infrastructure (rather than tuning infrastructure for the apps) is the future of IT.

I disagree with both of these.

So let’s start with “open cloud is good, enterprise cloud is bad” stance. In my view, making this comparison is like saying a kitchen knife is better than a swiss-army knife. A kitchen knife is simple, sharp and has no moving parts. Just like an open cloud it is designed to be a simple solution to a single, concrete problem. Ever try to prepare a full meal with a swiss-army knife? Sure, when you go camping, it’s probably fine. But when your mother-in-law is coming to dinner?

The fundamental difference between enterprise and open clouds is in their approach to the problem. Open cloud mentality comes from a service provider view of the world where the cloud is not built to support the business, but rather IS THE BUSINESS. The approach is to build from the bottom up for a narrow problem, just like one would if you were a software company and the cloud was your product, aimed at capturing a chunk of some market.

In the open cloud world, apps do not dictate the infrastructure; you start with the infrastructure, and go up the stack. In other words, the information technology infrastructure dictates the way the application solves a business problem.

In the enterprise world – it’s precisely the other way around . IT exists to support the business. The applications are king and they dictate the IT underneath. Is this the best scenario when it comes to simplifying the infrastructure and containing its cost? Definitely not! Is there an alternative for the enterprise? Definitely not! Reason? There’s an irreducible domain knowledge gap between IT and business. In the case of AWS, Salesforce or Google – IT is the business. In enterprise – IT’s job is to salute the business and support it.

Let’s take a concrete example (real life story by the way). We are now enjoying the dawning of the age of big data. Say some entrepreneur decides to take advantage of Hadoop and Lucene and build a new engine for parsing and aggregating bioinformatics data, and can extract results that were never before possible. He then sells his marvelous, vertically focused innovation to Pharma companies. If I’m at Pfizer and I don’t buy it, but my rivals at Roche buy – I get left behind. But say my IT does not do Hadoop and Lucene and I can’t take the solution to run in a public cloud because of regulatory compliance. Now what do I do?

If you guessed that I call my CIO and tell him to stand up the environment that will support this, you’re right. IT has to follow the lead of the business, or the whole business fails. This happens over and over again. Over time, IT has to support an extremely diverse environment. Conceivably, the gap may shrink over time as IT becomes an ever increasingly dominant business process in any vertical, but don’t plan on it happening this month. Or even next month.

Now, there is a common view that it doesn’t have to be that way that stems from an elegant but very one-dimensional comparison between IT infrastructure and electricity. I.e. application infrastructure is a commodity, just like electricity. All apps should be built to run on top of this common, standardized infrastructure, and just like we all have the same shape of electrical outlets (except for, well, the Europeans) this is where we’ll all be soon.

It sounds great, but I’m sad to say that I have to call bullshit. Electricity and application infrastructure is not the same. Unlike with electricity, there is massive innovation at the bottom part of the application stack. We didn’t even use virtualization until recently. Yesterday it was all about disk, today it is all about SSD.

We don’t know what new paradigms will emerge in the coming years. This innovation shakes the entire stack. Going back to my example, Hadoop was not widely used just not too long ago. Had it not existed, the new app would not have been possible and IT would not have had to buy new infrastructure and deploy unknown middleware on it. But because it does, IT has to adjust. And tomorrow will be a new paradigm and IT will have to adjust again and again.

Commoditization and standardization can only happen in stagnant industries like electricity generation and distribution or growing potatoes, where the world has pretty much stopped. Until that kind of stable stagnation becomes a common theme in the application infrastructure space, there will always be expensive enterprise clouds and open, inexpensive, commodity clouds. The enterprise will be constantly configuring its swiss army knife, aimed at minimizing the pain of dealing with diversity in the stack.