What is Kubernetes hybrid cloud/multi-cloud?

Most users of Kubernetes need to run multiple clusters: for dev, test, production, to segregate the work of teams, projects, or application types. Some users (an increasing number) also want freedom to run them across multiple private and/or public cloud platforms. That’s Kubernetes multi-cloud. This is sometimes called “hybrid cloud,” where an organization runs one private cloud and one public cloud estate.

Benefits of multi-cloud

Like running multiple Kubernetes clusters on the same infrastructure, running multiple clusters on multiple cloud infrastructures lets you isolate tenants and workload types from one another, gain resilience by distributing critical workloads to different availability zones, and optimize placement of workloads (e.g., put applications and data in the same regions as customers). Using multiple public clouds can also help with:

• Avoiding public cloud lock-in: By building your applications so that they can take advantage of multiple clouds, you are ensuring that you are not beholden to a single vendor. Should issues with that vendor arise, you can simply take your clusters and workloads elsewhere.

• Cost arbitrage: Perhaps an extreme use of avoiding vendor lock-in is to constantly monitor the cost of various public cloud solutions and run workloads where they are most economical, lowering operating costs.

• Enhancing availability and disaster recovery: Public cloud providers work hard behind the scenes to ensure that their services remain available. But problems do happen, and when they do, you often see even extremely large providers knocked offline. By architecting your infrastructure to work with multiple cloud providers, you can ensure that traffic simply fails over to another cloud.

Challenges of multi-cloud Kubernetes

As you might imagine, working with a Kubernetes multi-cloud environment is more complex than working with a single cluster, or even a multi-cluster environment on a single infrastructure or platform. Multi-cloud Kubernetes also involves:

• Diverse APIs: Each public cloud provider has its own way to create and manage resources, and its own API(s) for doing so. Creating a virtual machine or even an entire Kubernetes cluster on Amazon Web Services is conceptually similar, but technically completely different from doing the same task on Google Cloud or Microsoft Azure. And it’s hard to paper over these differences by choosing a single tool (e.g., Ansible) equipped with so-called “providers” for different cloud APIs. Making things more complicated: the communities around different clouds tend to prefer unique tools (like Amazon CloudFormation) for automating and allocating resources. More complicated still: public cloud services differ importantly from one another in some respects — some do things like object storage or access and permissions in unique ways — and it makes sense to leverage optimal solutions on each platform. These differences require you to have not just management code for each provider, but also the skills to create and manage that code.

• Differences in monitoring: Similarly, each cloud provider has its own monitoring service, and multiple providers may not provide data in the same form or in the same way.

• Networking: Different clouds are, of course, going to be on different networks. As a result, you will need to consider not just the ability for pods to reach each other, but their ability to discover each other in the first place.

• Security: Security is always a concern, but any time you are dealing with the public internet, those concerns are naturally magnified. A Kubernetes multi-cloud architecture requires extra attention to vulnerability points.

So what can we do about these issues?

Kubernetes multi-cloud solutions

Some of the issues that arise in creating a Kubernetes multi-cloud architecture are purely technical. For example, you can connect multiple clusters as though they are on a single network using a Virtual Private Network or other networking tools such as Tungsten Fabric or Project Calico.

Other issues are more cultural: if you use a provider’s proprietary tools and APIs, you are effectively locking yourself into their environment. Sure, you can move your data any time, but what about all that code that relies on those products for managing the environment and workflow?

The best way to make use of a multi-cloud Kubernetes environment is to make sure that you’re using the capabilities of Kubernetes itself, and that any tools you use to deploy or manage it can run on any environment you might consider. The same goes for Kubernetes monitoring[b][c][d]; if you rely on standard Kubernetes-adjacent tools such as Prometheus and Grafana, you won’t be introducing incompatibilities and complexity.

For example, Mirantis Kubernetes Engine (formerly Docker Enterprise) can run on AWS, Google Cloud, VMware, and other cloud providers, so you can move your workloads at any time. What’s more, the Kubernetes clusters it deploys are standard Kubernetes, so your applications will work with any standard Kubernetes cluster. Mirantis Container Cloud even simplifies the process by abstracting the creation of clusters itself so that all you need to do is specify the provider.

In short, the key is to ensure that you’re making use of Kubernetes itself, and not the proprietary APIs of individual providers. In this way you’ll create a Kubernetes multi-cloud environment that is manageable and flexible.