Google Cloud Nested Virtualization: How to Enable It on a GCE Instance

Nick Chase - July 17, 2019 - ,

There are times when you need to run a virtual machine — but you’re already ON a virtual machine.  Fortunately, it’s possible, but you need to enable nested virtualization.  For me, this comes up often when I’m running OpenStack or Kubernetes on a Google Compute Engine instance on the Google Cloud Platform (GCP).  To help solve the problem, you can enable Google Cloud nested virtualization.  Follow these steps:

  1. Install the latest version of the Google virtualization beta command-line tool  gcloud (no need to sign in to your Google account).
  2. Create a new instance so you have a base disk to work with.  Because you’ll eventually want to use the image in a zone that includes Google Cloud nested virtualization,  create it in zone us-central1-b.  You can do this from the UI, or using the command line. By default, the disk will have the same name as the instance:
    gcloud compute instances create temp-image-base --image-family=projects/ubuntu-os-cloud/global/images/family/ubuntu-1604-lts --zone=us-central1-b
  3. Stop the instance:
    gcloud compute instances stop temp-image-base --zone=us-central1-b
  4. Now create a new disk, based on that disk, with Google Cloud nested virtualization enabled:

    gcloud compute images create nested-vm-image \
      --source-disk=temp-image-base --source-disk-zone=us-central1-b \
      --licenses="https://www.googleapis.com/compute/v1/projects/vm-options/global/licenses/enable-vmx"
  5. Next create the new instance using the new image:
    gcloud compute instances create nested-vm --zone us-central1-b --image=nested-vm-image --machine-type=n1-standard-32 --boot-disk-size=250GB
  6. Connect to the instance:
    gcloud compute ssh nested-vm --zone=us-central1-b
  7. Confirm that nested virtualization is enabled by looking for a non-zero response to:
    > grep -cw vmx /proc/cpuinfo
    > 1
  8. Finally, install a hypervisor  for your nested GCP (KVM is a fantastic choice.)
    sudo apt-get update && sudo apt-get install qemu-kvm -y

There you have it. That’s all you need to know about setting up nested virtualization. GCP can now run another VM instance from within the platform.

While these instructions only apply to GCP when you want to enable nested virtualization, Google Cloud isn’t the only platform where you’ll encounter this problem., you can easily run into the same issues on bare metal machines running an OS like Windows Server, or on cloud instances that come from another cloud platform, such as Amazon Web Services.  The solution depends on your environment. For example, if you’re trying to run nested virtualization on your desktop intel machine (maybe for better data privacy), it might be a matter of changing BIOS settings. For Hyper-V, you need to change settings on the VM itself (assuming that other requirements, such as an appropriate processor, are met).

For Amazon Web Services, in most cases, when you run an “instance” that is, itself, a virtual machine, so if you want to run Virtualbox or other virtualization software such as VMware or Hyper-V, you’re going to have a problem.  Fortunately, many cloud service providers also include the ability to rent “bare metal” instances, which give you direct access to the hardware. From there, you will need to follow the same instructions as you would for local hardware.

banner-img
From Virtualization to Containerization
Learn how to move from monolithic to microservices in this free eBook
Download Now
How is Cloud Native Changing the Landscape of Edge and 5G? [Recording]

Late last year, Mirantis hosted a Cloud Native and Coffee panel featuring CTO Adam Parco, Global Field CTO Shaun O’Meara, Director of Technical Marketing Nick Chase, and special guest Darragh Grealish, CTO of 56K Cloud. Below are highlights of the discussion that touch on what edge is and how developers can bring cloud native innovation to edge computing and 5G. Watch …

How is Cloud Native Changing the Landscape of Edge and 5G? [Recording]
Moving to Cloud Native: How to Move Apps from Monolithic to Microservices

Enterprises face the challenge of consistently deploying and managing applications in production, at scale. Fortunately, there are more technologies and tools available today than ever before. However, transitioning from a traditional, monolithic architecture to a cloud native one comes with its own unique challenges. Below, you will find a list of the critical first steps you need to take when …

Moving to Cloud Native: How to Move Apps from Monolithic to Microservices
Mirantis Newsletter - January 2022

Every month, Mirantis sends out a newsletter chronicling top industry and company news. Below you’ll find links to blogs, tutorials, videos, and the latest updates to our enterprise, open source, and training offerings. If you don’t currently receive the newsletter, you can subscribe by clicking the button on the top right. Mirantis Brings Secure Registries to Any Kubernetes Distro Launched earlier this …

Mirantis Newsletter - January 2022
Technical training
Learn Kubernetes & OpenStack from Deployment Experts
Prep for certification!
View schedule
LIVE WEBINAR
Manage your cloud-native container environment with Mirantis Container Cloud

Wednesday, January 5 at 10:00 am PST
SAVE SEAT
Mirantis Webstore
Purchase Kubernetes support
SHOP NOW