First Implementation of NVMe over Fabric Support in OpenStack

Brad Vandehey - October 25, 2016 -

Fast growth of data and technology innovation is driving the emergence of a new/next generation Storage architecture in Cloud Data Centers. Storage is evolving from proprietary Highly Available, Dual Controller custom appliances to open software defined clustered/scale-out storage based on industry standard servers.

As part of this storage evolution, NVMe SSD adoption is increasing in Data Centers by moving the low latency storage closer to the processor thru PCI connectivity and accessed over NVMe (NVM Express (NVMe)) standard protocol. NVMe local usage is well understood and the initial adoption of these SSDs has demonstrated performance improvements for IO-intensive applications, due to the drives’ latency and throughput characteristics.

Another, less obvious progression happening in Data Centers is to disaggregate storage from compute to improve Data Center operational efficiency and increase flexibility at the rack level. Released in June 2016, the NVMe over Fabrics (NVMeoF) specification (NVMe Over Fabrics Overview) relies on RDMA to access high performance remote SSDs without sacrificing performance.

To bring this innovation more quickly to market, Intel, Mirantis and Supermicro have partnered to engineer the first implementation of NVMeoF that unlocks the value of NVMe over Fabrics for OpenStack. It may be years before we see such implementations in traditional storage systems. OpenStack provides the ideal environment to innovate quickly around NVMe and NVMeoF technologies.

Mirantis added a new Cinder driver to support the NVMe over Fabric reference implementation based on Intel SPDK NVMeoF target software (SPDK NVMeoF Target), and modified Nova to support attaching NVMeoF volumes to VMs. The implementation was validated with a SuperMicro All-Flash NVMe 2U server running Intel SPDK NVMeoF target. This All-Flash Array supports up to 24 NVMe Drives ( (SYS-2028U-TN24R4T) currently housing Intel® Data Center P3700 NVMe SSDs and Mellanox NICs for RDMA support. At OpenStack Summit in Barcelona October 25-28, 2016 we will be demonstrating this functionality and highlight how we are making this available to the OpenStack community for further innovation.

Next Steps

Following OpenStack Summit we plan to extend this work to include upstreaming into the Ocata release and delivering performance benchmarks.

Join Us

The demonstration will be shown on Thursday, October 27, at 11:25 AM, in the Marketplace Theatre, as part of the Mirantis breakout session at OpenStack Summit in Barcelona, Spain, October 25-28, 2016.

From Virtualization to Containerization
Learn how to move from monolithic to microservices in this free eBook
Download Now
How is Cloud Native Changing the Landscape of Edge and 5G? [Recording]

Late last year, Mirantis hosted a Cloud Native and Coffee panel featuring CTO Adam Parco, Global Field CTO Shaun O’Meara, Director of Technical Marketing Nick Chase, and special guest Darragh Grealish, CTO of 56K Cloud. Below are highlights of the discussion that touch on what edge is and how developers can bring cloud native innovation to edge computing and 5G. Watch …

How is Cloud Native Changing the Landscape of Edge and 5G? [Recording]
Moving to Cloud Native: How to Move Apps from Monolithic to Microservices

Enterprises face the challenge of consistently deploying and managing applications in production, at scale. Fortunately, there are more technologies and tools available today than ever before. However, transitioning from a traditional, monolithic architecture to a cloud native one comes with its own unique challenges. Below, you will find a list of the critical first steps you need to take when …

Moving to Cloud Native: How to Move Apps from Monolithic to Microservices
Mirantis Newsletter - January 2022

Every month, Mirantis sends out a newsletter chronicling top industry and company news. Below you’ll find links to blogs, tutorials, videos, and the latest updates to our enterprise, open source, and training offerings. If you don’t currently receive the newsletter, you can subscribe by clicking the button on the top right. Mirantis Brings Secure Registries to Any Kubernetes Distro Launched earlier this …

Mirantis Newsletter - January 2022
Service Mesh for Mere Mortals
A Guide to Istio and How to Use Service Mesh Platforms
Istio in the Enterprise: Security & Scale Out Challenges for Microservices in k8s

Presented with Tetrate
Mirantis Webstore
Purchase Kubernetes support