Integrating OpenStack Cloud Nova Volume storage with Isilon

What do you use to store your OpenStack volumes? If you’re building an OpenStack cloud, you’ll need that block level storage to give to your instances through an iSCSI session by default. If  you want to have intelligent, flexible, scalable, and reliable physical storage for this purpose, one alternative you might consider is the Isilon IQ clustered storage system. We’ve created a driver you can use with it.

Now a part of EMC, Isilon was created to meet the needs of high-performance computing environments to provide both scalability and performance. The core of this uses an intelligent OneFS distributed file system. Key features:

• 60-second on-the-fly scaling of capacity and performance
• A single file system with a single global namespace
• The ability to place mission-critical, latency-sensitive data on SSDs
• High utilization rates going up to 80%.

• iSCSI
• Xen Storage Manager
• Nexenta
• SAN-style (Solaris-hosted iSCSI, SolidFire)

Let’s look at how this driver works.

Nova Volume driver

Nova uses the volume manager to control attachable block storage devices (volumes) and this control strategy is seen in the driver used:

This is a simplified scheme, but it does a good job of describing what happens. Volume creation, attaching, and detaching tasks go through the volume driver to physical storage—which is how the latter understands calls that come through Nova API. But there was no Isilon driver for OpenStack until we decided to change that.

Rationale for running OpenStack Nova on Isilon

Reliable and scalable storage solution is the base of any kind of cloud deployment and Isilon is an good example of such storage. Deploying a cloud system means not only running virtual machines but also storage for vital data such as databases and application data sets. All this data should be stored in a reliable/redundant way that allows it to be always available above minimum I/O requirements. That makes Isilon and OpenStack an effective combination.

By using this driver, OpenStack Compute virtual machine volumes support the following Isilon features:

• Single, shared global namespace
• Isilon IQ Storage seamlessly scaled to 15.5 petabytes
• Support for InfiniBand or Gigabyte Ethernet for intracluster communication
• Unmatched aggregate throughput from a single file system up to 100 gigabytes per second
• Industry-leading high availability for clustered storage systems
• Powerful, easy-to-use, web-based management
• 60-second scaling of both capacity and performance
• Unlimited snapshots and clones

How the OpenStack Nova Volume Isilon driver works

When you use our driver (download link at the end of the post) with Isilon for OpenStack, the workflow looks something like this:

1. Nova-compute tells nova-volume to create a volume through the queue.
2. Nova-volume goes to the nova.conf file and finds out what driver to use (the Isilon driver in this case).
3. Nova-volume forwards the Isilon driver all information needed for the volume creation.
4. The Isilon driver changes nova API calls to the calls Isilon IQ Storage can understand—there are some group of flags in the nova.conf file that help this process.
5. Isilon creates a volume (a logical unit in Isilon’s terms) and the ISCSI target for it and returns a response back to the driver.
6. The driver sends information about volume creation (successful or not) to the nova-volume.
7. Nova-volume writes data and creates a volume in the DB.

You can download the driver here. We look forward to your comments.