Mirantis | The #1 Pure Play OpenStack Company

Segregation in Grizzly: Availability Zones vs. Host Aggregates

You’ve probably been wondering what to make of the new features that have surfaced in the OpenStack Grizzly release – the list is substantial. That’s why on Thursday, May 23, I’ll be presenting a live Mirantis webinar about this and many of OpenStack Grizzly’s other new capabilities. It’s free – you can sign up on the right. To give you some flavor of what is covered, I’ve put together this post covering one of the topics on the agenda.

Want to learn more?

What’s New in OpenStack Grizzly

Segregation of OpenStack nodes in Grizzly

As your cluster begins to grow in size, you will very likely begin to think about ways in which to segregate different areas of the cluster for various reasons. Perhaps you want you make sure you have multiple copies of your data in multiple regions, or you want to ensure continuity of service if a particular data center’s power goes down. On the other hand, you might want to separate different parts of your cluster based on capabilities; you may want your customers to be able to request the fastest disks, or require that a particular flavor of instance is only booted on a particular type of hardware.

OpenStack’s Grizzly release brings a number of new features related to partitioning your cluster. Some of these, such as cells, are related to scaling the OpenStack compute service. Others, such as regions, involve geographical placement of data or compute resources.

Grizzly also introduces the concept of host aggregates, which control which machines host particular VM’s. If you think that this sounds like Folsom’s availability zones, you’re not alone; the two are very commonly confused. But where availability zones are a customer-facing capability, host aggregates are meant to be used by administrators to separate hardware by particular properties, and are not seen by customers.

Let’s take a look at how these two capabilities differ, and when each is most appropriate.

OpenStack and Availability zones

Let’s start with the more familiar idea of availability zones. An availability zone is a way in which the user can specify a particular “location” in which a host should boot. The most common usage for availability zones is to group together servers in terms of, well, availability. For larger clusters, this might be defined in terms of geography, as in “as long as the Chicago data center has Internet connectivity, these servers will be available.” For smaller clusters, it might be defined in terms of more tangible measures, such as “as long as this particular power supply is up, these servers will be available.”

To specify an availability zone in which your host will boot, you simply need to specify it using the availability-zone flag, as in:

In this case, we’re specifying that the large instance clocktower  will boot up in the  chicago  availability zone.

(The availability zone for a server itself is set in the  nova.conf  file using node_availability_zone.)

So for the user, availability zones are fairly straightforward; pick a zone, start a VM. Host aggregates, on the other hand, are the administrator’s domain.

Host aggregates

Where availability zones are designed for users to be able to choose where to host their virtual machines, host aggregates are intended as a way to group servers that have a particular quality to them. While that might indeed be geographical, as in “all hosts in the Chicago data center”, or availability based, as in “all hosts in this rack”, host aggregates are instead designed for grouping servers by capability, such as “all hosts using solid-state drives,” or “all hosts with more than 32GB of memory”.

In fact, there aren’t any particular requirements for using host aggregates in OpenStack; they’re completely arbitrary, and subject to the whim of an administrator. For example, while it’s common to create aggregates such as “all hosts with SSDs” (so users know they’re getting the fastest disk access) or “all hosts with trusted hardware” (for secure applications), we can go ahead and create an aggregate such as “all hosts administered by Joe” (assuming that Joe is our ace admin, and his servers never, ever go down unexpectedly).

To create that zone, we use the aggregate-create command:

Notice that we’ve created this aggregate within the chicago availability zone, rather than the default nova zone; aggregates can be used to further subdivide availability zones, but this parameter is optional.

So the big question is, if availability zones and host aggregates both segregate a cluster, why do we need both of them?

What a host aggregate brings to the party that availability zones don’t

Availability zones are handy for allowing users to specify a particular group of servers on which they want their host to run, but beyond that they don’t do much more than serve as a bucket. In this example, using an availability zone, our users can specify that a VM should be started up in the Chicago data center.

Host aggregates, on the other hand, serve as an intelligent way for schedulers to know where to place VM’s based on some sort of characteristic. In this example, we might want to enable users to easily boot their most mission-critical VMs on servers that are administered by Joe, rather than leaving them to fate.

In general, the workflow for using host aggregates looks like this:

  1. Create a new aggregate.
  2. Set a particular property for that aggregate, such as ssd=true , or in our case, joeistheboss=true .
  3. Add qualifying hosts to this aggregate.
  4. Create a flavor that requires this particular property.
  5. Instantiate hosts using this flavor.

Let’s look at how this would work in our case.

1) First, we need to make sure that the scheduler supports host aggregates. To do that, check the  /etc/nova/nova.conf  file on the nova-scheduler server to make sure that scheduler_default_filters  includes AggregateInstanceExtraSpecsFilter , as in:

2) Next, create the aggregate itself:

This command creates a new aggregate, joesdomain , in the  chicago  availability zone, and enables us to see the id , as in:

3)  Now specify the  joeistheboss  property using that id :

4)  Of course we need to add some hosts do that aggregate so we have somewhere to boot the VM’s:

5)  To tie all of this together, create a flavor that requires the  joeistheboss  property:

This creates the new flavor and specifies the  extra_specs  property, as you can see with the  flavor-show  command:

6)  Finally, users who are particularly concerned about their VMs can use this flavor to make sure that their VMs are hosted in joesdomain :

The scheduler in this case knows the following:

  • flavor joe.large requires  joeistheboss  to be true
  • all hosts in the  joesdomain  aggregate have joeistheboss=true
  • the coruscant , tatooine , and  naboo  hosts are in the  joesdomain  aggregate

Accordingly, the scheduler knows to start this new VM on one of those three hosts.

 What do I use when?

So now that we know the difference between availability zones and host aggregates, which do we use, and when?

As a user, there’s really no decision; only admins can create host aggregates, so availability zones are your only choice, unless they’ve already been set up.

As an admin planning for your customers, however, you do have a decision to make.  In general, you’ll need to consider the following:

  • Is there a clear physical separation between hosts, either physically or redundantly?  If so, you will probably want to use availability zones.
  • Is the separation based on hardware capabilities?  If so, you will probably want to use hardware aggregates.
  • Are hosts within a particular “category” spread across multiple locations?  If so, you will probably want to use hardware aggregates so that you can group together hosts from multiple availability zones.  (In this case, you can create an aggregate with the appropriate metadata in each zone.)
  • Do you want users to consciously choose a “category” for their VMs?  If so, you will probably want to use availability zones, as users can specify them directly.

Summary

While it’s easy to mix up availability zones and host aggregates, each is used for a different purpose, and each gives you control over the segregation of your cluster in a different way. Availability zones enable users to specifically choose from a group of hosts. Aggregates enable an administrator to more easily specify the way in which hardware is utilized.  In this example we’ve chosen the somewhat lighthearted example of requiring a particular system administrator, but the point is that you have the ability to control host aggregates on completely arbitrary grounds.

Any questions, on this or other topics about what’s new in Grizzly? See the webcast or submit a comment below.

18 comments
Google Plus Mirantis

18 Responses

  1. Lingxian Kong

    hi:

    in Grizzly, there is a new feature called “cell”, what’s the diference between them(cell, availability zone and aggregate)?

    Your feedback is appreciated.

    May 19, 2013 22:36
    • Russell Bryant

      Cells exist at a layer above host aggregates / availability zones. A cell may contain one or more host aggregates.

      May 20, 2013 09:47
    • Nick Chase

      A cell is something that’s set up by an administrator and isn’t controllable at all by users; you have a parent cell with just nova-api installed, and the child cells below it that have all the nova services EXCEPT nova-api installed. When a request comes in, it’s first scheduled to a group of cells, and then sent to a particular host within the group by the group’s host-scheduler.

      So it differs from the other two is that rather than specifying it directly (as in availability zones) or indirectly (via flavors, as in host aggregates), cells are completely invisible to the user.

      Does that help?

      May 21, 2013 09:55
      • Lingxian Kong

        Thanks Nick for your reply! For what you said, I have another question. If cells are invisible to the users, how can we use it? We all know that there are some schedulers using which the request can be passed to the target cell, where the *target* cell comes from if the user can’t specify it?

        May 26, 2013 06:26
        • Russell Bryant

          There is a cells scheduler. The one in Grizzly is very primitive, and just chooses a random cell. There is a patch up right now for adding filtering and weighting to the cells scheduler, similar to the filter scheduler used for hosts. Once that is in place, more filters can be added for making more intelligent cell scheduling decisions. Here is the patch: https://review.openstack.org/#/c/16221/

          May 27, 2013 07:29
  2. Russell Bryant

    I think this post misses a very critical piece of information. Host aggregates and availability zones are effectively two views of the same thing in Grizzly. Host Aggregates *are* availability zones. An administrator creates a host aggregate and its associated metadata. A host aggregate is the definition of an availability zone exposed to a user. The old configuration option in nova.conf is deprecated and will be removed in favor of creating AZs using host aggregates.

    https://blueprints.launchpad.net/nova/+spec/aggregate-based-availability-zones

    May 20, 2013 09:46
  3. Bulent Abali

    Imaginary HA scenario: Suppose we want to boot VMs in pairs. Each VM in a pair of VMs must run on one of two different geographic locations for HA reasons. How would you specify this with the host aggregates, and can filters be setup to schedule VM pairs as required? Thanks for the advice

    May 31, 2013 05:52
    • Russell Bryant

      The only way to do that in Grizzly is to just set up completely independent nova deployments (regions). Each would have its own API endpoint. You would talk to each one to start an instance in that region.

      Havana will have a filter scheduler for cell scheduling which will allow us to implement more interesting policy for cell scheduling (where multiple cells can be in different locations under the same API endpoint).

      May 31, 2013 17:14
  4. Paul Max

    Hi Russel, i have gone thru this post as well as your blog on Host Aggregates and Availability Zones. If I understand this correctly, going forward, all availability zones will have a host aggregate tied to it but not all host aggregates necessarily need to have an availability zone. Is this accurate?
    Also, is there a direct exposure of availabilty zones via OpenStack API? (i.e can i list which availability zones are, umm, available and then pick one for launching my instance?) Or is this all done thru host aggregate related APIs?
    If it is the latter, then its not a fool-proof method because not all host aggregates would be tied to availability zones.
    Would appreciate if you can correct me where I got it wrong? thanks!

    June 4, 2013 06:55
  5. Paul Max

    I have a few questions around Availability zones and host aggregates in Grizzly.

    Are all availability zones always associated with host aggregates but not all host aggregates are associated with availability zones, correct? Or is this changing so that there is always 1:1 correspondence between availability zone and host aggregates?

    Is there a way for non-admin users to discover availability zone via API? Access to host aggregate APIs seems to be limited to admin users. Without discovering AZs, it would be hard for end users to place their VM in appropriate AZ.

    Would appreciate help! TIA. -paul

    June 4, 2013 08:49

Continuing the Discussion

  1. Dell Open Source Ecosystem Digest #18. Issue Highlight: “OpenStack Rocks in Poland” - Dell TechCenter - TechCenter - Dell Community

    [...] “OpenStack Grizzly Webinar Preview: Availability Zones vs. Host Aggregates” by Nick [...]

    May 17, 201309:56
  2. Dell Open Source Ecosystem Digest #18. Issue Highlight: “OpenStack Rocks in Poland” | ServerGround.net

    [...] “OpenStack Grizzly Webinar Preview: Availability Zones vs. Host Aggregates” by Nick [...]

    May 17, 201310:00
  3. Server King » Dell’s Digest for May 18, 2013

    [...] “OpenStack Grizzly Webinar Preview: Availability Zones vs. Host Aggregates” by Nick [...]

    May 18, 201315:36

Some HTML is OK


or, reply to this post via trackback.