 |
OpenStack Networking – FlatManager and FlatDHCPManager
July 26, 2012
Over time, networking in OpenStack has been evolving from a simple, barely usable model, to one that aims to support full customer isolation. To address different user needs, OpenStack comes with a handful of “network managers”. A network manager defines the network topology for a given OpenStack deployment. As of the current stable “Essex” release of OpenStack, one can choose from three different types of network managers: FlatManager, FlatDHCPManager, VlanManager. I’ll discuss the first two of them here. FlatManager and FlatDHCPManager have lots in common. They both rely on the concept of bridged networking, with a single bridge device. Let’s consider her the example of a multi-host network; we’ll look at a single-host use case in a subsequent post. For each compute node, there is a single virtual bridge created, the name of which is specified in the Nova configuration file using this option: flat_network_bridge=br100 All the VMs spawned by OpenStack get attached to this dedicated bridge.  Network bridging on OpenStack compute node This approach (single bridge per compute node) suffers from a common known limitation of bridged networking: a linux bridge can be attached only to a signle physical interface on the host machine (we could get away with VLAN interfaces here, but this is not supported by FlatDHCPManager and FlatManager). Because of this, there is no L2 isolation between hosts. They all share the same ARP broadcast domain. The idea behind FlatManager and FlatDHCPManager is to have one “flat” IP address pool defined throughout the cluster. This address space is shared among all user instances, regardless of which tenant they belong to. Each tenant is free to grab whatever address is available in the pool. FlatManagerFlatManager provides the most primitive set of operations. Its role boils down just to attaching the instance to the bridge on the compute node. By default, it does no IP configuration of the instance. This task is left for the systems administrator and can be done using some external DHCP server or other means.  FlatManager network topology FlatDHCPManagerFlatDHCPManager plugs a given instance into the bridge, and on top of that provides a DHCP server to boot up from. On each compute node:
 FlatDHCPManager – network topology As for dnsmasq, FlatDHCPManager creates a static lease file per compute node to guarantee the same IP address for the instance over time. The lease file is constructed based on instance data from the Nova database, namely MAC, IP and hostname. The dnsmasq server is supposed to hand out addresses only to instances running locally on the compute node. To achieve this, instance data to be put into DHCP lease file are filtered by the ‘host’ field from the ‘instances’ table. Also, the default gateway option in dnsmasq is set to the bridge’s IP address. On the diagram below you san see that it will be given a different default gateway depending on which compute node the instance lands.  Network gateways for instances running on different compute nodes
Below I’ve shown the routing table from vm_1 and for vm_3 – each of them has a different default gateway: root@vm_1:~# route -n root@vm_3:~# route -n
By default, all the VMs in the “flat” network can see one another regardless of which tenant they belong to. One can enforce instance isolation by applying the following flag in nova.conf: allow_same_net_traffic=False This configures IPtables policies to prevent any traffic between instances (even inside the same tenant), unless it is unblocked in a security group. From practical standpoint, “flat” managers seem to be usable for homogenous, relatively small, internal corporate clouds where there are no tenants at all, or their number is very limited. Typically, the usage scenario will be a dynamically scaled web server farm or an HPC cluster. For this purpose it is usually sufficient to have a single IP address space where IP address management is offloaded to some central DHCP server or is managed in a simple way by OpenStack’s dnsmasq. On the other hand, flat networking can struggle with scalability, as all the instances share the same L2 broadcast domain. These issues (scalability + multitenancy) are in some ways addressed by VlanManager, which will be covered in an upcoming blog posts. 18 comments18 Responses
|
Piotr Siwczak
Manish
). In FlatDHCP, VMs within a compute node are able to talk to each other but not outside. So, if we consider your diagram 10.0.0.5 cannot ping/ssh to 10.0.0.2. But 10.0.0.5 can ping/ssh to 10.0.0.6. I’ve added rule for ssh and ping. Could you possibly know what am I doing wrong? My nova.conf also matches what you gave above.
Tamale
hjg
Antonio
S.A.
Dustin
Stratos
It’ll be nice to see nova.conf files additionally.
August 3, 2012 06:17