Automated deployment of OpenContrail with SaltStack

This week, tcp cloud® launched a new version of TCP Virtual Private Cloud built on OpenContrail as SDN / NFV Controller. We decided for this network setupneutron plugin after spending more than 6 month of development and 2 month of production environment running vanilla OpenStack implementation with High Availability Neutron L3 Agent. We have worked at installation, development, testing and measuring of OpenContrail networking since OpenStack Summit at Atlanta, where we saw it for the first time. Our main contribution to this project was development of automated deployment for complete OpenStack infrastrucute with OpenContrail as neutron plugin. 

OpenContrail is released under an Apache 2.0 License by Juniper. To understand how it works you should read the slides and overview at opencontrail.org, as well as the documentation available at juniper.net or the book Day One Understanding OpenContrail.

This will be a mostly technical, but I will not talk about what OpenContrail actually does or how to deploy whole OpenStack or how powerful this solution really is. I am going to explain how to use our salt-opencontrail-formula for integration with an arbitrary OpenStack. 

I have to say that it was a little bit challenge for us, because it is not common vendor plugin that we are used to work with and it is not actually part of the official OpenStack community repositories? (maybe in Juno release Contrail will be included as official vendor plugin). As John Deatherage wrote at his blog, there are several ways how you can deploy Contrail, but if you want to deploy it as production system you should not use any of these. The reason is that the default deployment is done by Fabric and installs whole Openstack environment along the Contrail services. You do not have possibility to customise the OpenStack part and you do not know how it is actually configured. There is no official documentation for manual deployment except GitHub. OpenStack is modular system with various architectures and components. Therefore you should have possibility todefine properties. This is the reason why we refactored the installation code from Fabric and created a new SaltStack formula.

Prepare Environment

We use RedHat based OpenStack Havana running on CentOS 6.4 and OpenContrail 1.05. Download rpm package from juniper website and create local YUM repository for all nodes in infrastructure. Do not use RDO OpenStack Havana, because OpenContrail contains modified nova and neutron packages. Basically you can prepare these control nodes with CentOS 6.4:

Default Fabric automation runs setup_all with following set of tasks:

Against them we have created these sls roles, which can be use independently and even redundantly for high availability (e.g. 2 control nodes):

I cover just formula and installation for OpenContrail, but in tcp cloud we have created formulas for whole OpenStack environment including keystone, mysql, glance, cinder, nova, heat with modular neutron, cinder and nova plugins. These formulas might be released in the future. They are included inside of product TCP Private Cloud. For this purpose I describe manual changes inside of OpenStack configuration files that are different from http://docs.openstack.org

Deploy OpenStack

You should install OpenStack environment (Contrail repository only) and customized these specific projects:

NOVA

set libvirt vif driver in /etc/nova.conf at Controller and Compute node.

libvirt_vif_driver = nova_contrail_vif.contrailvif.VRouterVIFDriver

NEUTRON

Install openstack-neutron-server and openstack-neutron-contrail at Controller node. There must be configured: 

neutron.conf

core_plugin = neutron.plugins.juniper.contrail.contrailplugin.ContrailPlugin

/etc/neutron/plugins/opencontrail/ContrailPlugin.ini

[APISERVER]
api_server_ip = 10.0.106.34
api_server_port = 8082
multi_tenancy = False

[KEYSTONE]
admin_user=admin
admin_password=pswd
admin_tenant_name=admin

HORIZON

Install contrail-openstack-dashboard and verify these modules /etc/openstack-dashboard/local_settings 

HORIZON_CONFIG['customization_module'] = 'contrail_openstack_dashboard.overrides'

Prepare OpenContrail controller

Because of standalone solution, you have to install basic rabbitmq-server and cassandra with default settings. Next there must be haproxy enabling options for HA. These are separate saltstack formulas, so manually configure haproxy.conf

Create Pillar for roles

Now you have to create pillar with definition for all roles located at OpenContrail controller. As you can see, some definitions are repeated, because roles are independent and can run on different servers and scale up.

opencontrail:
  common:
    identity:
      engine: keystone
      host: 10.0.106.30
      port: 35357
      token: pwd
      password: pwd
    network:
      engine: neutron
      host: 10.0.106.34
      port: 9696
  config:
    enabled: true
    id: 1
    network:
      engine: neutron
      host: 10.0.106.34
      port: 9696
    bind:
      address: 10.0.106.34
    database:
      members:
      - host: 10.0.106.34
        port: 9160
    cache:
      host: 10.0.106.34
    identity:
      engine: keystone
      region: RegionOne
      host: 10.0.106.30
      port: 35357
      user: admin
      password: pwd
      token: pwd
      tenant: admin
    members:
    - host: 10.0.106.34
      id: 1
  control:
    enabled: true
    bind:
      address: 10.0.106.34
    master:
      host: 10.0.106.34
    members:
    - host: 10.0.106.34
      id: 1
  collector:
    enabled: true
    bind:
      address: 10.0.106.34
    master:
      host: 10.0.106.34
    database:
      members:
      - host: 10.0.106.34
        port: 9160
  database:
    enabled: true
    name: 'Contrail'
    original_token: 0
    data_dirs:
    - /home/cassandra
    bind:
      host: 10.0.106.34
      port: 9042
      rpc_port: 9160
    members:
    - 10.0.106.34
  web:
    enabled: True
    bind:
      address: 10.0.106.34
    master:
      host: 10.0.106.34
    cache:
      engine: redis
      host: 10.0.106.34
      port: 6379
    members:
    - host: 10.0.106.34
      id: 1
    identity:
      engine: keystone
      host: 10.0.106.30
      port: 35357
      user: admin
      password: pwd
      token: pwd
      tenant: admin

OpenStack Controller

opencontrail:
  common:
    identity:
      engine: keystone
      host: 10.0.106.30
      port: 35357
      token: pwd
      password: pwd
    network:
      engine: neutron
      host: 10.0.106.34
      port: 9696

OpenStack Compute

opencontrail:
  common:
    identity:
      engine: keystone
      host: 10.0.106.30
      port: 35357
      token: pwd
      password: pwd
    network:
      engine: neutron
      host: 10.0.106.34
      port: 9697
  compute:
    enabled: True
    hostname: compute1
    discovery:
      host: 10.0.106.34
    interface:
      dev: bond0 
      default_pmac: a0:36:9f:02:95:2c
      address: 10.0.106.102
      gateway: 10.0.106.1
      mask: 24
      dns: 8.8.8.8
      mtu: 9000
    control_instances: 1

Deploy OpenContrail

Run this command at all infrastructure nodes:

salt-call state.sls opencontrail

Deploy Compute nodes 

Add virtual router

python /etc/contrail/provision_vrouter.py --host_name compute1 --host_ip 10.0.106.106 
--api_server_ip 10.0.106.34 --oper add --admin_user admin --admin_password pwd --admin_tenant_name admin

Add control BGP

Finally you have to add control BGP through python script stored in /etc/contrail at OpenContrail Controller:

python /etc/contrail/provision_control.py --api_server_ip 10.0.106.34 --api_server_port 
8082 --host_name network1.contrail.domain.com --host_ip 10.0.106.34 --router_asn 64512

Now you should have OpenContrail deployed. Run your instance first instance and try the amazing network performance. For whole automation contact us for Private Cloud

Jakub Pavlík - @JakubPav
TCP Cloud platform engineer