Automated deployment of OpenContrail with SaltStack

Jakub Pavlik - July 21, 2014 -

This week, tcp cloud® launched a new version of TCP Virtual Private Cloud built on OpenContrail as SDN / NFV Controller. We decided for this network setupneutron plugin after spending more than 6 month of development and 2 month of production environment running vanilla OpenStack implementation with High Availability Neutron L3 Agent. We have worked at installation, development, testing and measuring of OpenContrail networking since OpenStack Summit at Atlanta, where we saw it for the first time. Our main contribution to this project was development of automated deployment for complete OpenStack infrastrucute with OpenContrail as neutron plugin. 

OpenContrail is released under an Apache 2.0 License by Juniper. To understand how it works you should read the slides and overview at opencontrail.org, as well as the documentation available at juniper.net or the book Day One Understanding OpenContrail.

This will be a mostly technical, but I will not talk about what OpenContrail actually does or how to deploy whole OpenStack or how powerful this solution really is. I am going to explain how to use our salt-opencontrail-formula for integration with an arbitrary OpenStack. 

I have to say that it was a little bit challenge for us, because it is not common vendor plugin that we are used to work with and it is not actually part of the official OpenStack community repositories? (maybe in Juno release Contrail will be included as official vendor plugin). As John Deatherage wrote at his blog, there are several ways how you can deploy Contrail, but if you want to deploy it as production system you should not use any of these. The reason is that the default deployment is done by Fabric and installs whole Openstack environment along the Contrail services. You do not have possibility to customise the OpenStack part and you do not know how it is actually configured. There is no official documentation for manual deployment except GitHub. OpenStack is modular system with various architectures and components. Therefore you should have possibility todefine properties. This is the reason why we refactored the installation code from Fabric and created a new SaltStack formula.

Prepare Environment

We use RedHat based OpenStack Havana running on CentOS 6.4 and OpenContrail 1.05. Download rpm package from juniper website and create local YUM repository for all nodes in infrastructure. Do not use RDO OpenStack Havana, because OpenContrail contains modified nova and neutron packages. Basically you can prepare these control nodes with CentOS 6.4:

  • OpenStack Controller – Keystone, Glance, Cinder, Nova, Dashboard, Swift, Heat, Ceilometer, etc.
  • OpenContrail Controller – Control node, Config node, Collector node, Neutron.
  • OpenStack Compute node – Nova compute, OpenContrail compute (vRouter).

Default Fabric automation runs setup_all with following set of tasks:

  • Executing task ‘setup_all’
  • Executing task ‘bash_autocomplete_systemd’
  • Executing task ‘setup_database’
  • Executing task ‘setup_openstack’
  • Executing task ‘setup_cfgm’
  • Executing task ‘setup_control’
  • Executing task ‘setup_collector’
  • Executing task ‘setup_webui’
  • Executing task ‘setup_vrouter’
  • Executing task ‘prov_control_bgp’
  • Executing task ‘prov_external_bgp’
  • Executing task ‘compute_reboot’

Against them we have created these sls roles, which can be use independently and even redundantly for high availability (e.g. 2 control nodes):

  • common.sls – common configuration for all roles.
  • database.sls – configuration of the OpenContrail database.
  • config.sls – install and configure config node.
  • control.sls – install and configure control node.
  • collector.sls – install and configure collector node.
  • compute.sls – install and configure compute node.
  • web.sls – install modules into OpenStack Dashboard (Horizon).

I cover just formula and installation for OpenContrail, but in tcp cloud we have created formulas for whole OpenStack environment including keystone, mysql, glance, cinder, nova, heat with modular neutron, cinder and nova plugins. These formulas might be released in the future. They are included inside of product TCP Private Cloud. For this purpose I describe manual changes inside of OpenStack configuration files that are different from http://docs.openstack.org

Deploy OpenStack

You should install OpenStack environment (Contrail repository only) and customized these specific projects:

NOVA

set libvirt vif driver in /etc/nova.conf at Controller and Compute node.

libvirt_vif_driver = nova_contrail_vif.contrailvif.VRouterVIFDriver

NEUTRON

Install openstack-neutron-server and openstack-neutron-contrail at Controller node. There must be configured: 

neutron.conf

core_plugin = neutron.plugins.juniper.contrail.contrailplugin.ContrailPlugin

/etc/neutron/plugins/opencontrail/ContrailPlugin.ini

[APISERVER]
api_server_ip = 10.0.106.34
api_server_port = 8082
multi_tenancy = False
[KEYSTONE]
admin_user
=admin
admin_password
=pswd
admin_tenant_name
=admin

HORIZON

Install contrail-openstack-dashboard and verify these modules /etc/openstack-dashboard/local_settings 

HORIZON_CONFIG['customization_module'] = 'contrail_openstack_dashboard.overrides'

Prepare OpenContrail controller

Because of standalone solution, you have to install basic rabbitmq-server and cassandra with default settings. Next there must be haproxy enabling options for HA. These are separate saltstack formulas, so manually configure haproxy.conf

Create Pillar for roles

Now you have to create pillar with definition for all roles located at OpenContrail controller. As you can see, some definitions are repeated, because roles are independent and can run on different servers and scale up.

opencontrail:
  common:
    identity:
      engine: keystone
      host: 10.0.106.30
      port: 35357
      token: pwd
      password: pwd
    network:
      engine: neutron
      host: 10.0.106.34
      port: 9696
  config:
    enabled: true
    id: 1
    network:
      engine: neutron
      host: 10.0.106.34
      port: 9696
    bind:
      address: 10.0.106.34
    database:
      members:
      - host: 10.0.106.34
        port: 9160
    cache:
      host: 10.0.106.34
    identity:
      engine: keystone
      region: RegionOne
      host: 10.0.106.30
      port: 35357
      user: admin
      password: pwd
      token: pwd
      tenant: admin
    members:
    - host: 10.0.106.34
      id: 1
  control:
    enabled: true
    bind:
      address: 10.0.106.34
    master:
      host: 10.0.106.34
    members:
    - host: 10.0.106.34
      id: 1
  collector:
    enabled: true
    bind:
      address: 10.0.106.34
    master:
      host: 10.0.106.34
    database:
      members:
      - host: 10.0.106.34
        port: 9160
  database:
    enabled: true
    name: 'Contrail'
    original_token: 0
    data_dirs:
    - /home/cassandra
    bind:
      host: 10.0.106.34
      port: 9042
      rpc_port: 9160
    members:
    - 10.0.106.34
  web:
    enabled: True
    bind:
      address: 10.0.106.34
    master:
      host: 10.0.106.34
    cache:
      engine: redis
      host: 10.0.106.34
      port: 6379
    members:
    - host: 10.0.106.34
      id: 1
    identity:
      engine: keystone
      host: 10.0.106.30
      port: 35357
      user: admin
      password: pwd
      token: pwd
      tenant: admin

OpenStack Controller

opencontrail:
  common:
    identity:
      engine: keystone
      host: 10.0.106.30
      port: 35357
      token: pwd
      password: pwd
    network:
      engine: neutron
      host: 10.0.106.34
      port: 9696

OpenStack Compute

opencontrail:
  common:
    identity:
      engine: keystone
      host: 10.0.106.30
      port: 35357
      token: pwd
      password: pwd
    network:
      engine: neutron
      host: 10.0.106.34
      port: 9697
  compute:
    enabled: True
    hostname: compute1
    discovery:
      host: 10.0.106.34
    interface:
      dev: bond0
      default_pmac: a0:36:9f:02:95:2c
      address: 10.0.106.102
      gateway: 10.0.106.1
      mask: 24
      dns: 8.8.8.8
      mtu: 9000
    control_instances: 1

Deploy OpenContrail

Run this command at all infrastructure nodes:

salt-call state.sls opencontrail

Deploy Compute nodes 

Add virtual router

python /etc/contrail/provision_vrouter.py --host_name compute1 --host_ip 10.0.106.106 
--api_server_ip 10.0.106.34 --oper add --admin_user admin --admin_password pwd --admin_tenant_name admin

Add control BGP

Finally you have to add control BGP through python script stored in /etc/contrail at OpenContrail Controller:

python /etc/contrail/provision_control.py --api_server_ip 10.0.106.34 --api_server_port 
8082 --host_name network1.contrail.domain.com --host_ip 10.0.106.34 --router_asn 64512

Now you should have OpenContrail deployed. Run your instance first instance and try the amazing network performance. For whole automation contact us for Private Cloud

Jakub Pavlík – @JakubPav
TCP Cloud platform engineer

banner-img
From Virtualization to Containerization
Learn how to move from monolithic to microservices in this free eBook
Download Now
Radio Cloud Native – Week of May 11th, 2022

Every Wednesday, Nick Chase and Eric Gregory from Mirantis go over the week’s cloud native and industry news. This week they discussed: Docker Extensions Artificial Intelligence shows signs that it's reaching the common person Google Cloud TPU VMs reach general availability Google buys MobileX, folds into Google Cloud NIST changes Palantir is back, and it's got a Blanket Purchase Agreement at the Department of Health and Human …

Radio Cloud Native – Week of May 11th, 2022
Where do Ubuntu 20.04, OpenSearch, Tungsten Fabric, and more all come together? In the latest Mirantis Container Cloud releases!

In the last several weeks we have released two updates to Mirantis Container Cloud - versions 2.16 and 2.17, which bring a number of important changes and enhancements. These are focused on both keeping key components up to date to provide the latest functionality and security fixes, and also delivering new functionalities for our customers to take advantage of in …

Where do Ubuntu 20.04, OpenSearch, Tungsten Fabric, and more all come together? In the latest Mirantis Container Cloud releases!
Monitoring Kubernetes costs using Kubecost and Mirantis Kubernetes Engine [Transcript]

Cloud environments & Kubernetes are becoming more and more expensive to operate and manage. In this demo-rich workshop, Mirantis and Kubecost demonstrate how to deploy Kubecost as a Helm chart on top of Mirantis Kubernetes Engine. Lens users will be able to visualize their Kubernetes spend directly in the Lens desktop application, allowing users to view spend and costs efficiently …

Monitoring Kubernetes costs using Kubecost and Mirantis Kubernetes Engine [Transcript]
LIVE WEBINAR
Getting started with Kubernetes part 2: Creating K8s objects with YAML

Thursday, December 30, 2021 at 10:00 AM PST
SAVE SEAT
LIVE WEBINAR
Manage your cloud-native container environment with Mirantis Container Cloud

Wednesday, January 5 at 10:00 am PST
SAVE SEAT
LIVE WEBINAR
Istio in the Enterprise: Security & Scale Out Challenges for Microservices in k8s

Presented with Tetrate
SAVE SEAT