Sahara updates in the OpenStack Juno release

Nikita Konovalov - November 6, 2014 - , , , , , , , , ,

The OpenStack Juno release includes several new features for the Sahara project, including the following:

  • Two new provisioning plugins and the Cloudera Hadoop Distribution plugin are included.

  • Provisioning and EDP engines have been enhanced to support more cluster setups and job types.

  • Swift authorization offers better security.

  • The Sahara UI has been incorporated into the OpenStack Dashboard project.

  • Support for running in distributed mode is in alpha state.

  • Other updates are also available.

Plugin updates

The Apache Spark plugin enables Spark jobs to execute on top of Hadoop clusters. The plugin uses the same Hadoop distribution as the Cloudera plugin but with a different EDP engine. It supports Spark 0.9.1 and 1.0.0 installations.

The Vanilla plugin can now install Hadoop 2.4.1. Hadoop 2.3.0 is now deprecated; however, all existing clusters continue to operate normally.

The Cloudera Hadoop Distribution plugin allows Hadoop clusters setup using Cloudera Manager. Sahara installs Cloudera Manager 5, which then provisions Hadoop 2.3.0 to VMs.

The plugin can perform all kind of operations, including cluster scaling and running EDP jobs.

EDP enhancements

New EDP engines in Sahara allow it to execute new EDP job types, such as:

  • Oozie workflow engine – responsible for MapReduce, Pig, Hive, and Streaming jobs

  • Spark engine – executes Spark jobs on the clusters provisioned with a new Apache Spark plugin

Swift authorization improvement

The Swift authorization mechanism no longer requires storing credentials internally, which was a noted security concern.  Keystone trusts are now used for Swift authorization.

UI migration

The Sahara UI has been incorporated in the OpenStack Dashboard project. All panels are now available under the Project tab in the Data Processing section. This migration simplifies the Sahara deployment process, eliminating the need to customize the OpenStack Dashboard with Sahara’s panels.

Horizon has adopted Bootstrap 3 to improve the appearance of all the Sahara UI tables.Workflows and detail views have been updated.

Support for running in distributed mode

Sahara now has alpha support for running in distributed mode. Two process types can run on separate OpenStack nodes. The API process serves HTTP requests and builds the tasks for the engine. The engine process actually executes the tasks and reports back to the API. Distributed mode allows as many APIs and engines as needed per OpenStack cloud.

The all-in-one entry point has been renamed to a more descriptive sahara-all, which is still recommended.  

Other updates

Sahara now supports more flexible configurations depending on the cloud installation, such as:

  • Regions support – If OpenStack is running in a multi-region mode,  you can configure Sahara to operate inside a specific region.

  • Ceilometer integration – Sahara sends notifications each time the Cluster changes state.

  • Security Groups support – Sahara can now set up a security group for instances in a node group. You can use an existing security group or create a new one.

  • Anti-affinity – The internal implementation now uses the Server Groups API. The change should not be visible from the user’s perspective as the mechanism is used for new clusters. The clusters created with Icehouse Sahara will continue to operate normally.

See Sahara’s updated Juno features in action in this video.
Deploy Mirantis Secure Registry on any Kubernetes (Minikube, EKS, GKE, K0S, etc.)

Note: this blog post was originally published by Avinash Desireddy on Medium. You can view the original post here. Docker Containers, Kubernetes, CNCF, and many other relevant projects completely changed how we package, ship, and run applications. As you all know, Kubernetes has become a defacto standard for running applications. At the same time, container registries and chart repositories play a …

Deploy Mirantis Secure Registry on any Kubernetes (Minikube, EKS, GKE, K0S, etc.)
Software Supply Chain Security on Any Kubernetes with Mirantis Secure Registry 3.0

Security and cloud infrastructure availability concerns have been in the news of late with the recent Log4j vulnerabilities and outages at some of the world’s largest public cloud providers. The security and integrity of your container-based images has never been more important. Many have taken to Kubernetes to assist in the deployment and management of their container-based workloads, and are leveraging …

Software Supply Chain Security on Any Kubernetes with Mirantis Secure Registry 3.0
A Year in Review: A Look Back at the Most Powerful Mirantis Resources from 2021

2021 has been quite the year - and while there have been plenty of not-so-good times, we at Mirantis would like to take a moment to focus on the good. We are thankful for the opportunity to provide our readers with informative, accurate, and, above all, educational content via our company blog. We try not only to include helpful information …

A Year in Review: A Look Back at the Most Powerful Mirantis Resources from 2021
Service Mesh for Mere Mortals
A Guide to Istio and How to Use Service Mesh Platforms
Technical training
Learn Kubernetes & OpenStack from Deployment Experts
Prep for certification!
View schedule
Getting started with Kubernetes part 2: Creating K8s objects with YAML

Thursday, December 30, 2021 at 10:00 AM PST