Austin Summit Preview: Architecture and Best Practices to Deploy Hadoop and Spark Clusters with Sahara

Nikita Konovalov - March 22, 2016 - , , , ,

As the Austin OpenStack Summit approaches, we’ll be bringing you previews of some of the sessions we’re participating in. First up: our session on deploying big data solutions with Hadoop and OpenStack.

Hadoop is the world standard for big data platforms, built with performance and data consistency in mind. While the Hadoop ecosystem today is used by major enterprises all over the world, that doesn’t mean it’s easy to deploy. Hadoop has always been considered to be a bare metal solution, and building huge clusters requires a lot of patience and expertise from DevOps teams.

Deploying Hadoop to an OpenStack private cloud instead of bare metal raises questions and concerns, including:

  • Can Hadoop keep its performance on VMs?
  • How reliable is virtualized storage for HDFS data?
  • Will the cloud reduce deployment complexity or it is going to bring its own caveats?

The simple answer to all of these questions is yes, you can. To get the details, join us for Architecture and Best Practices to Deploy Hadoop and Spark Clusters with Sahara at the OpenStack Summit in Austin on Wednesday, April 27.  I’ll be speaking, along with Sergey Lukjanov, the Sahara PTL, and Paul Work, who manages the Cluster Infrastructure Lab in Intel’s Cloud Products Group. We’ll share what we’ve learned from performance tests in the Intel high performance lab with Mirantis OpenStack installed there. We’ll compare bare metal Hadoop to the virtualized cluster installed by the OpenStack Sahara service and share Hadoop settings and configurations to best utilize CPU and I/O resources, as well as best practices for selection of storage settings, and more. We hope to see you there!

Add this session to your summit calendar: Architecture and Best Practices to Deploy Hadoop and Spark Clusters with Sahara

If you haven’t registered for the OpenStack Summit yet, register here.

banner-img
From Virtualization to Containerization
Learn how to move from monolithic to microservices in this free eBook
Download Now
Radio Cloud Native – Week of May 11th, 2022

Every Wednesday, Nick Chase and Eric Gregory from Mirantis go over the week’s cloud native and industry news. This week they discussed: Docker Extensions Artificial Intelligence shows signs that it's reaching the common person Google Cloud TPU VMs reach general availability Google buys MobileX, folds into Google Cloud NIST changes Palantir is back, and it's got a Blanket Purchase Agreement at the Department of Health and Human …

Radio Cloud Native – Week of May 11th, 2022
Where do Ubuntu 20.04, OpenSearch, Tungsten Fabric, and more all come together? In the latest Mirantis Container Cloud releases!

In the last several weeks we have released two updates to Mirantis Container Cloud - versions 2.16 and 2.17, which bring a number of important changes and enhancements. These are focused on both keeping key components up to date to provide the latest functionality and security fixes, and also delivering new functionalities for our customers to take advantage of in …

Where do Ubuntu 20.04, OpenSearch, Tungsten Fabric, and more all come together? In the latest Mirantis Container Cloud releases!
Monitoring Kubernetes costs using Kubecost and Mirantis Kubernetes Engine [Transcript]

Cloud environments & Kubernetes are becoming more and more expensive to operate and manage. In this demo-rich workshop, Mirantis and Kubecost demonstrate how to deploy Kubecost as a Helm chart on top of Mirantis Kubernetes Engine. Lens users will be able to visualize their Kubernetes spend directly in the Lens desktop application, allowing users to view spend and costs efficiently …

Monitoring Kubernetes costs using Kubecost and Mirantis Kubernetes Engine [Transcript]
Technical training
Learn Kubernetes & OpenStack from Deployment Experts
Prep for certification!
View schedule
WHITEPAPER
The Definitive Guide to Container Platforms
READ IT NOW
LIVE WEBINAR
Istio in the Enterprise: Security & Scale Out Challenges for Microservices in k8s

Presented with Tetrate
SAVE SEAT