This event has ended. View the official site or create your own event → Check it out
This event has ended. Create your own
View analytic
Thursday, August 20 • 11:00am - 11:40am
Fully Fault tolerant Streaming workflows at Scale using Apache Mesos & Spark Streaming - AkhilDas, Sigmoid

Sign up or log in to save this to your schedule and see who's attending!

Reliability, maintainability and scalability are the key concerns while designing any data intensive application . With the advent of realtime streaming platforms like Apache Spark, Storm etc, it is important that these computation frameworks adhere to all of them. One has to face a lot of challenges while implementing a realtime spark streaming pipeline in production. In this presentation we will showcase how to overcome these challenges and develop a fully, scalable fault tolerant streaming system with the help of Apache Mesos which not only makes it easier to deploy and manage the resources but also helps in handling varying data loads through dynamic resource management and allocation .



AkhilDas is a Software Developer at Sigmoid with focus on distributed computing, big data analytics, scaling and optimising performance. Sigmoid has worked with over 25 customers in the Big data space to get them real time insights on Tbs of data using Apache Spark and Spark Streaming. | | Previously, he had a talk on Internals of Spark and Spark Streaming at Real time data processing and cloud computing Meetup which happened in Bangalore.

Thursday August 20, 2015 11:00am - 11:40am
Grand Ballroom B