Loading…
#MesosCon 2015 has ended

Sign up or log in to bookmark your favorites and sync them to your phone or calendar.

Framework [clear filter]
Thursday, August 20
 

11:00am

Mesos at Bloomberg - Skand Gupta, Bloomberg
BVault is a massive scalable, archiving and e-discovery solution for communications and has been adopted by more than 800 enterprises, processing more than 220 million daily messages and archived more than 90 billion communication objects. The service is optimized for fast deployment of data-centric and processing-intensive applications using elastic cloud computing strategies namely, Container-based resource management (Mesos) to abstract heterogeneous data center assets as a homogeneous set of resources, prefabricated hardware in secure and geographically distributed data centers to provide on-demand capacity management, and Continuous development and integration using containers as an emerging standard in cloud infrastructure.

We share the lessons learned along the way, and tools we needed to build that enable us to validate, monitor, and operate the platform in production.

Speakers
SG

Skand Gupta

Skand Gupta is technical lead and engineering manager at Bloomberg LP. He has experience building large scale distributed systems in shared infrastructure and search space. He currently leads the Mesos infrastructure team for Bloomberg Vault.


Thursday August 20, 2015 11:00am - 11:40am
Grand Ballroom C

2:00pm

Heterogeneous Resource Scheduling Using Apache Mesos for Cloud Native Frameworks - Sharma Podila, Netflix
Netflix customers stream more than two billion hours of content each month, accounting for over a third of downstream Internet traffic. At this scale, real time insights are critical to a smooth streaming experience. Sharma Podila will present an in-depth look at the technology - including the Fenzo task scheduler for Apache Mesos and Mantis, a cloud native event stream processing platform - that Netflix uses to gain real time operational insights into system behavior. The talk will focus on the design and usability of Fenzo and will provide details on how it uses CPU, memory, and network bandwidth resources for job scheduling. The presentation will showcase bin packing, task locality, resource affinity, task constraints, and autoscaling of the slave cluster. To close Sharma will talk about future directions and opportunities to collaborate with the Mesos community.

Speakers
avatar for Sharma Podila

Sharma Podila

Senior Software Engineer, Netflix
Sharma Podila is a Senior Software Enginner in the Edge Engineering team at Netflix, Inc. His current work includes Fenzo, an open source generic scheduler with plugin based optimizations, developing cloud native Mesos frameworks, and evolving the microservices platform at Netfli... Read More →


Thursday August 20, 2015 2:00pm - 2:40pm
Willow

2:00pm

Resource Optimization with Charmander - Marcel Neuhausler, AT&T Foundry
Charmander is a closed loop data analysis and optimization environment for a container based infrastructure. Charmander is open source (https://github.com/att-innovate/charmander) and is based on Mesos, Docker, cAdvisor, InfluxDB, Apache Spark, and Redis. In this presentation, Marcel Neuhausler will showcase different use-cases for Charmander, provide a technical overview of the components, and deep dive into its own Mesos Framework. 

Speakers
MN

Marcel Neuhausler

Marcel Neuhausler is a Principal Software Engineer at the AT&T Foundry in Palo Alto focusing on resource-conscious distributed systems.


Thursday August 20, 2015 2:00pm - 2:40pm
Grand Ballroom C

2:50pm

Crossing the Streams: Leveraging Kubernetes Services From Other Mesos Frameworks and Vice Versa - Timothy St. Clair, Red Hat
Kubernetes provides a declarative model in order to describe micro-services. However, auto-discovering those services outside of kubernetes requires add-ons. In this presentation we will explore how using the kubernetes framework and Mesos-DNS we can bridge the gap, and provide cluster wide service discovery.

Speakers
avatar for Timothy St. Clair

Timothy St. Clair

Principal Engineer, Red Hat
Timothy St. Clair is a Principal Software Engineer at Red Hat in the Emerging Technology office. He is a PMC member and committer to the Apache Mesos Project, and has worked on the development and integration of various open source distributed systems projects, including: Mesos, Kubernetes... Read More →


Thursday August 20, 2015 2:50pm - 3:30pm
Grand Ballroom C

4:00pm

Apache Cotton: MySQL on Mesos - Yan Xu, Twitter
Apache Cotton (previously named Mysos) is an Apache Mesos framework for running MySQL instances. It dramatically simplifies the management of a MySQL cluster and is designed to offer:
- Efficient hardware utilization through multi-tenancy (in performance-isolated containers)- High reliability through preserving the MySQL state during failure and automatic backing up to/restoring from HDFS- An automated self-service option for bringing up new MySQL clusters- High availability through automatic MySQL master failover- An elastic solution that allows users to easily scale up and down a MySQL cluster by changing the number of slave instances
We’ll share our experience developing and using this framework.

Speakers
avatar for Yan Xu

Yan Xu

Software Engineer, Apple


Thursday August 20, 2015 4:00pm - 4:40pm
Grand Ballroom C

4:50pm

Distributed CI: Scaling Jenkins on Mesos and Marathon - Roger Ignazio, Puppet Labs
As engineering teams and product offerings expand, monolithic Jenkins instances grow and become single points of failure. By running Jenkins on Mesos and Marathon, masters can be scaled horizontally and executors provisioned dynamically as demand increases. This presentation will demonstrate that running Jenkins at-scale is more feasible than ever. The architecture provided by Mesos moves Continuous Integration (CI) from being synonymous with Jenkins to Jenkins being a component of the overall CI system.

This presentation will cover:
- Conventional scaling techniques and their shortcomings
- Scaling Jenkins masters and slaves horizontally on Mesos and Marathon
- Dynamically creating and executing jobs
- Using Elasticsearch, Logstash, and Kibana (ELK) to store, query, and visualize run data

Thursday August 20, 2015 4:50pm - 5:30pm
Grand Ballroom C
 
Friday, August 21
 

10:40am

Myriad: Resource Sharing Beyond Boundaries - Mohit Soni, PayPal
Last year at MesosCon, eBay presented a novel solution which allows Mesos and Apache YARN/Hadoop to co-exist without statically partitioning the data center. That solution evolved into a community project called "Myriad" and is under active development. In this talk, we will present the latest architecture of "Myriad", how it improves overall resource utilization in the datacenter, and how it enables dynamic allocation of resources between workloads running on Mesos and YARN.

Speakers
SM

Santosh Marella

Santosh Marella is a Software Engineer at MapR Technologies. Since joining MapR, Santosh worked on numerous projects starting from MapR's admin infrastructure, MapReduce and YARN. He currently leads MapR's efforts on Myriad. Prior to MapR, Santosh was a Software Engineer at Google... Read More →
MS

Mohit Soni

Software Engineer, PayPal
Mohit works as an engineer in platform team at eBay Inc. His job is to focus on maximizing efficiency, increasing agility, and reducing cost. Mohit presented in past at DockerCon 2014, Hadoop India Summit 2011 and BarCamp 2010. You can follow Mohit as mohitsoni on GitHub and Twit... Read More →


Friday August 21, 2015 10:40am - 11:20am
Grand Ballroom C

11:30am

Generalizing Software Deployment - The Many Meanings of "Update" - Bill Farner, Twitter
Shipping code is an everyday part of the life of a developer. And when shipping code in a Mesos cluster, that workflow is managed by individual schedulers. In the case of Apache Aurora, developers are empowered by a streamlined deployment mechanism. This talk will cover the evolution of how Aurora manages deployments, and describe some of the challenges imposed by the wide variance in requirements. Finally, there will be discussion of how deployments on Aurora currently work for major services at Twitter.

Speakers
BF

Bill Farner

Bill Farner has been a Software Engineer at Twitter for the last five years, serving as the Technical Lead on Aurora since its inception four years ago. He previously worked at Google on Production Engineering and storage infrastructure. Bill holds a MS in Computer Engineering from... Read More →


Friday August 21, 2015 11:30am - 12:10pm
Grand Ballroom C

1:30pm

Docker Swarm on Mesos - Tim Chen, Mesosphere & Victor Vieux, Docker
Docker has become a popular way for developers to package their applications for deployment, but as applications get more complex there is a growing need to deploy them at scale, orchestrating them across dozens or thousands of machines. With Docker Swarm, you can talk to Mesos using the Docker command line client and start Docker containers on your Mesos cluster as easily as you would do it on your development laptop. In this presentation, Victor Vieux (Docker Engineer, Docker) and Timothy Chen (Mesosphere Engineer, Mesosphere) will explain why Mesos is the “gold standard” for running Docker containers at scale and will demo how to setup Docker Swarm on Mesos (using DCOS) and how to do a `docker run` against your Mesos cluster easily.

Speakers
TC

Tim Chen

Tim Chen is on the Mesosphere core team and focuses on containerization and big data frameworks. He is also a PMC/committer on Apache Drill and Apache Mesos, and contributed to other open source projects such as Spark, Kafka, Hive. Before joining Mesosphere, Tim past experiences includes... Read More →
avatar for Victor Vieux

Victor Vieux

Docker Core Engineer, Docker, Inc
Victor Vieux works as a software engineer at Docker, Inc. He was one of the few guys working on the Docker internals, he's now working on Docker Swarm, a Docker-native clustering system. Originally from Paris, France, where he was famous for his work on robots (programming them in... Read More →


Friday August 21, 2015 1:30pm - 2:10pm
Grand Ballroom C

2:20pm

Scaling a Highly-Available Scheduler Using the Mesos Replicated Log: Pitfalls and Lessons Learned - Kevin Sweeney, Twitter
Mesos frameworks can go from prototype to production at scale very quickly. As your users realize how easy it is to spin up tasks, they will start doing so gratefully and rapidly. This talk will arm you with techniques for writing a framework scheduler for a large-scale Mesos cluster using Apache Aurora as a case study. Aurora routinely manages 100s of thousands of tasks per Mesos cluster. To do this it maintains the state of each task and quickly reacts to changes in the cluster. To talk to Mesos Aurora relies on the Mesos JNI library. For high-availability support it relies on the Mesos replicated log for durable storage. As its clusters grew, Aurora pushed against the limits of these libraries. This talk will explore the tools the Aurora scheduler has used to meet these challenges, including Apache Thrift for schema management, and the importance of normalization.

Speakers
avatar for Kevin Sweeney

Kevin Sweeney

Software Engineer - Cloud Infrastructure, Twitter
"Kevin Sweeney is a software engineer at Twitter and committer on Apache Aurora, working on it since 2012. You can find him on Twitter as @kts.



Friday August 21, 2015 2:20pm - 3:00pm
Grand Ballroom C