Loading…
#MesosCon 2015 has ended

Sign up or log in to bookmark your favorites and sync them to your phone or calendar.

Experience: Any [clear filter]
Thursday, August 20
 

2:00pm

Twitter’s Production Scale: Mesos and Aurora Operations - Joe Smith, Twitter
Twitter has used Aurora and Mesos to scale from only tens of nodes to tens of thousands. During the process of scaling up, the configuration, deployment, and operational procedures for this system have evolved and improved significantly. This talk will offer an operations perspective on the management of a Mesos+Aurora cluster, and cover many of the cluster management best practices that have evolved at Twitter from real-world production experience. It will explore methods that decrease operational overhead, as well as examples of outages and incidents to illustrate various failure domains. Furthermore, the talk will highlight current and future safeguards in Aurora and Mesos to mitigate impact from these failures in the future.

Speakers
avatar for Joe Smith

Joe Smith

Site Reliability Engineer, Slack
Joe Smith is a Site Reliability Engineer at Slack, working on the Identity, Compliance, & Security team. Previously he was at Twitter, and built the Aurora and Mesos cluster from tens of nodes to tens of thousands. As Mesos and Aurora SRE, he automated the build, deployment, management... Read More →


Thursday August 20, 2015 2:00pm - 2:40pm
Grand Ballroom B

4:00pm

Developing Frameworks for Apache Mesos - Joe Stein, Big Data Open Source Security
Using schedulers like Marathon and Aurora help to get your applications scheduled and executing on Mesos. In many cases it makes sense to build a framework and integrate directly. This talk will breakdown what is involved in building a framework, how to-do this with examples and why you would want to-do this. Frameworks are not only for generally available software applications (like Kafka, HDFS, Spark ,etc) but can also be used for custom internal R&D built software applications too.

Speakers
JS

Joe Stein

Big Data Open Source Security LLC, Big Data Open Source Security LLC
Joe Stein is an Apache Kafka committer and PMC member. A frequent speaker also on both Hadoop and Cassandra, Joe is the Founder and Principal Architect of Big Data Open Source Security LLC http://stealth.ly a professional services and product solutions company. Joe has been a distributed... Read More →


Thursday August 20, 2015 4:00pm - 4:40pm
Grand Ballroom D

4:50pm

Building A Machine Learning Platform to Predict User Behavior on Mesos - Jeremy Stanley, Sailthru
In 2014 Sailthru launched a new machine learning platform, named Sightlines, using Mesos. This platform predicts the behavior of hundreds of millions of users for hundreds of clients. In this talk, Jeremy Stanley and Alex Gaudio will tell the story of how (and why) this platform was built on Mesos, and outline some of the key challenges and detours taken along the way. This will include an overview of the goals and architecture of the system, and an account of the 10 month effort to develop, scale and stabilize the system for production.

Topics covered will include our use of AWS spot instances and Netflix’s Asgard to save costs, Marathon and Zookeeper to schedule jobs, detours with Chronos, Spark and Redis as our architecture evolved, and challenges we faced with maximizing resource utilization and ensuring we had identical and isolated test and production environments.

Speakers
JS

Jeremy Stanley

"Jeremy Stanley is the Chief Data Scientist & EVP of Engineering at Sailthru, where he is focused on building data-driven solutions for marketers that drive long-term customer engagement and optimize revenue opportunities.Prior to Sailthru, Jeremy was the CTO at Collective where he... Read More →


Thursday August 20, 2015 4:50pm - 5:30pm
Grand Ballroom B
 
Friday, August 21
 

10:40am

Myriad: Resource Sharing Beyond Boundaries - Mohit Soni, PayPal
Last year at MesosCon, eBay presented a novel solution which allows Mesos and Apache YARN/Hadoop to co-exist without statically partitioning the data center. That solution evolved into a community project called "Myriad" and is under active development. In this talk, we will present the latest architecture of "Myriad", how it improves overall resource utilization in the datacenter, and how it enables dynamic allocation of resources between workloads running on Mesos and YARN.

Speakers
SM

Santosh Marella

Santosh Marella is a Software Engineer at MapR Technologies. Since joining MapR, Santosh worked on numerous projects starting from MapR's admin infrastructure, MapReduce and YARN. He currently leads MapR's efforts on Myriad. Prior to MapR, Santosh was a Software Engineer at Google... Read More →
MS

Mohit Soni

Software Engineer, PayPal
Mohit works as an engineer in platform team at eBay Inc. His job is to focus on maximizing efficiency, increasing agility, and reducing cost. Mohit presented in past at DockerCon 2014, Hadoop India Summit 2011 and BarCamp 2010. You can follow Mohit as mohitsoni on GitHub and Twit... Read More →


Friday August 21, 2015 10:40am - 11:20am
Grand Ballroom C

11:30am

Generalizing Software Deployment - The Many Meanings of "Update" - Bill Farner, Twitter
Shipping code is an everyday part of the life of a developer. And when shipping code in a Mesos cluster, that workflow is managed by individual schedulers. In the case of Apache Aurora, developers are empowered by a streamlined deployment mechanism. This talk will cover the evolution of how Aurora manages deployments, and describe some of the challenges imposed by the wide variance in requirements. Finally, there will be discussion of how deployments on Aurora currently work for major services at Twitter.

Speakers
BF

Bill Farner

Bill Farner has been a Software Engineer at Twitter for the last five years, serving as the Technical Lead on Aurora since its inception four years ago. He previously worked at Google on Production Engineering and storage infrastructure. Bill holds a MS in Computer Engineering from... Read More →


Friday August 21, 2015 11:30am - 12:10pm
Grand Ballroom C

11:30am

Migrating over 1,000 Production Components to a Mesos-Based Platform-as-a-Service (PaaS) - Tom Petr, HubSpot
At HubSpot we built a turn-key PaaS system our engineers use to deploy all the different parts that make up the HubSpot application. In this talk we'll discuss the benefits of using Mesos as a starting point for deploying a PaaS, the open source framework we created called Singularity, and the lessons learned from moving our entire production application to this stack. We will enumerate many of the specific benefits that became apparent after the system was in place, including enhanced security, reliability, operational simplicity, and cost efficiency. Lastly, we will shed light on both the ways in which we extended what Mesos offered in order to empower our product developers, and how we moved over 100 software load balancers into Mesos to make our infrastructure team more efficient.

Speakers
avatar for Tom Petr

Tom Petr

Engineering Lead, HubSpot
Tom Petr is an Engineering Lead at HubSpot. Prior to working on Vitess and Kubernetes, he was a maintainer of Singularity, an open source Mesos framework. Tom has spoken at multiple conferences about HubSpot's platform infrastructure.


Friday August 21, 2015 11:30am - 12:10pm
Grand Ballroom B

11:30am

Enabling Hybrid Workloads with Docker/Mesos at Orbitz - Steve Hoffman, Orbitz
The promise of Mesos to create a datacenter-centric resource pool is a great idea. However, a decade old company with many different types of existing systems and unique workloads can't simply mass migrate to Mesos. That doesn't mean Mesos can't pay a huge part in unifying disparate workloads that construct existing systems while opening the door to new Mesos-centric development.

In this presentation, Steve will cover 3 very different use cases that are being unified under Apache Mesos including: Automated builds of existing code artifacts, automated builds of AWS AMIs via packer in EC2, and rolling deploys & supervision of Docker microservices across environments. The addition of Docker on Mesos has enabled this, but that doesn't mean that all Mesos slaves should be identical. See how the addition of localized support services on slaves can compliment your Docker on Mesos use cases.

Speakers
avatar for Steve Hoffman

Steve Hoffman

Architect, Journera, Inc.
Tech guy at Journera. Minutia @ http://bit.ly/bacoboy


Friday August 21, 2015 11:30am - 12:10pm
Grand Ballroom D

1:30pm

Satellite: A Monitoring Application for Mesos - Sunil Abraham, Two Sigma Investments
Satellite is a monitoring and alerting solution for Mesos. Satellite’s powerful event streams processing engine allows /ops teams to build custom monitoring and alerting solutions that will improve uptime and reliability of the underlying compute resources in the cluster. In this talk, you'll learn how Two Sigma is using Satellite to monitor and take action on the hosts in our Mesos cluster. After a quick introduction to the architecture of Satellite, we'll walk through the process of defining a rule, adding a Pagerduty alert, and programmatically taking action on the state of the cluster by enabling and disabling hosts in the cluster.

Speakers
SA

Sunil Abraham

Sunil Abraham is a developer at Two Sigma Investments. Previously, Sunil was a Fulbright Scholar at the University of Waterloo working on public health and studied math and statistics at Oxford and Cambridge.


Friday August 21, 2015 1:30pm - 2:10pm
Willow

2:20pm

Simplifying Maintenance with Mesos - Benjamin Mahler, Twitter
You have computing resources that your developers want to leverage. However, providing developers with direct access to machines would set you up for an operational nightmare: a critical security vulnerability comes along and now you need to reboot all of your kernels to apply the fix, without any downtime.. good luck!

Mesos is a “cluster management” layer that provides developers with access to computing resources while encouraging behavior that leads to simplified maintenance. This talk will address the challenges of maintenance when dealing with Mesos clusters running multiple frameworks (e.g. services, storage, batch compute). We’ll explore a current proposal for adding simple and flexible maintenance primitives in Mesos to address these concerns and enable tooling for automated maintenance.

Speakers
avatar for Benjamin Mahler

Benjamin Mahler

Software Engineer, Mesosphere
Benjamin Mahler is a committer and PMC member of Apache Mesos and has been working on Mesos since 2012. Benjamin now works at Mesosphere as a technical lead and has given Mesos related talks at several conferences and companies. His interests include distributed systems, fault tolerance... Read More →



Friday August 21, 2015 2:20pm - 3:00pm
Grand Ballroom B