#MesosCon 2015: Full Schedule

8:30am PDT

Pre-Conference Hackathon hosted by Moz

The hackathon is free but registration is required. Click here for more information and to register.

We've organized a pre-conference Apache Mesos hackathon, and we hope you'll join us!

What to Expect

The hackathon will begin at 8:30am with breakfast, and we'll begin hacking immediately at that time. For participants looking for projects to work on, we'll make space available for you to meet and share ideas. Lunch and snacks will be available throughout the day, and lunch will be provided. We'll also have a strong wifi connection, and up-to-date vagrant images that will help you get up and running with Mesos and many popular frameworks.

Hackathon Projects

What should you work on? It's really up to you, as long as it's Mesos-related. Here are a few broad categories of projects to consider.

Frameworks (build your own Mesos framework or add features to an existing framework)
Mesos Core (squashing bugs, adding new features, or even proposing a design document)
Ops (tools to make it easier to operate a Mesos cluster; examples may include monitoring, logging, etc)

Need a project or team to join? First register for the hackathon, then join the Mesos dev@mesos.apache.org mailing list (by emailing dev-subscribe@mesos.apache.org) and join in an email thread we'll kick off two weeks before the hackathon to brainstorm ideas and optionally form teams.

Hackathon Rules
The hackathon has a few basic rules:

Teams may be no larger than 5 participants
Coding may not begin until the hackathon officially begins, although we encourage you to come up with a plan ahead of time!
Source code from the hackathon must be fully available on github, bitbucket, or a public source repository in order to be successfully submitted

These rules are subject to be changed in the future. Common sense applies here: if you think you're doing something that's against the rules or that gives you an unfair advantage, you probably shouldn't do it.

THANK YOU to our hackathon sponsor and host

This year's hackathon is sponsored by SignalFx, and hosted by Moz.

Wednesday August 19, 2015 8:30am - 4:30pm PDT
Moz 1100 2nd Avenue, #500, Seattle

Hackathon

3:00pm PDT

Early Registration

Wednesday August 19, 2015 3:00pm - 5:00pm PDT
2nd Floor Foyer

6:00pm PDT

Pre-Conference Meetup hosted by Twitter

Details Below. Please RSVP here to attend.

#MesosCon is taking place in Seattle August 20-1st, and we're pleased to welcome the Apache Mesos community by hosting a pre-conference meetup and social at Twitter's Seattle office. The event will include ample opportunities to meet other members of the community, and we'll organize ourselves into groups for unconference-style discussions for part of the evening. Bring ideas about the future direction for the project and its related ecosystem of tools, uses of Mesos to discuss, or features you'd like to implement.

Event Agenda

6:00PM: Doors open; networking, pizza, and drinks
7:00PM: Form unconference discussion groups
7:10PM: Unconference sessions
7:45PM: Each unconference session reports back to the larger group
8:00: Networking
9:00PM: End of Event

Event Location and Check In

Twitter Seattle - 1501 4th Ave STE 1900 Seattle, WA 98121 Head in the 4th avenue entrance of the Century Square building and head up the escalators to find the check-in booth. Please bring valid ID.

*Note: No press will be admitted to this event; please e-mail press@twitter.com if you have any questions. Twitter reserves the right to refuse entry to this event. Twitter also reserves the right to remove individuals from Twitter's premises for disruptive or disorderly behavior, or for any other reason in Twitter's sole discretion. Please arrive before 6:30 to ensure admission to the building.

Wednesday August 19, 2015 6:00pm - 9:00pm PDT
Twitter Seattle 1501 4th Ave. STE 1900. Seattle

Meetup

7:30am PDT

Registration and Continental Breakfast

Thursday August 20, 2015 7:30am - 9:00am PDT
2nd Floor Foyer

Break Registration

9:00am PDT

Keynote: State of Mesos - Benjamin Hindman, Apache Mesos VP

Speakers

Ben Hindman

Mesosphere Founder - Apache Mesos Co-Creator, Mesosphere

Ben is one of the creators of Apache Mesos, a platform for building and running resource-efficient distributed systems at scale. Ben started working on Mesos as a PhD student at Berkeley before he brought it to Twitter where it runs on thousands of machines. An academic at heart... Read More →

Thursday August 20, 2015 9:00am - 9:35am PDT
Grand Ballroom

Keynote

9:35am PDT

Keynote: Enabling Microservices Frameworks to Solve Business Problems - Kenneth Owens, CTO of Cloud Infrastructure Services at Cisco

Developers are driving the market for cloud consumption and leading each industry into the new era of software defined disruption. There are no longer questions about elastic and flexible agile development as the way to innovate and reduce time to market for businesses. However, Physical and cloud Infrastructure does not enable application development platforms natively nor provide the ability to create applications that are cloud native with elastic services. In addition, businesses are moving to application development architectures leveraging microservices which are becoming more strategic to their business strategy. When making the decision to build and operate an application on physical or on a cloud platform, microservices became central to your application architecture and strategy.

Speakers

Kenneth Owens

Chief Technology Officer, Cloud Intercloud Services, Cisco

Ken Owens is Chief Technology Officer, Cloud Intercloud Services at Cisco Systems. Ken is responsible for creating and communicating technical/scientific vision and strategy for Cloud Intercloud Services (CIS) business. He brings a compelling view of technology trends in enterprise... Read More →

Thursday August 20, 2015 9:35am - 9:45am PDT
Grand Ballroom

Keynote

9:50am PDT

Keynote: Trading Simplicity for Performance When Designing Distributed Systems - Neha Narula

Predicting performance and debugging problems is critical to successfully running a distributed system in production, and a simple system is easier to reason about and understand. But for many systems, simplicity is fundamentally at odds with performance and scalability -- oftentimes, optimizing our systems to reduce latency and improve scalability requires more and more complex designs. Running complex systems at scale is even riskier because rare corner cases are guaranteed to occur.

An important example of this is scaling an application's data plane. Improving performance and fault tolerance in the data plane requires techniques like caching, replication, partitioning, and precomputation. These techniques greatly increase complexity by introducing problems like cache invalidation, rebalancing, managing staleness, and maintaining consistency.

In this talk, Neha will discuss this tension between simplicity and performance, and describe how to think about and consciously manage these trade-offs.

Speakers

Neha Narula

Director - Digital Currency Initiative, MIT

Neha Narula just completed a PhD at the Computer Science and Artificial Intelligance Lab at MIT building fast, scalable distributed systems and databases. In a previous life she helped launch a new version of Digg and was a Senior Software Engineer at Google, where she designed the... Read More →

Thursday August 20, 2015 9:50am - 10:30am PDT
Grand Ballroom

Keynote

10:30am PDT

Break

Thursday August 20, 2015 10:30am - 11:00am PDT
2nd Floor Foyer

Break

11:00am PDT

Challenges in Optimizing Job Scheduling On Mesos - Alex Gaudio, Sailthru

As a distributed systems kernel, Mesos solves complex scheduling problems. These problems include: Bin Packing of tasks given resource constraints; Dominant Resource Allocation to prioritize Framework offers; user-defined optimistic or pessimistic Scheduler algorithms. When job allocation stops working properly, we see Framework starvation, dead locks, underutilization and other issues. In this talk, Alex Gaudio will present a unique interpretation of task scheduling on Mesos. By drawing on his experiences building Mesos Frameworks and infrastructure for Sailthru's Data Science team, he will provide attendees with ways to fundamentally question how they use Mesos, and offer practical advice to improve sub-optimal usage patterns.

Speakers

Alex Gaudio

Senior Data Scientist and Engineer, Sailthru

Alex Gaudio is a Senior Data Scientist and Engineer at Sailthru, where he builds machine learning algorithms and distributed data pipelines on Mesos. He is the primary developer of Relay.Mesos, an intelligent auto-scaler for Mesos that functions like a thermostat. He also develops... Read More →

Thursday August 20, 2015 11:00am - 11:40am PDT
Willow

Experience: Advanced Internals / Extensions

11:00am PDT

Mesos at Bloomberg - Skand Gupta, Bloomberg

BVault is a massive scalable, archiving and e-discovery solution for communications and has been adopted by more than 800 enterprises, processing more than 220 million daily messages and archived more than 90 billion communication objects. The service is optimized for fast deployment of data-centric and processing-intensive applications using elastic cloud computing strategies namely, Container-based resource management (Mesos) to abstract heterogeneous data center assets as a homogeneous set of resources, prefabricated hardware in secure and geographically distributed data centers to provide on-demand capacity management, and Continuous development and integration using containers as an emerging standard in cloud infrastructure.

We share the lessons learned along the way, and tools we needed to build that enable us to validate, monitor, and operate the platform in production.

Speakers

Skand Gupta

Skand Gupta is technical lead and engineering manager at Bloomberg LP. He has experience building large scale distributed systems in shared infrastructure and search space. He currently leads the Mesos infrastructure team for Bloomberg Vault.

Thursday August 20, 2015 11:00am - 11:40am PDT
Grand Ballroom C

Experience: Intermediate Framework

11:00am PDT

Fully Fault tolerant Streaming workflows at Scale using Apache Mesos & Spark Streaming - AkhilDas, Sigmoid

Reliability, maintainability and scalability are the key concerns while designing any data intensive application . With the advent of realtime streaming platforms like Apache Spark, Storm etc, it is important that these computation frameworks adhere to all of them. One has to face a lot of challenges while implementing a realtime spark streaming pipeline in production. In this presentation we will showcase how to overcome these challenges and develop a fully, scalable fault tolerant streaming system with the help of Apache Mesos which not only makes it easier to deploy and manage the resources but also helps in handling varying data loads through dynamic resource management and allocation .

Speakers

AkhilDas

AkhilDas is a Software Developer at Sigmoid with focus on distributed computing, big data analytics, scaling and optimising performance. Sigmoid has worked with over 25 customers in the Big data space to get them real time insights on Tbs of data using Apache Spark and Spark Streaming.Previously... Read More →

Thursday August 20, 2015 11:00am - 11:40am PDT
Grand Ballroom B

Experience: Intermediate Ops

11:00am PDT

Mesos and the Architecture of the New Datacenter - Thomas Rampelberg, Mesosphere

Apache Mesos has the ability to run on every private and cloud instance, anywhere. In this talk, Thomas Rampelberg (Datacenter Developer at Mesosphere) will trace Mesos from its origins as a “fine-grained resource manager” at UC Berkeley to becoming a full-fledged distributed kernel for resource management across the datacenter and cloud. He will explain the momentum behind the “single computer” abstraction that has put Mesos at the center of one of the most exciting architecture shifts in recent information technology history. Rampelberg will explain the role of Mesos in unifying other frameworks, share examples of how Mesos is empowering the new class of “datacenter developers” to program directly against datacenter resources, and draw correlations to how the Linux kernel revolutionized the server industry.

Speakers

Thomas Rampelberg

Thomas, one of Mesosphere’s Datacenter Developer and creator of the new Mesos CLI contributes to the open source Apache Mesos UI and has helped build other interfaces for Apache Mesos tools. Before coming to Mesosphere, Thomas was working on distributed systems at BitTorrent.

Thursday August 20, 2015 11:00am - 11:40am PDT
Grand Ballroom D

User Experience: Beginner

11:50am PDT

The New Mesos HTTP API - Vinod Kone, Twitter, Isabel Jimenez, Mesosphere

The previous Mesos API provided language bindings (a.k.a. drivers) for frameworks to interact with Mesos. Framework developers don’t like dealing with dependencies in native libraries, and with Mesos’ new HTTP API, they no longer have to! In this session, (Vinod Kone, Technical Lead for Mesos at Twitter and Isabel Jimenez, Software Engineer at Mesosphere) will provide a comprehensive walk-through of recent advancements with the Mesos API, explaining the design rationale and highlighting specific improvements that simplify writing frameworks to Mesos.

Speakers

Isabel Jimenez

Distributed systems engineer, Mesosphere

Isabel Jimenez is a Software Engineer at Mesosphere.

Vinod Kone

Vinod Kone is the Technical Lead for Mesos at Twitter. Vinod leads the Mesos team. Mesos is a fundamental component of Twitter's data center infrastructure powering a lot of its key services.

Thursday August 20, 2015 11:50am - 12:30pm PDT
Willow

Experience: Advanced Internals / Extensions

11:50am PDT

Preemptive Task Scheduling in Mesos Framework - Li Jin, Two Sigma

Two Sigma uses Apache Mesos as the core of its computing infrastructure. To solve the scheduling problem of lots of resource-hungry computation tasks, we extended Dominant Resource Fairness (DRF), the core algorithm in Mesos allocator, to support preemption so the resource allocation remains fair with continuously changing demands. We will explain our extension of DRF and how it allows us to achieve very short per-user wait times in a constantly saturated Mesos cluster. We will also discuss how the core Mesos allocator itself could be enhanced to use this technique.

Speakers

Li Jin

Two Sigma LLC, Software Engineer

Li Jin is a distributed systems developer in Two Sigma. Li works on a Mesos framework that runs thousands of computing tasks every day. In his spare time, Li likes to read distributed system papers, hiking and winter sports.

Thursday August 20, 2015 11:50am - 12:30pm PDT
Grand Ballroom C

Experience: Intermediate Internals / Extensions

11:50am PDT

How to Monitor Mesos - Alexis Le-Quoc, Datadog

By providing a robust abstraction over core computing resources, Mesos does away with what has been until now the foundation of most monitoring systems: the individual host. With that gone, what should monitoring of applications running on Mesos revolve around? In this talk, Alexis Le-Quoc argues that imperative monitoring of hosts must give way to declarative monitoring, built on tags, tasks and queries. With concrete examples and live monitoring data, he will present a better way to monitor Mesos.

Speakers

Alexis Le-Quoc

Datadog

Thursday August 20, 2015 11:50am - 12:30pm PDT
Grand Ballroom B

Experience: Intermediate Ops

12:30pm PDT

Lunch

Thursday August 20, 2015 12:30pm - 2:00pm PDT
Grand Ballroom

Break

12:45pm PDT

Intel Lunchtime Keynote

Thursday August 20, 2015 12:45pm - 12:52pm PDT
Grand Ballroom

Keynote

12:55pm PDT

Mesos - EGO integration: new tools for enterprise cluster management

Our team works on integration of Mesos and IBM Platform EGO technologies. The idea is to provide Mesos users with new capabilities, such as hierarchical resource management, centralized administration, flexible reservation and sharing mechanisms, time-based policies - and other features required for managing enterprise data centers. The integration is performed by developing a Mesos resource allocator plug-in module, that connects to a Platform EGO system - a technology with years of experience of managing enterprise server clusters in many different industries. In this talk, I'll describe the Platform EGO policy capabilities, and how we bring them to the Mesos ecosystem for plug-in deployment in the new and existing installations.

Presenter

Khalid Ahmed

Distinguished Engineer, IBM

Khalid Ahmed is an STSM, Chief Architect of Infrastructure Software at IBM Platform. He works on the design and architecture of large scale grid and cloud computing systems with focus on scheduling, resource, workload and data management. In over 20 years at industry experience he... Read More →

Thursday August 20, 2015 12:55pm - 1:10pm PDT
Grand Ballroom

Lightning Talk

1:02pm PDT

Melting Snowflakes with microservices-infrastructure

You're building your infrastructure on Mesos, this takes time and talent. You've got the chops, but is your time worth figuring out the integration? Service discovery? Configuration management? Key management? microservices-infrastructure is the known good configuration that will melt snowflake configurations. I will talk about some challenges in integrating all the components we have, and why it's good for you.

Presenter

Brian Hicks

Based in St. Louis, Brian Hicks works for Asteris LLC, a consultancy focused on providing support and services for Mesos deployments. He enjoys cats and tweeting about cats.

Thursday August 20, 2015 1:02pm - 1:07pm PDT
Grand Ballroom

Lightning Talk

1:10pm PDT

Mesos Frameworks and Modules for Routers and Switches to tackle Network Degradation

In order to meet Quality of Service requirements for Distributed applications on server clusters, network devices such as routers and switches have always been monoliths that are statically (over) provisioned.

With evolving support for containers and linux based application hosting on Network devices, we hook up network devices to Mesos and look at the network infrastructure as a source of network resources and telemetry data. This session will introduce QoSon, a distributed network monitoring framework and discuss its use cases with Marathon and SDN.

Presenter

Akshat Sharma

Akshat Sharma is a Technical Marketing Engineer at Cisco Systems. He has 6 years of experience in the Networking industry with focus on automation, SDN, Web-scale Service Provider and Datacenter deployments and routing protocols. When he’s not busy architecting DevOps and continuous... Read More →

Thursday August 20, 2015 1:10pm - 1:15pm PDT
Grand Ballroom

Lightning Talk

1:17pm PDT

Building Docker Images on Mesos with Portainer

If you're running Apache Mesos & Docker, you're going to need to build Docker images. Portainer is an open source Apache Mesos framework developed at DueDil for remotely building Docker images on a cluster. In this talk Tom will explain how Portainer works, and show how DueDil uses it to build thousands of images for production and CI with zero additional capacity.

Presenter

Tom Arnfeld

Tom is a Data Engineer at DueDil, a business information company based in London. He's been working with Apache Mesos for the past 18 months, developing and deploying frameworks and other data-related applications with Mesos and Docker.

Thursday August 20, 2015 1:17pm - 1:22pm PDT
Grand Ballroom

Lightning Talk

1:25pm PDT

Role Quotas

Quotas provide non-revocable resource guarantees without tying reservations to particular Mesos Agents. Quota is not a limit, but rather a set of resources reserved for a specific role in an entire cluster; it can be viewed as guaranteed allocation for a role. These resources cannot be hijacked by other roles and are guaranteed to be available for the role if there are enough resources in the cluster.

Presenter

Alex Rukletsov

Alex Rukletsov is on the Mesosphere core team in Hamburg and focuses on open source Apache Mesos. During past years Alex was segmenting medical images and investigating behaviour of human vessels in several German research institutes. His areas of interests include distributed systems... Read More →

Thursday August 20, 2015 1:25pm - 1:30pm PDT
Grand Ballroom

Lightning Talk

1:30pm PDT

Ceph on Mesos

The ceph-mesos project is a Apache Mesos framework for scaling a Ceph cluster. The framework has a REST API interface to select the binaries to execute and the framework executors launch Ceph Docker containers on Mesos slaves accordingly.

In conjunction with the development of RGWFS, a HCFS over Ceph, our team is building a Big Data analytics platform on top of Apache Mesos using Ceph as its backend storage. We have developed a Apache Mesos framework to manage a Ceph cluster with all other Big Data related applications. The ceph-mesos framework enables us to deploy Ceph within 10 seconds on a freshly installed 19 node rack since the binaries are launched within a Docker container. We plan to do more testing and share performance test results and usecase scenarios.

Presenter

Zhidong Yu

Engineering Manager, Intel Corporation

Thursday August 20, 2015 1:30pm - 1:35pm PDT
Grand Ballroom

Lightning Talk

1:37pm PDT

Centralized logging solution using ELK stack

There are quite a few logging solution for Docker but none of them truly fit Mesos Landscape. There isn't a centralized solution for Mesos logs.

We have developed a logging solution that is self sustainable without having to manage logging infrastructure. Lookup for the logs for any app running under Mesos is extremely fast and highly indexed. It is horizontally scalable and all the components are dockerized and are running under Mesos.

Presenter

Imran Shaikh

Imran Shaikh breathes DevOps, embodies it and that is what will ooze out if you cut him. He is a Lead Systems Engineer working at YellowPages. He has an industry experience of 10+ year working with Fortune 500 companies. He has worked extensively developing, architecting and managing... Read More →

Thursday August 20, 2015 1:37pm - 1:42pm PDT
Grand Ballroom

Lightning Talk

2:00pm PDT

Heterogeneous Resource Scheduling Using Apache Mesos for Cloud Native Frameworks - Sharma Podila, Netflix

Netflix customers stream more than two billion hours of content each month, accounting for over a third of downstream Internet traffic. At this scale, real time insights are critical to a smooth streaming experience. Sharma Podila will present an in-depth look at the technology - including the Fenzo task scheduler for Apache Mesos and Mantis, a cloud native event stream processing platform - that Netflix uses to gain real time operational insights into system behavior. The talk will focus on the design and usability of Fenzo and will provide details on how it uses CPU, memory, and network bandwidth resources for job scheduling. The presentation will showcase bin packing, task locality, resource affinity, task constraints, and autoscaling of the slave cluster. To close Sharma will talk about future directions and opportunities to collaborate with the Mesos community.

Speakers

Sharma Podila

Senior Software Engineer, Netflix

Sharma Podila is a Senior Software Enginner in the Edge Engineering team at Netflix, Inc. His current work includes Fenzo, an open source generic scheduler with plugin based optimizations, developing cloud native Mesos frameworks, and evolving the microservices platform at Netfli... Read More →

Thursday August 20, 2015 2:00pm - 2:40pm PDT
Willow

Experience: Advanced Framework

2:00pm PDT

Resource Optimization with Charmander - Marcel Neuhausler, AT&T Foundry

Charmander is a closed loop data analysis and optimization environment for a container based infrastructure. Charmander is open source (https://github.com/att-innovate/charmander) and is based on Mesos, Docker, cAdvisor, InfluxDB, Apache Spark, and Redis. In this presentation, Marcel Neuhausler will showcase different use-cases for Charmander, provide a technical overview of the components, and deep dive into its own Mesos Framework.

Speakers

Marcel Neuhausler

Marcel Neuhausler is a Principal Software Engineer at the AT&T Foundry in Palo Alto focusing on resource-conscious distributed systems.

Thursday August 20, 2015 2:00pm - 2:40pm PDT
Grand Ballroom C

Experience: Intermediate Framework

2:00pm PDT

Twitter’s Production Scale: Mesos and Aurora Operations - Joe Smith, Twitter

Twitter has used Aurora and Mesos to scale from only tens of nodes to tens of thousands. During the process of scaling up, the configuration, deployment, and operational procedures for this system have evolved and improved significantly. This talk will offer an operations perspective on the management of a Mesos+Aurora cluster, and cover many of the cluster management best practices that have evolved at Twitter from real-world production experience. It will explore methods that decrease operational overhead, as well as examples of outages and incidents to illustrate various failure domains. Furthermore, the talk will highlight current and future safeguards in Aurora and Mesos to mitigate impact from these failures in the future.

Speakers

Joe Smith

Site Reliability Engineer, Slack

Joe Smith is a Site Reliability Engineer at Slack, working on the Identity, Compliance, & Security team. Previously he was at Twitter, and built the Aurora and Mesos cluster from tens of nodes to tens of thousands. As Mesos and Aurora SRE, he automated the build, deployment, management... Read More →

Thursday August 20, 2015 2:00pm - 2:40pm PDT
Grand Ballroom B

Ops Experience: Any

2:50pm PDT

Supporting Stateful Services on Mesos using Persistence Primitives - Jie Yu, Twitter & Michael Park, Mesosphere

This talk will discuss the persistence primitives recently built into Mesos. Persistent primitives provide native support for running stateful services like Cassandra and MySQL in Mesos, which makes it possible for us to further consolidate workloads in the data center and improve overall cluster utilization. The goal of persistent primitives is to allow a framework to have guaranteed access to its lost state even after task failover or slave restart. To achieve that, we first discuss a primitive that allows tasks to write persistent data which won’t be garbage collected after tasks finish. Then, we discuss the primitive that allows frameworks to dynamically reserve resources for their stateful tasks upon launching so that the stateful tasks can always be re-launched on the same slave that contains their persisted state.

Speakers

Michael Park

Software Engineer, Facebook

I'm a software engineer at Facebook, working on the C++ libraries and standards team. My focus for C++ is to introduce pattern matching to facilitate better code.

Jie Yu

Tech Lead, Mesosphere

Jie Yu is a Tech Lead at Mesosphere, Inc, focused on containerization, storage and networking. Before joining Mesosphere, he was a software engineer at Twitter. Jie obtained his PhD in Computer Science and Engineering from the University of Michigan where he conducted research for... Read More →

Thursday August 20, 2015 2:50pm - 3:30pm PDT
Willow

Experience: Intermediate Internals / Extensions

2:50pm PDT

Mesos at OpenTable - Pablo Delgado

Opentable has been using Apache Mesos for production workloads and for running critical parts of their production services for more than a year. Not only did Mesos help deploying resilient / elastic standalone applications and services , but also the distributed / fault-tolerant frameworks like Apache Spark for Data processing and machine learning. Mesos enabled Opentable to run multiple distributed applications across the same infrastructure at scale. Pablo will tell the story of how Opentable started with Mesos, the pain points of dealing with an hybrid Mesos + non-Mesos environment and how to survive in the transition.

Speakers

Pablo Delgado

Senior Data Engineer, OpenTable, Inc.

Thursday August 20, 2015 2:50pm - 3:30pm PDT
Grand Ballroom D

Experience: Intermediate User

2:50pm PDT

Scaling at eBay with Mesos - Soma Shekar Oruganti, eBay

eBay Inc. has embraced Jenkins for it’s continuous integration and delivery processes. It was challenging to efficiently use the infrastructure resources to accommodate the continuous integration process at the scale of eBay. Apache Mess and Marathon have made it possible for eBay to build a highly scalable, available and self healing processes, with efficient utilization of resources. Along the journey, eBay had challenges and overcome them effectively, and contributed back the solutions to the open source community with contributions to Jenkins framework for Mesos.

Speakers

Soma Shekar Oruganti

MTS 1, Paypal Inc

Soma Shekar works as principal software engineer with Global Platform and Infrastructure Team at eBay Inc. The team is responsible for the tools used by eBay Inc.(both eBay and Paypal) developer and QA community for their Continuous Integration and Delivery processes. eBay have embraced... Read More →

CIaaS1.0 1200 pdf

Thursday August 20, 2015 2:50pm - 3:30pm PDT
Grand Ballroom B

Experience: Intermediate User

2:50pm PDT

Crossing the Streams: Leveraging Kubernetes Services From Other Mesos Frameworks and Vice Versa - Timothy St. Clair, Red Hat

Kubernetes provides a declarative model in order to describe micro-services. However, auto-discovering those services outside of kubernetes requires add-ons. In this presentation we will explore how using the kubernetes framework and Mesos-DNS we can bridge the gap, and provide cluster wide service discovery.

Speakers

Timothy St. Clair

Principal Engineer, Red Hat

Timothy St. Clair is a Principal Software Engineer at Red Hat in the Emerging Technology office. He is a PMC member and committer to the Apache Mesos Project, and has worked on the development and integration of various open source distributed systems projects, including: Mesos, Kubernetes... Read More →

Thursday August 20, 2015 2:50pm - 3:30pm PDT
Grand Ballroom C

Framework Experience: Advanced

3:30pm PDT

Break

Thursday August 20, 2015 3:30pm - 4:00pm PDT
2nd Floor Foyer

Break

4:00pm PDT

Mesos Gets Pluggable: Introducing Mesos Modules - Kapil Arya, Mesosphere & Niklas Nielsen, Mesosphere

One of the most exciting new innovations is a simple, pluggable approach to building customizations for Mesos. This makes it as easy to extend or customize Mesos as it is to install plug-ins into your web browser. In this talk, (Niklas Nielsen and Kapil Arya, Distributed Systems Engineers at Mesosphere) will provide a technical introduction to Mesos Modules - the new way to customize Mesos without having to recompile / relink for each specific use case. They will demonstrate how to build and use Mesos Modules, including an example of how to write an allocator module for custom scheduling policies across a cluster. The presenters will also explain how Mesos Modules will help grow the ecosystem of developers building new features for Mesos. Customization will be one of strongest and most flexible attributes of the Mesos distributed systems kernel!

Speakers

Kapil Arya

Kapil Arya is an Engineer at Mesosphere focussing on the core team. He recently finished his PhD at Northeastern University where he was the lead developer of the open source distributed checkpointing project DMTCP, and contributed to the reversible debugger FReD. Interning at VMware... Read More →

Niklas Nielsen

Niklas Nielsen is an engineer at Mesosphere. He is a C++ hacker and is works on the Mesos core. Before joining Mesosphere he worked at Adobe on the Flash VM and Google V8, as well as a secret next-generation VM. He did his Master's thesis at Lawrence Livermore National Lab, working... Read More →

Thursday August 20, 2015 4:00pm - 4:40pm PDT
Willow

Experience: Advanced Internals / Extensions

4:00pm PDT

Apache Cotton: MySQL on Mesos - Yan Xu, Twitter

Apache Cotton (previously named Mysos) is an Apache Mesos framework for running MySQL instances. It dramatically simplifies the management of a MySQL cluster and is designed to offer:
- Efficient hardware utilization through multi-tenancy (in performance-isolated containers)- High reliability through preserving the MySQL state during failure and automatic backing up to/restoring from HDFS- An automated self-service option for bringing up new MySQL clusters- High availability through automatic MySQL master failover- An elastic solution that allows users to easily scale up and down a MySQL cluster by changing the number of slave instances
We’ll share our experience developing and using this framework.

Speakers

Yan Xu

Softwae Engineer, Apple

Yan Xu is an engineering manager at Apple where he focuses on Kubernetes and containerization. He has also been a committer and PMC member of Apache Mesos since its early days when he worked at Twitter.

Thursday August 20, 2015 4:00pm - 4:40pm PDT
Grand Ballroom C

Experience: Intermediate Framework

4:00pm PDT

Running Stateful Services with Mesos - Arunabha Ghosh, Moz & Ankan Mukherjee, Moz

The Mesos system allows services to be decoupled from machines, however this decoupling creates problems for legacy apps that rely on persistent state. Traditional SQL databases are a prime example of such apps. It is still possible, however to run such apps under Mesos and gain the operational advantages it provides. We will cover techniques used at Moz to successfully run SQL databases and other legacy persistent state services under Mesos. The presentation will also cover challenges, best practices and a look at how to leverage Mesos primitives for stateful services. We will demonstrate several such services running on Mesos.

Speakers

Arunabha Ghosh

Moz

Arunabha Ghosh is the Director of Engineering at Moz where among other things he leads the effort to move Moz onto Mesos. Prior to Moz, Arunabha worked at Yahoo and Google focusing on building large scale infrastructure. Arunabha leads the Systems Research group at the HackerDojo... Read More →

Ankan Mukherjee

Ankan Mukherjee is a senior engineer at Moz and is currently focussed on building the next generation cluster operating system for Moz's datacenters. Prior to working at Moz he donned many different roles in the enterprise software world - software engineer, technical architect, technical/project... Read More →

Thursday August 20, 2015 4:00pm - 4:40pm PDT
Grand Ballroom B

Experience: Intermediate Ops

4:00pm PDT

Developing Frameworks for Apache Mesos - Joe Stein, Big Data Open Source Security

Using schedulers like Marathon and Aurora help to get your applications scheduled and executing on Mesos. In many cases it makes sense to build a framework and integrate directly. This talk will breakdown what is involved in building a framework, how to-do this with examples and why you would want to-do this. Frameworks are not only for generally available software applications (like Kafka, HDFS, Spark ,etc) but can also be used for custom internal R&D built software applications too.

Speakers

Joe Stein

Big Data Open Source Security LLC, Big Data Open Source Security LLC

Joe Stein is an Apache Kafka committer and PMC member. A frequent speaker also on both Hadoop and Cassandra, Joe is the Founder and Principal Architect of Big Data Open Source Security LLC http://stealth.ly a professional services and product solutions company. Joe has been a distributed... Read More →

Thursday August 20, 2015 4:00pm - 4:40pm PDT
Grand Ballroom D

User Experience: Any

4:50pm PDT

Building A Machine Learning Platform to Predict User Behavior on Mesos - Jeremy Stanley, Sailthru

In 2014 Sailthru launched a new machine learning platform, named Sightlines, using Mesos. This platform predicts the behavior of hundreds of millions of users for hundreds of clients. In this talk, Jeremy Stanley and Alex Gaudio will tell the story of how (and why) this platform was built on Mesos, and outline some of the key challenges and detours taken along the way. This will include an overview of the goals and architecture of the system, and an account of the 10 month effort to develop, scale and stabilize the system for production.

Topics covered will include our use of AWS spot instances and Netflix’s Asgard to save costs, Marathon and Zookeeper to schedule jobs, detours with Chronos, Spark and Redis as our architecture evolved, and challenges we faced with maximizing resource utilization and ensuring we had identical and isolated test and production environments.

Speakers

Jeremy Stanley

"Jeremy Stanley is the Chief Data Scientist & EVP of Engineering at Sailthru, where he is focused on building data-driven solutions for marketers that drive long-term customer engagement and optimize revenue opportunities.Prior to Sailthru, Jeremy was the CTO at Collective where he... Read More →

Thursday August 20, 2015 4:50pm - 5:30pm PDT
Grand Ballroom B

Experience: Any Ops

4:50pm PDT

Mesos Networking - Christos Kozyrakis, Mesosphere and Spike Curtis, Project Calico

Because Mesos abstracts the physical location of containers, network connectivity, service discovery, and network security can be quite challenging. In this talk, we will review the networking features necessary to make Mesos practical in large scale, multi-tenant clusters. Specifically, we will describe how to assign a separate IP address per container, how to isolate containers on demand with a distributed firewall, and how to automate service discovery by integrating Mesos with DNS. We will demonstrate the first implementation of these features that uses Project Calico, a pure layer-3 approach to virtual networking for scalable datacenter.

Speakers

Spike Curtis

Spike Curtis is a Core Developer and Evangelist at Project Calico. He has been a developer in advanced network technologies since 2010. Prior to his work in networks he was a researcher in experimental quantum computing at the University of Oxford.

Christos Kozyrakis

Mesosphere

Christos Kozyrakis is the Director of Research at Mesosphere. He focuses on automating resource management and raising efficiency in Mesosphere clusters. Christos has over twenty years of research experience on hardware architectures, runtime management environments, and system software... Read More →

Thursday August 20, 2015 4:50pm - 5:30pm PDT
Grand Ballroom D

Experience: Beginner User

4:50pm PDT

Per Container Network Monitoring and Isolation in Mesos - Jie Yu, Twitter

This talk will discuss the per container network monitoring and isolation feature introduced in Mesos 0.21.0 and our experiences in using it at Twitter. With this feature enabled, each container is assigned a separate network stack which isolates the network traffic from other containers and allows operators and users to easily get its network statistics. This greatly simplifies the network monitoring, health checking and troubleshooting for jobs, and provides a more fairly shared and predictable network to users. This feature is completely transparent to tasks, and does not require changes to the underlying network infrastructure. To achieve that, the key idea is to let containers share the same host IP and use ports to multiplex network packets. We will show you the implications of this approach and the lessons we learned during the deployment and ongoing use of this feature.

Speakers

Jie Yu

Tech Lead, Mesosphere

Thursday August 20, 2015 4:50pm - 5:30pm PDT
Willow

Experience: Intermediate Internals / Extensions

4:50pm PDT

Distributed CI: Scaling Jenkins on Mesos and Marathon - Roger Ignazio, Puppet Labs

As engineering teams and product offerings expand, monolithic Jenkins instances grow and become single points of failure. By running Jenkins on Mesos and Marathon, masters can be scaled horizontally and executors provisioned dynamically as demand increases. This presentation will demonstrate that running Jenkins at-scale is more feasible than ever. The architecture provided by Mesos moves Continuous Integration (CI) from being synonymous with Jenkins to Jenkins being a component of the overall CI system.

This presentation will cover:
- Conventional scaling techniques and their shortcomings
- Scaling Jenkins masters and slaves horizontally on Mesos and Marathon
- Dynamically creating and executing jobs
- Using Elasticsearch, Logstash, and Kibana (ELK) to store, query, and visualize run data

Thursday August 20, 2015 4:50pm - 5:30pm PDT
Grand Ballroom C

Experience: Intermediate Framework

8:00pm PDT

Attendee Reception at the Experience Music Project Museum

Open to all #MesosCon attendees, join us at the Experience Music Project (EMP), a leading-edge nonprofit museum, dedicated to the ideas and risk-taking that fuel contemporary popular culture. Appetizers and drinks included as well as the opportunity to check out some of the current exhibits. A shuttle will be provided to/from the Sheraton.

Thursday August 20, 2015 8:00pm - 10:30pm PDT
Experience Music Project

8:00am PDT

Registration and Continental Breakfast

Friday August 21, 2015 8:00am - 9:00am PDT
2nd Floor Foyer

Break

9:00am PDT

Announcements

Friday August 21, 2015 9:00am - 9:05am PDT
Grand Ballroom

Keynote

9:05am PDT

Keynote: How we run Siri - Robert Lacroix & Brian Sumner, Apple

Robert Lacroix & Brian Sumner discuss "How we run Siri."

Speakers

Robert Lacroix

Brian Sumner

Friday August 21, 2015 9:05am - 9:20am PDT
Grand Ballroom

Keynote

9:20am PDT

Keynote: Growing the Mesos Ecosystem - Florian Leibert, CEO at Mesosphere

Details coming soon.

Moderators

Florian Leibert

Florian is CEO at Mesosphere.

Friday August 21, 2015 9:20am - 9:30am PDT
Grand Ballroom

Keynote

9:35am PDT

Keynote: Silence is Golden: Coordination-Avoiding Systems Design - Peter Bailis

Computer networks make it difficult to design scalable, robust distributed systems that exhibit good performance. Networks can be slow, have limited capacity, and are often unreliable. In an ideal world, we'd build systems that don't rely on the network at all. Unfortunately, as a slew of negative results like the CAP Theorem illustrate, this isn't always possible. Traditional systems abstractions like ACID transactions fundamentally require synchronous communication, or coordination, to implement. As a result, coordination-free systems designs often forego many programmer-friendly abstractions. These systems leave the task of reasoning about correctness to the application developer or, worse, to the end user.
In this talk, I'll discuss an alternative: system designs that coordinate only when necessary to guarantee application correctness. This coordination avoidance maximizes scalability and robustness by minimizing reliance on the network. To illustrate the power of coordination-avoiding systems design, I'll present several case studies from our research spanning database isolation guarantees, indexes and constraints, and open source applications. Perhaps surprisingly, even though traditional implementations of these tasks rely on coordination, many of these tasks don't actually require coordination for correctness. The resulting systems are among the fastest prototypes ever built and operated at scale. Based on these case studies, I'll provide concrete and practical design principles for reasoning about and applying coordination avoidance in the wild.

Speakers

Peter Bailis

Peter Bailis is a final-year Ph.D. candidate at UC Berkeley working in databases and distributed systems. As part of his dissertation work, he has studied and built high performance distributed data management systems for large scale transaction processing, data serving, and statistical... Read More →

Friday August 21, 2015 9:35am - 10:15am PDT
Grand Ballroom

Keynote

10:10am PDT

Break

Friday August 21, 2015 10:10am - 10:40am PDT
2nd Floor Foyer

Break

10:40am PDT

Myriad: Resource Sharing Beyond Boundaries - Mohit Soni, PayPal

Last year at MesosCon, eBay presented a novel solution which allows Mesos and Apache YARN/Hadoop to co-exist without statically partitioning the data center. That solution evolved into a community project called "Myriad" and is under active development. In this talk, we will present the latest architecture of "Myriad", how it improves overall resource utilization in the datacenter, and how it enables dynamic allocation of resources between workloads running on Mesos and YARN.

Speakers

Santosh Marella

Santosh Marella is a Software Engineer at MapR Technologies. Since joining MapR, Santosh worked on numerous projects starting from MapR's admin infrastructure, MapReduce and YARN. He currently leads MapR's efforts on Myriad. Prior to MapR, Santosh was a Software Engineer at Google... Read More →

Mohit Soni

Software Engineer, PayPal

Mohit works as an engineer in platform team at eBay Inc. His job is to focus on maximizing efficiency, increasing agility, and reducing cost. Mohit presented in past at DockerCon 2014, Hadoop India Summit 2011 and BarCamp 2010. You can follow Mohit as mohitsoni on GitHub and Twit... Read More →

Friday August 21, 2015 10:40am - 11:20am PDT
Grand Ballroom C

Experience: Any Framework

10:40am PDT

Save Millions by Efficient Resource Utilization Through Mesos - Smarth Madan, PayPal

When it comes to hardware, most companies underutilize it; sometimes letting it lying idle for days without activity. When we started the project to build a staging environment for PayPal, we had the opportunity to design an environment, which is dynamic, on demand, and highly available. Through Mesos and the Aurora scheduler, we are able to deploy 200+ VMs simultaneously and create a mini PayPal environment for developers to connect from their VMs, desktops or mobile devices. This presentation will cover how we are reducing operational cost of hardware and building a scalable environment, which is improving the efficiency for PayPal developers.

Speakers

Smarth Madan

Smarth has been developing software at PayPal for the past 3 years. He is currently working on a project to build a staging environment for PayPal. He completed his MS in Software Engineering from San Jose State University and has been working in IT for more than 5 years now. He... Read More →

Friday August 21, 2015 10:40am - 11:20am PDT
Grand Ballroom D

Experience: Beginner User

10:40am PDT

Global Control of Decentralized Mesos Clusters - Daniel Giribet, Schibsted

Apache Mesos and the frameworks running on top of them are a great resource management solution but they lack some features to make it suitable for automatic scaling and communication of microservices that run across different locations and vendors. In this presentation Schibsted will discuss these limitations and present their plan for a Global Scheduler capable of controlling independent Mesos clusters across multiple distant datacenters and service providers.

Speakers

Daniel Giribet

Daniel Giribet is the Infrastructure Platform Development leader at Schibsted Products and Technology. He holds a Computer Science degree and has been focusing on video processing, web engineering, systems architecture and content management. Has worked in indie projects and also... Read More →

Friday August 21, 2015 10:40am - 11:20am PDT
Grand Ballroom B

Experience: Intermediate Ops

10:40am PDT

Securing your Mesos Cluster - Adam Bordelon, Mesosphere & Joris Van Remoortere, Mesosphere

Security features are in high demand as organizations increasingly trust their Mesos clusters with critical production workloads and sensitive data, while at the same time providing cluster access to naive/untrusted users. The same questions keep coming up:

How can Mesos prevent unauthorized users from intercepting sensitive data or forging communications?
How can Mesos prevent unauthorized frameworks from launching malicious jobs and prevent unauthorized slaves from stealing tasks and data?
How can Mesos ensure users and frameworks can only access their own tasks/data?

In this first ever MesosCon security deep-dive, Adam and Joris will walk through the best practices and latest advances in securing Mesos clusters, including encryption, authentication, and authorization features.

Speakers

Adam Bordelon

Mesosphere, Distributed System Engineer

Adam Bordelon is a distributed systems architect at Mesosphere and an Apache Mesos committer. Before joining Mesosphere, Adam lead development on Hadoop core at MapR, built distributed systems for recommendations at Amazon, and re-architected the LabVIEW compiler at National Instruments... Read More →

Joris Van Remoortere

Joris Van Remoortere is a Senior Software Engineer at Mesosphere. He focuses on large-scale, low-latency, high-throughput systems. He alleviates scaling issues and makes performance and reliability improvements on many projects, including Orly, messaging and payment systems, web tiers... Read More →

Friday August 21, 2015 10:40am - 11:30am PDT
Willow

Experience: Intermediate Internals / Extensions

11:30am PDT

Generalizing Software Deployment - The Many Meanings of "Update" - Bill Farner, Twitter

Shipping code is an everyday part of the life of a developer. And when shipping code in a Mesos cluster, that workflow is managed by individual schedulers. In the case of Apache Aurora, developers are empowered by a streamlined deployment mechanism. This talk will cover the evolution of how Aurora manages deployments, and describe some of the challenges imposed by the wide variance in requirements. Finally, there will be discussion of how deployments on Aurora currently work for major services at Twitter.

Speakers

Bill Farner

Bill Farner has been a Software Engineer at Twitter for the last five years, serving as the Technical Lead on Aurora since its inception four years ago. He previously worked at Google on Production Engineering and storage infrastructure. Bill holds a MS in Computer Engineering from... Read More →

Friday August 21, 2015 11:30am - 12:10pm PDT
Grand Ballroom C

Experience: Any Framework

11:30am PDT

Migrating over 1,000 Production Components to a Mesos-Based Platform-as-a-Service (PaaS) - Tom Petr, HubSpot

At HubSpot we built a turn-key PaaS system our engineers use to deploy all the different parts that make up the HubSpot application. In this talk we'll discuss the benefits of using Mesos as a starting point for deploying a PaaS, the open source framework we created called Singularity, and the lessons learned from moving our entire production application to this stack. We will enumerate many of the specific benefits that became apparent after the system was in place, including enhanced security, reliability, operational simplicity, and cost efficiency. Lastly, we will shed light on both the ways in which we extended what Mesos offered in order to empower our product developers, and how we moved over 100 software load balancers into Mesos to make our infrastructure team more efficient.

Speakers

Tom Petr

Engineering Lead, HubSpot

Tom Petr is an Engineering Lead at HubSpot. Prior to working on Vitess and Kubernetes, he was a maintainer of Singularity, an open source Mesos framework. Tom has spoken at multiple conferences about HubSpot's platform infrastructure.

Friday August 21, 2015 11:30am - 12:10pm PDT
Grand Ballroom B

Experience: Any Ops

11:30am PDT

Enabling Hybrid Workloads with Docker/Mesos at Orbitz - Steve Hoffman, Orbitz

The promise of Mesos to create a datacenter-centric resource pool is a great idea. However, a decade old company with many different types of existing systems and unique workloads can't simply mass migrate to Mesos. That doesn't mean Mesos can't pay a huge part in unifying disparate workloads that construct existing systems while opening the door to new Mesos-centric development.

In this presentation, Steve will cover 3 very different use cases that are being unified under Apache Mesos including: Automated builds of existing code artifacts, automated builds of AWS AMIs via packer in EC2, and rolling deploys & supervision of Docker microservices across environments. The addition of Docker on Mesos has enabled this, but that doesn't mean that all Mesos slaves should be identical. See how the addition of localized support services on slaves can compliment your Docker on Mesos use cases.

Speakers

Steve Hoffman

Architect, Journera, Inc.

Tech guy at Journera. Minutia @ http://bit.ly/bacoboy

Friday August 21, 2015 11:30am - 12:10pm PDT
Grand Ballroom D

Experience: Any User

11:30am PDT

Rocket and the Need for the App Container Specification - Brandon Philips, CoreOS

Rocket is a simple daemon-free tool that enables users to run containerized apps on their systems free of host dependencies. Containers running under rocket execute like regular processes and can be managed using existing process management tools like upstart, systemd, runit and etc.

Rocket is also an implementation of the “App Container spec,” which defines how to define and build containerized applications using tooling like tar and PGP, and then host these files easily using standard protocols like HTTP. The goal of the spec is to enable independent and creative implementations of container runtimes and build tools.

This presentation will give an overview of the App Container spec and why the spec is important to the Mesos community. This session invites attendees to learn how they work and how to get involved.

Speakers

Brandon Philips

Brandon Philips is the CTO of CoreOS where he is helping to build modern Linux server infrastructure. Prior to CoreOS, he worked at Rackspace hacking on cloud monitoring and was a Linux kernel developer at SUSE. As a graduate of Oregon State's Open Source Lab he is passionate about... Read More →

Friday August 21, 2015 11:30am - 12:10pm PDT
Willow

Experience: Intermediate Internals / Extensions

12:10pm PDT

Lunch

Friday August 21, 2015 12:10pm - 1:30pm PDT
Grand Ballroom

Break

1:30pm PDT

Tactical Mesos: How Internet-Scale Ad Bidding Works on Mesos / Aurora - Dobromir Montauk, TellApart

Real Time Bidding on the large Internet exchanges (Doubleclick, Facebook, etc) requires large-scale, low-latency serving systems: >100K QPS at peak with <100ms tail response times. Time is very literally money. Dobromir will present TellApart's full stack in excruciating detail (if you want it), which includes Mesos/Aurora, ZK service discovery, Finagle-Mux RPC, and a Lambda architecture with Voldemort as the serving layer.

Speakers

Dobromir Montauk

Software Engineer, Twitter

Nine years at Google, including work on Google+ Stream backend and Search Infrastructure. Recently joined TellApart as Uber Tech Lead working on infrastructure. In charge of TellApart bidding platform performance: >100K QPS with sub-10ms latencies. Optimized the Bidders to run at... Read More →

MesosCon Preso Tactical Mesos (public) pdf

Friday August 21, 2015 1:30pm - 2:10pm PDT
Grand Ballroom B

Experience: Advanced Ops

1:30pm PDT

Satellite: A Monitoring Application for Mesos - Sunil Abraham, Two Sigma Investments

Satellite is a monitoring and alerting solution for Mesos. Satellite’s powerful event streams processing engine allows /ops teams to build custom monitoring and alerting solutions that will improve uptime and reliability of the underlying compute resources in the cluster. In this talk, you'll learn how Two Sigma is using Satellite to monitor and take action on the hosts in our Mesos cluster. After a quick introduction to the architecture of Satellite, we'll walk through the process of defining a rule, adding a Pagerduty alert, and programmatically taking action on the state of the cluster by enabling and disabling hosts in the cluster.

Speakers

Sunil Abraham

Sunil Abraham is a developer at Two Sigma Investments. Previously, Sunil was a Fulbright Scholar at the University of Waterloo working on public health and studied math and statistics at Oxford and Cambridge.

Friday August 21, 2015 1:30pm - 2:10pm PDT
Willow

Experience: Any Internals / Extensions

1:30pm PDT

Docker Swarm on Mesos - Tim Chen, Mesosphere & Victor Vieux, Docker

Docker has become a popular way for developers to package their applications for deployment, but as applications get more complex there is a growing need to deploy them at scale, orchestrating them across dozens or thousands of machines. With Docker Swarm, you can talk to Mesos using the Docker command line client and start Docker containers on your Mesos cluster as easily as you would do it on your development laptop. In this presentation, Victor Vieux (Docker Engineer, Docker) and Timothy Chen (Mesosphere Engineer, Mesosphere) will explain why Mesos is the “gold standard” for running Docker containers at scale and will demo how to setup Docker Swarm on Mesos (using DCOS) and how to do a `docker run` against your Mesos cluster easily.

Speakers

Tim Chen

Tim Chen is on the Mesosphere core team and focuses on containerization and big data frameworks. He is also a PMC/committer on Apache Drill and Apache Mesos, and contributed to other open source projects such as Spark, Kafka, Hive. Before joining Mesosphere, Tim past experiences includes... Read More →

Victor Vieux

Docker Core Engineer, Docker, Inc

Victor Vieux works as a software engineer at Docker, Inc. He was one of the few guys working on the Docker internals, he's now working on Docker Swarm, a Docker-native clustering system. Originally from Paris, France, where he was famous for his work on robots (programming them in... Read More →

Friday August 21, 2015 1:30pm - 2:10pm PDT
Grand Ballroom C

Experience: Intermediate Framework

1:30pm PDT

Workshop: A Field Guide to Mesos Frameworks - Adam Bordelon, Mesosphere; Elizabeth Lingg, Mesosphere; Niklas Nielsen, Mesosphere

Frameworks are the lifeblood of Mesos, since they provide the distributed applications and services that run on top of Mesos. Many users are perfectly content using the various frameworks already built for Mesos, but others may need support for a new application or require custom behavior that is not already available. In this workshop, experts from Mesosphere will explain why, when, and how you should build your own framework. In addition to the basics of resource offers and status updates, they will dive into “enterprise-grade” requirements like high availability, authentication, advanced scheduling, and persistent state. Whether you are building your own custom framework or just getting to know the existing Mesos frameworks, this workshop will give you the tools and knowledge you need.

Speakers

Adam Bordelon

Mesosphere, Distributed System Engineer

Elizabeth Lingg

Elizabeth Lingg a Technical Lead Software Engineer at Mesosphere and applies her interest in algorithms and A.I. to distributed systems. Prior to Mesosphere, she worked on reasoning engines at Oracle, using Apache Lucene, Open NLP, WikiDb, various graph traversal algorithms, WordNet... Read More →

Niklas Nielsen

Friday August 21, 2015 1:30pm - 3:00pm PDT
Grand Ballroom D

Experience: Beginner Workshop

2:20pm PDT

Simplifying Maintenance with Mesos - Benjamin Mahler, Twitter

You have computing resources that your developers want to leverage. However, providing developers with direct access to machines would set you up for an operational nightmare: a critical security vulnerability comes along and now you need to reboot all of your kernels to apply the fix, without any downtime.. good luck!

Mesos is a “cluster management” layer that provides developers with access to computing resources while encouraging behavior that leads to simplified maintenance. This talk will address the challenges of maintenance when dealing with Mesos clusters running multiple frameworks (e.g. services, storage, batch compute). We’ll explore a current proposal for adding simple and flexible maintenance primitives in Mesos to address these concerns and enable tooling for automated maintenance.

Speakers

Benjamin Mahler

Software Engineer, Mesosphere

Benjamin Mahler is a committer and PMC member of Apache Mesos and has been working on Mesos since 2012. Benjamin now works at Mesosphere as a technical lead and has given Mesos related talks at several conferences and companies. His interests include distributed systems, fault tolerance... Read More →

maintenance pdf

Friday August 21, 2015 2:20pm - 3:00pm PDT
Grand Ballroom B

Experience: Any Ops

2:20pm PDT

Scaling a Highly-Available Scheduler Using the Mesos Replicated Log: Pitfalls and Lessons Learned - Kevin Sweeney, Twitter

Mesos frameworks can go from prototype to production at scale very quickly. As your users realize how easy it is to spin up tasks, they will start doing so gratefully and rapidly. This talk will arm you with techniques for writing a framework scheduler for a large-scale Mesos cluster using Apache Aurora as a case study. Aurora routinely manages 100s of thousands of tasks per Mesos cluster. To do this it maintains the state of each task and quickly reacts to changes in the cluster. To talk to Mesos Aurora relies on the Mesos JNI library. For high-availability support it relies on the Mesos replicated log for durable storage. As its clusters grew, Aurora pushed against the limits of these libraries. This talk will explore the tools the Aurora scheduler has used to meet these challenges, including Apache Thrift for schema management, and the importance of normalization.

Speakers

Kevin Sweeney

Software Engineer - Cloud Infrastructure, Twitter

"Kevin Sweeney is a software engineer at Twitter and committer on Apache Aurora, working on it since 2012. You can find him on Twitter as @kts.

kts MesosCon2015Slides Final pdf

Friday August 21, 2015 2:20pm - 3:00pm PDT
Grand Ballroom C

Experience: Intermediate Framework

2:20pm PDT

The Scheduler Meets the Network - Arunabha Ghosh, Moz

A vital part of datacenters is the network which enables communications. As the size
and complexity of applications deployed on Mesos grows, close integration with the underlying network will become crucial. Traditionally, networks have been statically configured, but the recent rise of network virtualization combined with techniques like overlay networks have enabled unprecedented network programability and control.

Mesos provides an ideal platform to leverage the dynamic capabilities of modern computer networks due to the scheduler’s complete knowledge and control over jobs in the system. The talk will showcase several features enabled by such integration such as IP per job, programmable network isolation, easy service discovery etc. The talk will conclude with a look towards the future evolution of networks and tighter integration with Mesos.

Speakers

Arunabha Ghosh

Moz

Friday August 21, 2015 2:20pm - 3:00pm PDT
Willow

Experience: Intermediate Internals / Extensions

3:00pm PDT

Break

Friday August 21, 2015 3:00pm - 3:30pm PDT
2nd Floor Foyer

Break

3:30pm PDT

Keynote: Cloud Trends, DevOps and Microservices - Adrian Cockcroft - Technology Fellow, Battery Ventures

What has happened in the last year, and where are we heading next?

Speakers

Adrian Cockcroft

Technology Fellow, Battery Ventures

Adrian Cockcroft has had a long career working at the leading edge of technology. He’s always been fascinated by what comes next, and he writes and speaks extensively on a range of subjects. At Battery, he advises the firm and its portfolio companies about technology issues and... Read More →

Friday August 21, 2015 3:30pm - 4:10pm PDT
Grand Ballroom

Keynote

4:15pm PDT

Hackathon Winner Demos, and Closing Remarks

Friday August 21, 2015 4:15pm - 4:45pm PDT
Grand Ballroom

Keynote