Loading…
This event has ended. View the official site or create your own event → Check it out
This event has ended. Create your own
View analytic

Sign up or log in to bookmark your favorites and sync them to your phone or calendar.

Wednesday, August 19
 

8:30am

Pre-Conference Hackathon hosted by Moz
The hackathon is free but registration is required. Click here for more information and to register. 

We've organized a pre-conference Apache Mesos hackathon, and we hope you'll join us!

What to Expect

The hackathon will begin at 8:30am with breakfast, and we'll begin hacking immediately at that time. For participants looking for projects to work on, we'll make space available for you to meet and share ideas. Lunch and snacks will be available throughout the day, and lunch will be provided. We'll also have a strong wifi connection, and up-to-date vagrant images that will help you get up and running with Mesos and many popular frameworks.

Hackathon Projects

What should you work on? It's really up to you, as long as it's Mesos-related. Here are a few broad categories of projects to consider.

  • Frameworks (build your own Mesos framework or add features to an existing framework)

  • Mesos Core (squashing bugs, adding new features, or even proposing a design document)

  • Ops (tools to make it easier to operate a Mesos cluster; examples may include monitoring, logging, etc)
Need a project or team to join? First register for the hackathon, then join the Mesos dev@mesos.apache.org mailing list (by emailing dev-subscribe@mesos.apache.org) and join in an email thread we'll kick off two weeks before the hackathon to brainstorm ideas and optionally form teams.

Hackathon Rules
The hackathon has a few basic rules:
  • Teams may be no larger than 5 participants
  • Coding may not begin until the hackathon officially begins, although we encourage you to come up with a plan ahead of time!
  • Source code from the hackathon must be fully available on github, bitbucket, or a public source repository in order to be successfully submitted
These rules are subject to be changed in the future. Common sense applies here: if you think you're doing something that's against the rules or that gives you an unfair advantage, you probably shouldn't do it.

THANK YOU to our hackathon sponsor and host

This year's hackathon is sponsored by SignalFx, and hosted by Moz.

 

Wednesday August 19, 2015 8:30am - 4:30pm
Moz 1100 2nd Avenue, #500, Seattle

3:00pm

Early Registration
Wednesday August 19, 2015 3:00pm - 5:00pm
2nd Floor Foyer

6:00pm

Pre-Conference Meetup hosted by Twitter

Details Below. Please RSVP here to attend.

#MesosCon is taking place in Seattle August 20-1st, and we're pleased to welcome the Apache Mesos community by hosting a pre-conference meetup and social at Twitter's Seattle office. The event will include ample opportunities to meet other members of the community, and we'll organize ourselves into groups for unconference-style discussions for part of the evening. Bring ideas about the future direction for the project and its related ecosystem of tools, uses of Mesos to discuss, or features you'd like to implement.

Event Agenda

6:00PM: Doors open; networking, pizza, and drinks
7:00PM: Form unconference discussion groups
7:10PM: Unconference sessions
7:45PM: Each unconference session reports back to the larger group
8:00: Networking
9:00PM: End of Event

Event Location and Check In

Twitter Seattle - 1501 4th Ave STE 1900 Seattle, WA 98121 Head in the 4th avenue entrance of the Century Square building and head up the escalators to find the check-in booth. Please bring valid ID.

*Note: No press will be admitted to this event; please e-mail press@twitter.com if you have any questions. Twitter reserves the right to refuse entry to this event. Twitter also reserves the right to remove individuals from Twitter's premises for disruptive or disorderly behavior, or for any other reason in Twitter's sole discretion. Please arrive before 6:30 to ensure admission to the building.



Wednesday August 19, 2015 6:00pm - 9:00pm
Twitter Seattle 1501 4th Ave. STE 1900. Seattle
 
Thursday, August 20
 

7:30am

9:00am

Keynote: State of Mesos - Benjamin Hindman, Apache Mesos VP
Speakers
avatar for Ben Hindman

Ben Hindman

Founder and Chief Architect, Mesosphere
Benjamin Hindman is a Founder and Chief Architect at Mesosphere where he leads a team building out core services for the Mesosphere Data Center Operating System (DCOS). Ben co-created Apache Mesos as a PhD student at UC Berkeley before bringing it to Twitter where it now runs on tens of thousands of machines powering Twitter's datacenters. An academic at heart, his research in programming languages and distributed systems has been published in... Read More →


Thursday August 20, 2015 9:00am - 9:35am
Grand Ballroom

9:35am

Keynote: Enabling Microservices Frameworks to Solve Business Problems - Kenneth Owens, CTO of Cloud Infrastructure Services at Cisco
Developers are driving the market for cloud consumption and leading each industry into the new era of software defined disruption. There are no longer questions about elastic and flexible agile development as the way to innovate and reduce time to market for businesses. However, Physical and cloud Infrastructure does not enable application development platforms natively nor provide the ability to create applications that are cloud native with elastic services. In addition, businesses are moving to application development architectures leveraging microservices which are becoming more strategic to their business strategy. When making the decision to build and operate an application on physical or on a cloud platform, microservices became central to your application architecture and strategy.

Speakers
avatar for Kenneth Owens

Kenneth Owens

Chief Technology Officer, Cloud Intercloud Services, Cisco
Ken Owens is Chief Technology Officer, Cloud Intercloud Services at Cisco Systems.  | Ken is responsible for creating and communicating technical/scientific vision and  | strategy for Cloud Intercloud Services (CIS) business. He brings a compelling view of  | technology trends in enterprise IT (e.g., infrastructure, computing, SaaS, virtualization,  | and cloud) and evangelizes the technology roadmap for the business. Ken... Read More →


Thursday August 20, 2015 9:35am - 9:45am
Grand Ballroom

9:50am

Keynote: Trading Simplicity for Performance When Designing Distributed Systems - Neha Narula
Predicting performance and debugging problems is critical to successfully running a distributed system in production, and a simple system is easier to reason about and understand. But for many systems, simplicity is fundamentally at odds with performance and scalability -- oftentimes, optimizing our systems to reduce latency and improve scalability requires more and more complex designs. Running complex systems at scale is even riskier because rare corner cases are guaranteed to occur.

An important example of this is scaling an application's data plane. Improving performance and fault tolerance in the data plane requires techniques like caching, replication, partitioning, and precomputation. These techniques greatly increase complexity by introducing problems like cache invalidation, rebalancing, managing staleness, and maintaining consistency.

In this talk, Neha will discuss this tension between simplicity and performance, and describe how to think about and consciously manage these trade-offs.

Speakers
avatar for Neha Narula

Neha Narula

Neha Narula just completed a PhD at the Computer Science and Artificial Intelligance Lab at MIT building fast, scalable distributed systems and databases. In a previous life she helped launch a new version of Digg and was a Senior Software Engineer at Google, where she designed the first version of Blobstore, a system for storing and serving petabytes of immutable data, and worked on Native Client, a system for running native code securely... Read More →


Thursday August 20, 2015 9:50am - 10:30am
Grand Ballroom

10:30am

Break
Thursday August 20, 2015 10:30am - 11:00am
2nd Floor Foyer

11:00am

Challenges in Optimizing Job Scheduling On Mesos - Alex Gaudio, Sailthru
As a distributed systems kernel, Mesos solves complex scheduling problems. These problems include: Bin Packing of tasks given resource constraints; Dominant Resource Allocation to prioritize Framework offers; user-defined optimistic or pessimistic Scheduler algorithms. When job allocation stops working properly, we see Framework starvation, dead locks, underutilization and other issues. In this talk, Alex Gaudio will present a unique interpretation of task scheduling on Mesos. By drawing on his experiences building Mesos Frameworks and infrastructure for Sailthru's Data Science team, he will provide attendees with ways to fundamentally question how they use Mesos, and offer practical advice to improve sub-optimal usage patterns.

Speakers
avatar for Alex Gaudio

Alex Gaudio

Senior Data Scientist and Engineer, Sailthru
Alex Gaudio is a Senior Data Scientist and Engineer at Sailthru, where he builds machine learning algorithms and distributed data pipelines on Mesos. He is the primary developer of Relay.Mesos, an intelligent auto-scaler for Mesos that functions like a thermostat. He also develops Stolos, a tool to manage task dependencies represented as a Directed Acyclic Multi-Graph using distributed queues. In his free time, Alex loves to build hardware... Read More →


Thursday August 20, 2015 11:00am - 11:40am
Willow

11:00am

Mesos at Bloomberg - Skand Gupta, Bloomberg
BVault is a massive scalable, archiving and e-discovery solution for communications and has been adopted by more than 800 enterprises, processing more than 220 million daily messages and archived more than 90 billion communication objects. The service is optimized for fast deployment of data-centric and processing-intensive applications using elastic cloud computing strategies namely, Container-based resource management (Mesos) to abstract heterogeneous data center assets as a homogeneous set of resources, prefabricated hardware in secure and geographically distributed data centers to provide on-demand capacity management, and Continuous development and integration using containers as an emerging standard in cloud infrastructure.

We share the lessons learned along the way, and tools we needed to build that enable us to validate, monitor, and operate the platform in production.

Speakers
SG

Skand Gupta

Skand Gupta is technical lead and engineering manager at Bloomberg LP. He has experience building large scale distributed systems in shared infrastructure and search space. He currently leads the Mesos infrastructure team for Bloomberg Vault.


Thursday August 20, 2015 11:00am - 11:40am
Grand Ballroom C

11:00am

Fully Fault tolerant Streaming workflows at Scale using Apache Mesos & Spark Streaming - AkhilDas, Sigmoid
Reliability, maintainability and scalability are the key concerns while designing any data intensive application . With the advent of realtime streaming platforms like Apache Spark, Storm etc, it is important that these computation frameworks adhere to all of them. One has to face a lot of challenges while implementing a realtime spark streaming pipeline in production. In this presentation we will showcase how to overcome these challenges and develop a fully, scalable fault tolerant streaming system with the help of Apache Mesos which not only makes it easier to deploy and manage the resources but also helps in handling varying data loads through dynamic resource management and allocation .

Speakers
A

AkhilDas

AkhilDas is a Software Developer at Sigmoid with focus on distributed computing, big data analytics, scaling and optimising performance. Sigmoid has worked with over 25 customers in the Big data space to get them real time insights on Tbs of data using Apache Spark and Spark Streaming. | | Previously, he had a talk on Internals of Spark and Spark Streaming at Real time data processing and cloud computing Meetup which happened in Bangalore.


Thursday August 20, 2015 11:00am - 11:40am
Grand Ballroom B

11:00am

Mesos and the Architecture of the New Datacenter - Thomas Rampelberg, Mesosphere
Apache Mesos has the ability to run on every private and cloud instance, anywhere. In this talk, Thomas Rampelberg (Datacenter Developer at Mesosphere) will trace Mesos from its origins as a “fine-grained resource manager” at UC Berkeley to becoming a full-fledged distributed kernel for resource management across the datacenter and cloud. He will explain the momentum behind the “single computer” abstraction that has put Mesos at the center of one of the most exciting architecture shifts in recent information technology history. Rampelberg will explain the role of Mesos in unifying other frameworks, share examples of how Mesos is empowering the new class of “datacenter developers” to program directly against datacenter resources, and draw correlations to how the Linux kernel revolutionized the server industry. 

Speakers
avatar for Thomas Rampelberg

Thomas Rampelberg

Thomas, one of Mesosphere’s Datacenter Developer and creator of the new Mesos CLI contributes to the open source Apache Mesos UI and has helped build other interfaces for Apache Mesos tools. Before coming to Mesosphere, Thomas was working on distributed systems at BitTorrent.


Thursday August 20, 2015 11:00am - 11:40am
Grand Ballroom D

11:50am

The New Mesos HTTP API - Vinod Kone, Twitter, Isabel Jimenez, Mesosphere
The previous Mesos API provided language bindings (a.k.a. drivers) for frameworks to interact with Mesos. Framework developers don’t like dealing with dependencies in native libraries, and with Mesos’ new HTTP API, they no longer have to! In this session, (Vinod Kone, Technical Lead for Mesos at Twitter and Isabel Jimenez, Software Engineer at Mesosphere) will provide a comprehensive walk-through of recent advancements with the Mesos API, explaining the design rationale and highlighting specific improvements that simplify writing frameworks to Mesos. 

Speakers
avatar for Isabel Jimenez

Isabel Jimenez

Distributed systems engineer, Mesosphere
Isabel Jimenez is a Software Engineer at Mesosphere.
VK

Vinod Kone

Vinod Kone is the Technical Lead for Mesos at Twitter. Vinod leads the Mesos team. Mesos is a fundamental component of Twitter's data center infrastructure powering a lot of its key services.


Thursday August 20, 2015 11:50am - 12:30pm
Willow

11:50am

Preemptive Task Scheduling in Mesos Framework - Li Jin, Two Sigma
Two Sigma uses Apache Mesos as the core of its computing infrastructure. To solve the scheduling problem of lots of resource-hungry computation tasks, we extended Dominant Resource Fairness (DRF), the core algorithm in Mesos allocator, to support preemption so the resource allocation remains fair with continuously changing demands. We will explain our extension of DRF and how it allows us to achieve very short per-user wait times in a constantly saturated Mesos cluster. We will also discuss how the core Mesos allocator itself could be enhanced to use this technique.

Speakers
LJ

Li Jin

Two Sigma LLC, Software Engineer
Li Jin is a distributed systems developer in Two Sigma. Li works on a Mesos framework that runs thousands of computing tasks every day. In his spare time, Li likes to read distributed system papers, hiking and winter sports.


Thursday August 20, 2015 11:50am - 12:30pm
Grand Ballroom C

11:50am

How to Monitor Mesos - Alexis Le-Quoc, Datadog
By providing a robust abstraction over core computing resources, Mesos does away with what has been until now the foundation of most monitoring systems: the individual host. With that gone, what should monitoring of applications running on Mesos revolve around? In this talk, Alexis Le-Quoc argues that imperative monitoring of hosts must give way to declarative monitoring, built on tags, tasks and queries. With concrete examples and live monitoring data, he will present a better way to monitor Mesos.

Speakers

Thursday August 20, 2015 11:50am - 12:30pm
Grand Ballroom B

12:30pm

Lunch
Thursday August 20, 2015 12:30pm - 2:00pm
Grand Ballroom

12:45pm

Intel Lunchtime Keynote
Thursday August 20, 2015 12:45pm - 12:52pm
Grand Ballroom

12:55pm

Mesos - EGO integration: new tools for enterprise cluster management
Our team works on integration of Mesos and IBM Platform EGO technologies. The idea is to provide Mesos users with new capabilities, such as hierarchical resource management, centralized administration, flexible reservation and sharing mechanisms, time-based policies  - and other features required for managing enterprise data centers. The integration is performed by developing a Mesos resource allocator plug-in module, that connects to a Platform EGO system - a technology with years of experience of managing enterprise server clusters in many different industries. In this talk, I'll describe the Platform EGO policy capabilities, and how we bring them to the Mesos ecosystem for plug-in deployment in the new and existing installations.

Presenter
KA

Khalid Ahmed

STSM, Chief Architect of Infrastructure Software, IBM Platform
Khalid Ahmed is an STSM, Chief Architect of Infrastructure Software at IBM Platform. He works on the design and architecture of large scale grid and cloud computing systems with focus on scheduling, resource, workload and data management. In over 20 years at industry experience he has worked in a number of roles including development, product management and architecture. His latest interests include big data systems, container technology and data... Read More →

Thursday August 20, 2015 12:55pm - 1:10pm
Grand Ballroom

1:02pm

Melting Snowflakes with microservices-infrastructure
You're building your infrastructure on Mesos, this takes time and talent. You've got the chops, but is your time worth figuring out the integration? Service discovery? Configuration management? Key management? microservices-infrastructure is the known good configuration that will melt snowflake configurations. I will talk about some challenges in integrating all the components we have, and why it's good for you.

Presenter
BH

Brian Hicks

Based in St. Louis, Brian Hicks works for Asteris LLC, a consultancy focused on providing support and services for Mesos deployments. He enjoys cats and tweeting about cats.

Thursday August 20, 2015 1:02pm - 1:07pm
Grand Ballroom

1:10pm

Mesos Frameworks and Modules for Routers and Switches to tackle Network Degradation
In order to meet Quality of Service requirements for Distributed applications on server clusters, network devices such as routers and switches have always been monoliths that are statically (over) provisioned.

With evolving support for containers and linux based application hosting on Network devices, we hook up network devices to Mesos and look at the network infrastructure as a source of network resources and telemetry data. This session will introduce QoSon, a distributed network monitoring framework and discuss its use cases with Marathon and SDN.

Presenter
AS

Akshat Sharma

Akshat Sharma is a Technical Marketing Engineer at Cisco Systems. He has 6 years of experience in the Networking industry with focus on automation, SDN, Web-scale Service Provider and Datacenter deployments and routing protocols. When he’s not busy architecting DevOps and continuous integration solutions for Routers and Switches, he spends his time learning the ropes on sys-admin tools, solving home-grown automation requirements using his... Read More →

Thursday August 20, 2015 1:10pm - 1:15pm
Grand Ballroom

1:17pm

Building Docker Images on Mesos with Portainer

 If you're running Apache Mesos & Docker, you're going to need to build Docker images. Portainer is an open source Apache Mesos framework developed at DueDil for remotely building Docker images on a cluster. In this talk Tom will explain how Portainer works, and show how DueDil uses it to build thousands of images for production and CI with zero additional capacity.



Presenter
TA

Tom Arnfeld

Tom is a Data Engineer at DueDil, a business information company based in London. He's been working with Apache Mesos for the past 18 months, developing and deploying frameworks and other data-related applications with Mesos and Docker.

Thursday August 20, 2015 1:17pm - 1:22pm
Grand Ballroom

1:25pm

Role Quotas

Quotas provide non-revocable resource guarantees without tying reservations to particular Mesos Agents. Quota is not a limit, but rather a set of resources reserved for a specific role in an entire cluster; it can be viewed as guaranteed allocation for a role. These resources cannot be hijacked by other roles and are guaranteed to be available for the role if there are enough resources in the cluster.



Presenter
AR

Alex Rukletsov

Alex Rukletsov is on the Mesosphere core team in Hamburg and focuses on open source Apache Mesos. During past years Alex was segmenting medical images and investigating behaviour of human vessels in several German research institutes. His areas of interests include distributed systems, object recognition, probabilistic and heuristic algorithms.

Thursday August 20, 2015 1:25pm - 1:30pm
Grand Ballroom

1:30pm

Ceph on Mesos

The ceph-mesos project is a Apache Mesos framework for scaling a Ceph cluster. The framework has a REST API interface to select the binaries to execute and the framework executors launch Ceph Docker containers on Mesos slaves accordingly.


In conjunction with the development of RGWFS, a HCFS over Ceph, our team is building a Big Data analytics platform on top of Apache Mesos using Ceph as its backend storage. We have developed a Apache Mesos framework to manage a Ceph cluster with all other Big Data related applications. The ceph-mesos framework enables us to deploy Ceph within 10 seconds on a freshly installed 19 node rack since the binaries are launched within a Docker container. We plan to do more testing and share performance test results and usecase scenarios.

Presenter
avatar for Zhidong Yu

Zhidong Yu

Engineering Manager, Intel Corporation

Thursday August 20, 2015 1:30pm - 1:35pm
Grand Ballroom

1:37pm

Centralized logging solution using ELK stack

There are quite a few logging solution for Docker but none of them truly fit Mesos Landscape. There isn't a centralized solution for Mesos logs.

We have developed a logging solution that is self sustainable without having to manage logging infrastructure. Lookup for the logs for any app running under Mesos is extremely fast and highly indexed. It is horizontally scalable and all the components are dockerized and are running under Mesos.



Presenter
IS

Imran Shaikh

Imran Shaikh breathes DevOps, embodies it and that is what will ooze out if you cut him. He is a Lead Systems Engineer working at YellowPages. He has an industry experience of 10+ year working with Fortune 500 companies. He has worked extensively developing, architecting and managing cloud technologies at YP as well as Yahoo. He has substantial experience running a globally distributed production environment on thousands of systems running... Read More →

Thursday August 20, 2015 1:37pm - 1:42pm
Grand Ballroom

2:00pm

Heterogeneous Resource Scheduling Using Apache Mesos for Cloud Native Frameworks - Sharma Podila, Netflix
Netflix customers stream more than two billion hours of content each month, accounting for over a third of downstream Internet traffic. At this scale, real time insights are critical to a smooth streaming experience. Sharma Podila will present an in-depth look at the technology - including the Fenzo task scheduler for Apache Mesos and Mantis, a cloud native event stream processing platform - that Netflix uses to gain real time operational insights into system behavior. The talk will focus on the design and usability of Fenzo and will provide details on how it uses CPU, memory, and network bandwidth resources for job scheduling. The presentation will showcase bin packing, task locality, resource affinity, task constraints, and autoscaling of the slave cluster. To close Sharma will talk about future directions and opportunities to collaborate with the Mesos community.

Speakers
avatar for Sharma Podila

Sharma Podila

Senior Software Engineer, Netflix
Sharma Podila is a Senior Software Engineer at Netflix, Inc. He is the author of open source Fenzo scheduling library for Mesos frameworks. His current work includes developing cloud native Mesos frameworks for Netflix project Titus, a Docker based application deployment platform, and project Mantis, a reactive stream processing platform. Sharma is looking forward to chatting about and collaborating on Mesos frameworks and schedulers.


Thursday August 20, 2015 2:00pm - 2:40pm
Willow

2:00pm

Resource Optimization with Charmander - Marcel Neuhausler, AT&T Foundry
Charmander is a closed loop data analysis and optimization environment for a container based infrastructure. Charmander is open source (https://github.com/att-innovate/charmander) and is based on Mesos, Docker, cAdvisor, InfluxDB, Apache Spark, and Redis. In this presentation, Marcel Neuhausler will showcase different use-cases for Charmander, provide a technical overview of the components, and deep dive into its own Mesos Framework. 

Speakers
MN

Marcel Neuhausler

Marcel Neuhausler is a Principal Software Engineer at the AT&T Foundry in Palo Alto focusing on resource-conscious distributed systems.


Thursday August 20, 2015 2:00pm - 2:40pm
Grand Ballroom C

2:00pm

Twitter’s Production Scale: Mesos and Aurora Operations - Joe Smith, Twitter
Twitter has used Aurora and Mesos to scale from only tens of nodes to tens of thousands. During the process of scaling up, the configuration, deployment, and operational procedures for this system have evolved and improved significantly. This talk will offer an operations perspective on the management of a Mesos+Aurora cluster, and cover many of the cluster management best practices that have evolved at Twitter from real-world production experience. It will explore methods that decrease operational overhead, as well as examples of outages and incidents to illustrate various failure domains. Furthermore, the talk will highlight current and future safeguards in Aurora and Mesos to mitigate impact from these failures in the future.

Speakers
avatar for Joe Smith

Joe Smith

SRE Tech Lead for Aurora and Mesos, Twitter
Joe Smith is a Senior Site Reliability Engineer at Twitter, and built the Aurora and Mesos cluster from tens of nodes to tens of thousands. As Mesos and Aurora SRE, he automated the build, deployment, management, repair, and maintenance of the production clusters. In addition, he spearheaded the migration of Twitter’s production services onto Aurora and Mesos from bare metal machines. He presented at SCaLE, the Reliability Meetup in... Read More →


Thursday August 20, 2015 2:00pm - 2:40pm
Grand Ballroom B

2:50pm

Supporting Stateful Services on Mesos using Persistence Primitives - Jie Yu, Twitter & Michael Park, Mesosphere
This talk will discuss the persistence primitives recently built into Mesos. Persistent primitives provide native support for running stateful services like Cassandra and MySQL in Mesos, which makes it possible for us to further consolidate workloads in the data center and improve overall cluster utilization. The goal of persistent primitives is to allow a framework to have guaranteed access to its lost state even after task failover or slave restart. To achieve that, we first discuss a primitive that allows tasks to write persistent data which won’t be garbage collected after tasks finish. Then, we discuss the primitive that allows frameworks to dynamically reserve resources for their stateful tasks upon launching so that the stateful tasks can always be re-launched on the same slave that contains their persisted state.

Speakers
avatar for Michael Park

Michael Park

Distributed Systems Engineer, Mesosphere
I’m a committer for the Apache Mesos project, and work as a Distributed Systems Engineer at Mesosphere. Within the realm of computer science, I’m very much intrigued by language design, compiler construction, and distributed systems. I’m also an active member of the ISO C++ Standards Committee.
JY

Jie Yu

Software Engineer, Mesosphere
Jie Yu is a Software Engineer at Mesosphere, Inc, focused on Mesos development. He has worked on several key components in Mesos including containerization, persistence primitives and replicated logs. Before joining Mesosphere, he was a software engineer at Twitter. Jie obtained his PhD in Computer Science and Engineering from the University of Michigan where he conducted research for concurrent and event-driven systems.


Thursday August 20, 2015 2:50pm - 3:30pm
Willow

2:50pm

Mesos at OpenTable - Pablo Delgado
Opentable has been using Apache Mesos for production workloads and for running critical parts of their production services for more than a year. Not only did Mesos help deploying resilient / elastic standalone applications and services , but also the distributed / fault-tolerant frameworks like Apache Spark for Data processing and machine learning. Mesos enabled Opentable to run multiple distributed applications across the same infrastructure at scale. Pablo will tell the story of how Opentable started with Mesos, the pain points of dealing with an hybrid Mesos + non-Mesos environment and how to survive in the transition.

Speakers
PD

Pablo Delgado

Senior Data Engineer, OpenTable, Inc.


Thursday August 20, 2015 2:50pm - 3:30pm
Grand Ballroom D

2:50pm

Scaling at eBay with Mesos - Soma Shekar Oruganti, eBay
eBay Inc. has embraced Jenkins for it’s continuous integration and delivery processes. It was challenging to efficiently use the infrastructure resources to accommodate the continuous integration process at the scale of eBay. Apache Mess and Marathon have made it possible for eBay to build a highly scalable, available and self healing processes, with efficient utilization of resources. Along the journey, eBay had challenges and overcome them effectively, and contributed back the solutions to the open source community with contributions to Jenkins framework for Mesos.

Speakers
SS

Soma Shekar Oruganti

MTS 1, Paypal Inc
Soma Shekar works as principal software engineer with Global Platform and Infrastructure Team at eBay Inc. The team is responsible for the tools used by eBay Inc.(both eBay and Paypal) developer and QA community for their Continuous Integration and Delivery processes. eBay have embraced Jenkins, Mesos, Marathon and Aurora to accommodate for the Continuous Integration/Delivery processes at scale. eBay adopted the technologies overcoming challenges... Read More →



Thursday August 20, 2015 2:50pm - 3:30pm
Grand Ballroom B

2:50pm

Crossing the Streams: Leveraging Kubernetes Services From Other Mesos Frameworks and Vice Versa - Timothy St. Clair, Red Hat
Kubernetes provides a declarative model in order to describe micro-services. However, auto-discovering those services outside of kubernetes requires add-ons. In this presentation we will explore how using the kubernetes framework and Mesos-DNS we can bridge the gap, and provide cluster wide service discovery.

Speakers
avatar for Timothy St. Clair

Timothy St. Clair

Principal Engineer, Red Hat
Timothy St. Clair is a Principal Software Engineer at Red Hat in the Emerging Technology office. He is a PMC member and committer to the Apache Mesos Project, and has worked on the development and integration of various open source distributed systems projects, including: Mesos, Kubernetes, Tachyon, Condor, and Hadoop.


Thursday August 20, 2015 2:50pm - 3:30pm
Grand Ballroom C

3:30pm

Break
Thursday August 20, 2015 3:30pm - 4:00pm
2nd Floor Foyer

4:00pm

Mesos Gets Pluggable: Introducing Mesos Modules - Kapil Arya, Mesosphere & Niklas Nielsen, Mesosphere
One of the most exciting new innovations is a simple, pluggable approach to building customizations for Mesos. This makes it as easy to extend or customize Mesos as it is to install plug-ins into your web browser. In this talk, (Niklas Nielsen and Kapil Arya, Distributed Systems Engineers at Mesosphere) will provide a technical introduction to Mesos Modules - the new way to customize Mesos without having to recompile / relink for each specific use case. They will demonstrate how to build and use Mesos Modules, including an example of how to write an allocator module for custom scheduling policies across a cluster. The presenters will also explain how Mesos Modules will help grow the ecosystem of developers building new features for Mesos. Customization will be one of strongest and most flexible attributes of the Mesos distributed systems kernel!

Speakers
avatar for Kapil Arya

Kapil Arya

Kapil Arya is an Engineer at Mesosphere focussing on the core team. He recently finished his PhD at Northeastern University where he was the lead developer of the open source distributed checkpointing project DMTCP, and contributed to the reversible debugger FReD. Interning at VMware for several summers, Kapil contributed to solving the double-paging problem.
NN

Niklas Nielsen

Niklas Nielsen is an engineer at Mesosphere. He is a C++ hacker and is works on the Mesos core. Before joining Mesosphere he worked at Adobe on the Flash VM and Google V8, as well as a secret next-generation VM. He did his Master's thesis at Lawrence Livermore National Lab, working on debugging tools for future exascale systems.


Thursday August 20, 2015 4:00pm - 4:40pm
Willow

4:00pm

Apache Cotton: MySQL on Mesos - Yan Xu, Twitter
Apache Cotton (previously named Mysos) is an Apache Mesos framework for running MySQL instances. It dramatically simplifies the management of a MySQL cluster and is designed to offer:
- Efficient hardware utilization through multi-tenancy (in performance-isolated containers)- High reliability through preserving the MySQL state during failure and automatic backing up to/restoring from HDFS- An automated self-service option for bringing up new MySQL clusters- High availability through automatic MySQL master failover- An elastic solution that allows users to easily scale up and down a MySQL cluster by changing the number of slave instances
We’ll share our experience developing and using this framework.

Speakers
avatar for Yan Xu

Yan Xu

Apple, Software Engineer
Jiang Yan Xu is a software engineer at Twitter and has been working on Mesos for the past two years. Before joining Twitter he studied Computer Science at the University of Florida and conducted PhD research on cloud computing and P2P networking.


Thursday August 20, 2015 4:00pm - 4:40pm
Grand Ballroom C

4:00pm

Running Stateful Services with Mesos - Arunabha Ghosh, Moz & Ankan Mukherjee, Moz
The Mesos system allows services to be decoupled from machines, however this decoupling creates problems for legacy apps that rely on persistent state. Traditional SQL databases are a prime example of such apps. It is still possible, however to run such apps under Mesos and gain the operational advantages it provides. We will cover techniques used at Moz to successfully run SQL databases and other legacy persistent state services under Mesos. The presentation will also cover challenges, best practices and a look at how to leverage Mesos primitives for stateful services. We will demonstrate several such services running on Mesos.

Speakers
AG

Arunabha Ghosh

Moz
Arunabha Ghosh is the Director of Engineering at Moz where among other things he leads the effort to move Moz onto Mesos. Prior to Moz, Arunabha worked at Yahoo and Google focusing on building large scale infrastructure. Arunabha leads the Systems Research group at the HackerDojo in the Bay Area.
AM

Ankan Mukherjee

Ankan Mukherjee is a senior engineer at Moz and is currently focussed on building the next generation cluster operating system for Moz's datacenters. Prior to working at Moz he donned many different roles in the enterprise software world - software engineer, technical architect, technical/project manager to name a few.


Thursday August 20, 2015 4:00pm - 4:40pm
Grand Ballroom B

4:00pm

Developing Frameworks for Apache Mesos - Joe Stein, Big Data Open Source Security
Using schedulers like Marathon and Aurora help to get your applications scheduled and executing on Mesos. In many cases it makes sense to build a framework and integrate directly. This talk will breakdown what is involved in building a framework, how to-do this with examples and why you would want to-do this. Frameworks are not only for generally available software applications (like Kafka, HDFS, Spark ,etc) but can also be used for custom internal R&D built software applications too.

Speakers
JS

Joe Stein

Big Data Open Source Security LLC, Big Data Open Source Security LLC
Joe Stein is an Apache Kafka committer and PMC member. A frequent speaker also on both Hadoop and Cassandra, Joe is the Founder and Principal Architect of Big Data Open Source Security LLC http://stealth.ly a professional services and product solutions company. Joe has been a distributed systems developer and architect for over 12 years now having built backend systems that supported over one hundred million unique mobile devices a day processing... Read More →


Thursday August 20, 2015 4:00pm - 4:40pm
Grand Ballroom D

4:50pm

Building A Machine Learning Platform to Predict User Behavior on Mesos - Jeremy Stanley, Sailthru
In 2014 Sailthru launched a new machine learning platform, named Sightlines, using Mesos. This platform predicts the behavior of hundreds of millions of users for hundreds of clients. In this talk, Jeremy Stanley and Alex Gaudio will tell the story of how (and why) this platform was built on Mesos, and outline some of the key challenges and detours taken along the way. This will include an overview of the goals and architecture of the system, and an account of the 10 month effort to develop, scale and stabilize the system for production.

Topics covered will include our use of AWS spot instances and Netflix’s Asgard to save costs, Marathon and Zookeeper to schedule jobs, detours with Chronos, Spark and Redis as our architecture evolved, and challenges we faced with maximizing resource utilization and ensuring we had identical and isolated test and production environments.

Speakers
JS

Jeremy Stanley

"Jeremy Stanley is the Chief Data Scientist & EVP of Engineering at Sailthru, where he is focused on building data-driven solutions for marketers that drive long-term customer engagement and optimize revenue opportunities. | | Prior to Sailthru, Jeremy was the CTO at Collective where he led a team of data scientists, product managers and engineers in creating technology platforms that used machine learning and big data to solve digital... Read More →


Thursday August 20, 2015 4:50pm - 5:30pm
Grand Ballroom B

4:50pm

Mesos Networking - Christos Kozyrakis, Mesosphere and Spike Curtis, Project Calico
Because Mesos abstracts the physical location of containers, network connectivity, service discovery, and network security can be quite challenging. In this talk, we will review the networking features necessary to make Mesos practical in large scale, multi-tenant clusters. Specifically, we will describe how to assign a separate IP address per container, how to isolate containers on demand with a distributed firewall, and how to automate service discovery by integrating Mesos with DNS. We will demonstrate the first implementation of these features that uses Project Calico, a pure layer-3 approach to virtual networking for scalable datacenter.

Speakers
SC

Spike Curtis

Spike Curtis is a Core Developer and Evangelist at Project Calico. He has been a developer in advanced network technologies since 2010. Prior to his work in networks he was a researcher in experimental quantum computing at the University of Oxford.
CK

Christos Kozyrakis

Mesosphere
Christos Kozyrakis is the Director of Research at Mesosphere. He focuses on automating resource management and raising efficiency in Mesosphere clusters. Christos has over twenty years of research experience on hardware architectures, runtime management environments, and system software. His recent research focused on resource-efficient cloud computing, using machine-learning and dynamic control techniques to rightsize cloud workloads, increase... Read More →


Thursday August 20, 2015 4:50pm - 5:30pm
Grand Ballroom D

4:50pm

Per Container Network Monitoring and Isolation in Mesos - Jie Yu, Twitter
This talk will discuss the per container network monitoring and isolation feature introduced in Mesos 0.21.0 and our experiences in using it at Twitter. With this feature enabled, each container is assigned a separate network stack which isolates the network traffic from other containers and allows operators and users to easily get its network statistics. This greatly simplifies the network monitoring, health checking and troubleshooting for jobs, and provides a more fairly shared and predictable network to users. This feature is completely transparent to tasks, and does not require changes to the underlying network infrastructure. To achieve that, the key idea is to let containers share the same host IP and use ports to multiplex network packets. We will show you the implications of this approach and the lessons we learned during the deployment and ongoing use of this feature.

Speakers
JY

Jie Yu

Software Engineer, Mesosphere
Jie Yu is a Software Engineer at Mesosphere, Inc, focused on Mesos development. He has worked on several key components in Mesos including containerization, persistence primitives and replicated logs. Before joining Mesosphere, he was a software engineer at Twitter. Jie obtained his PhD in Computer Science and Engineering from the University of Michigan where he conducted research for concurrent and event-driven systems.


Thursday August 20, 2015 4:50pm - 5:30pm
Willow

4:50pm

Distributed CI: Scaling Jenkins on Mesos and Marathon - Roger Ignazio, Puppet Labs
As engineering teams and product offerings expand, monolithic Jenkins instances grow and become single points of failure. By running Jenkins on Mesos and Marathon, masters can be scaled horizontally and executors provisioned dynamically as demand increases. This presentation will demonstrate that running Jenkins at-scale is more feasible than ever. The architecture provided by Mesos moves Continuous Integration (CI) from being synonymous with Jenkins to Jenkins being a component of the overall CI system.

This presentation will cover:
- Conventional scaling techniques and their shortcomings
- Scaling Jenkins masters and slaves horizontally on Mesos and Marathon
- Dynamically creating and executing jobs
- Using Elasticsearch, Logstash, and Kibana (ELK) to store, query, and visualize run data

Speakers
avatar for Roger Ignazio

Roger Ignazio

Tech Lead, Mesosphere
Roger Ignazio is a tech lead at Mesosphere and the author of "Mesos in Action." He has spoken at several conferences, including MesosCon, ContainerCon, PuppetConf, and Velocity. His professional interests include distributed systems, fault tolerance, and great user experience. In his free time, he enjoys photography and the outdoors. He lives in Portland, Oregon with his wife Sarah and their two cats.


Thursday August 20, 2015 4:50pm - 5:30pm
Grand Ballroom C

8:00pm

Attendee Reception at the Experience Music Project Museum
Open to all #MesosCon attendees, join us at the Experience Music Project (EMP), a leading-edge nonprofit museum, dedicated to the ideas and risk-taking that fuel contemporary popular culture. Appetizers and drinks included as well as the opportunity to check out some of the current exhibits. A shuttle will be provided to/from the Sheraton.

Thursday August 20, 2015 8:00pm - 10:30pm
Experience Music Project
 
Friday, August 21
 

8:00am

9:00am

Announcements
Friday August 21, 2015 9:00am - 9:05am
Grand Ballroom

9:05am

Keynote: How we run Siri - Robert Lacroix & Brian Sumner, Apple
Robert Lacroix & Brian Sumner discuss "How we run Siri."


Friday August 21, 2015 9:05am - 9:20am
Grand Ballroom

9:20am

Keynote: Growing the Mesos Ecosystem - Florian Leibert, CEO at Mesosphere
Details coming soon.

Moderators
FL

Florian Leibert

Florian is CEO at Mesosphere.

Friday August 21, 2015 9:20am - 9:30am
Grand Ballroom

9:35am

Keynote: Silence is Golden: Coordination-Avoiding Systems Design - Peter Bailis
Computer networks make it difficult to design scalable, robust distributed systems that exhibit good performance. Networks can be slow, have limited capacity, and are often unreliable. In an ideal world, we'd build systems that don't rely on the network at all. Unfortunately, as a slew of negative results like the CAP Theorem illustrate, this isn't always possible. Traditional systems abstractions like ACID transactions fundamentally require synchronous communication, or coordination, to implement. As a result, coordination-free systems designs often forego many programmer-friendly abstractions. These systems leave the task of reasoning about correctness to the application developer or, worse, to the end user.
In this talk, I'll discuss an alternative: system designs that coordinate only when necessary to guarantee application correctness. This coordination avoidance maximizes scalability and robustness by minimizing reliance on the network. To illustrate the power of coordination-avoiding systems design, I'll present several case studies from our research spanning database isolation guarantees, indexes and constraints, and open source applications. Perhaps surprisingly, even though traditional implementations of these tasks rely on coordination, many of these tasks don't actually require coordination for correctness. The resulting systems are among the fastest prototypes ever built and operated at scale. Based on these case studies, I'll provide concrete and practical design principles for reasoning about and applying coordination avoidance in the wild.

Speakers
avatar for Peter Bailis

Peter Bailis

Peter Bailis is a final-year Ph.D. candidate at UC Berkeley working in databases and distributed systems. As part of his dissertation work, he has studied and built high performance distributed data management systems for large scale transaction processing, data serving, and statistical analytics. He is the recipient of the NSF Graduate Research Fellowship, the Berkeley Fellowship for Graduate Study, and best-of-conference citations for... Read More →


Friday August 21, 2015 9:35am - 10:15am
Grand Ballroom

10:10am

Break
Friday August 21, 2015 10:10am - 10:40am
2nd Floor Foyer

10:40am

Myriad: Resource Sharing Beyond Boundaries - Mohit Soni, PayPal
Last year at MesosCon, eBay presented a novel solution which allows Mesos and Apache YARN/Hadoop to co-exist without statically partitioning the data center. That solution evolved into a community project called "Myriad" and is under active development. In this talk, we will present the latest architecture of "Myriad", how it improves overall resource utilization in the datacenter, and how it enables dynamic allocation of resources between workloads running on Mesos and YARN.

Speakers
SM

Santosh Marella

Santosh Marella is a Software Engineer at MapR Technologies. Since joining MapR, Santosh worked on numerous projects starting from MapR's admin infrastructure, MapReduce and YARN. He currently leads MapR's efforts on Myriad. Prior to MapR, Santosh was a Software Engineer at Google where he worked on enterprise and CRM solutions.
MS

Mohit Soni

Software Engineer, PayPal
Mohit works as an engineer in platform team at eBay Inc. His job is to focus on maximizing efficiency, increasing agility, and reducing cost. Mohit presented in past at DockerCon 2014, Hadoop India Summit 2011 and BarCamp 2010. You can follow Mohit as mohitsoni on GitHub and Twitter.


Friday August 21, 2015 10:40am - 11:20am
Grand Ballroom C

10:40am

Save Millions by Efficient Resource Utilization Through Mesos - Smarth Madan, PayPal
When it comes to hardware, most companies underutilize it; sometimes letting it lying idle for days without activity. When we started the project to build a staging environment for PayPal, we had the opportunity to design an environment, which is dynamic, on demand, and highly available. Through Mesos and the Aurora scheduler, we are able to deploy 200+ VMs simultaneously and create a mini PayPal environment for developers to connect from their VMs, desktops or mobile devices. This presentation will cover how we are reducing operational cost of hardware and building a scalable environment, which is improving the efficiency for PayPal developers. 

Speakers
SM

Smarth Madan

Smarth has been developing software at PayPal for the past 3 years. He is currently working on a project to build a staging environment for PayPal. He completed his MS in Software Engineering from San Jose State University and has been working in IT for more than 5 years now. He is passionate about technology and explores new programming languages in his spare time. Also he keeps a close watch on all the latest gadgets. He has no previous... Read More →


Friday August 21, 2015 10:40am - 11:20am
Grand Ballroom D

10:40am

Global Control of Decentralized Mesos Clusters - Daniel Giribet, Schibsted
Apache Mesos and the frameworks running on top of them are a great resource management solution but they lack some features to make it suitable for automatic scaling and communication of microservices that run across different locations and vendors. In this presentation Schibsted will discuss these limitations and present their plan for a Global Scheduler capable of controlling independent Mesos clusters across multiple distant datacenters and service providers.

Speakers
avatar for Daniel Giribet

Daniel Giribet

Daniel Giribet is the Infrastructure Platform Development leader at Schibsted Products and Technology. He holds a Computer Science degree and has been focusing on video processing, web engineering, systems architecture and content management. Has worked in indie projects and also worked on global stuff at companies like Schibsted and the BBC. Some of his current projects at Schibsted include Mesos stuff, Agile processes & infrastructure... Read More →


Friday August 21, 2015 10:40am - 11:20am
Grand Ballroom B

10:40am

Securing your Mesos Cluster - Adam Bordelon, Mesosphere & Joris Van Remoortere, Mesosphere
Security features are in high demand as organizations increasingly trust their Mesos clusters with critical production workloads and sensitive data, while at the same time providing cluster access to naive/untrusted users. The same questions keep coming up:
  • How can Mesos prevent unauthorized users from intercepting sensitive data or forging communications?
  • How can Mesos prevent unauthorized frameworks from launching malicious jobs and prevent unauthorized slaves from stealing tasks and data?
  • How can Mesos ensure users and frameworks can only access their own tasks/data?
In this first ever MesosCon security deep-dive, Adam and Joris will walk through the best practices and latest advances in securing Mesos clusters, including encryption, authentication, and authorization features.

Speakers
AB

Adam Bordelon

Mesosphere, Distributed System Engineer
Adam Bordelon is a distributed systems architect at Mesosphere and an Apache Mesos committer. Before joining Mesosphere, Adam lead development on Hadoop core at MapR, built distributed systems for recommendations at Amazon, and re-architected the LabVIEW compiler at National Instruments. He completed his Master’s degree at Rice University, building a tool to analyze supercomputer performance data for bottlenecks and anomalies.
JV

Joris Van Remoortere

Joris Van Remoortere is a Senior Software Engineer at Mesosphere. He focuses on large-scale, low-latency, high-throughput systems. He alleviates scaling issues and makes performance and reliability improvements on many projects, including Orly, messaging and payment systems, web tiers, Hadoop, and Kafka.


Friday August 21, 2015 10:40am - 11:30am
Willow

11:30am

Generalizing Software Deployment - The Many Meanings of "Update" - Bill Farner, Twitter
Shipping code is an everyday part of the life of a developer. And when shipping code in a Mesos cluster, that workflow is managed by individual schedulers. In the case of Apache Aurora, developers are empowered by a streamlined deployment mechanism. This talk will cover the evolution of how Aurora manages deployments, and describe some of the challenges imposed by the wide variance in requirements. Finally, there will be discussion of how deployments on Aurora currently work for major services at Twitter.

Speakers
BF

Bill Farner

Bill Farner has been a Software Engineer at Twitter for the last five years, serving as the Technical Lead on Aurora since its inception four years ago. He previously worked at Google on Production Engineering and storage infrastructure. Bill holds a MS in Computer Engineering from the Rochester Institute of Technology.


Friday August 21, 2015 11:30am - 12:10pm
Grand Ballroom C

11:30am

Migrating over 1,000 Production Components to a Mesos-Based Platform-as-a-Service (PaaS) - Tom Petr, HubSpot
At HubSpot we built a turn-key PaaS system our engineers use to deploy all the different parts that make up the HubSpot application. In this talk we'll discuss the benefits of using Mesos as a starting point for deploying a PaaS, the open source framework we created called Singularity, and the lessons learned from moving our entire production application to this stack. We will enumerate many of the specific benefits that became apparent after the system was in place, including enhanced security, reliability, operational simplicity, and cost efficiency. Lastly, we will shed light on both the ways in which we extended what Mesos offered in order to empower our product developers, and how we moved over 100 software load balancers into Mesos to make our infrastructure team more efficient.

Speakers
TP

Tom Petr

HubSpot, Tech Lead
Tom Petr is an infrastructure tech lead at HubSpot and contributor to the Singularity Mesos Framework. He's previously given talks about Singularity, Mesos, and HubSpot's deployment stack at Twitter, Wistia, internally at HubSpot, and various tech meetups in Boston.


Friday August 21, 2015 11:30am - 12:10pm
Grand Ballroom B

11:30am

Enabling Hybrid Workloads with Docker/Mesos at Orbitz - Steve Hoffman, Orbitz
The promise of Mesos to create a datacenter-centric resource pool is a great idea. However, a decade old company with many different types of existing systems and unique workloads can't simply mass migrate to Mesos. That doesn't mean Mesos can't pay a huge part in unifying disparate workloads that construct existing systems while opening the door to new Mesos-centric development.

In this presentation, Steve will cover 3 very different use cases that are being unified under Apache Mesos including: Automated builds of existing code artifacts, automated builds of AWS AMIs via packer in EC2, and rolling deploys & supervision of Docker microservices across environments. The addition of Docker on Mesos has enabled this, but that doesn't mean that all Mesos slaves should be identical. See how the addition of localized support services on slaves can compliment your Docker on Mesos use cases.

Speakers
avatar for Steve Hoffman

Steve Hoffman

Senior Principal Software Engineer, Orbitz


Friday August 21, 2015 11:30am - 12:10pm
Grand Ballroom D

11:30am

Rocket and the Need for the App Container Specification - Brandon Philips, CoreOS
Rocket is a simple daemon-free tool that enables users to run containerized apps on their systems free of host dependencies. Containers running under rocket execute like regular processes and can be managed using existing process management tools like upstart, systemd, runit and etc.

Rocket is also an implementation of the “App Container spec,” which defines how to define and build containerized applications using tooling like tar and PGP, and then host these files easily using standard protocols like HTTP. The goal of the spec is to enable independent and creative implementations of container runtimes and build tools.

This presentation will give an overview of the App Container spec and why the spec is important to the Mesos community. This session invites attendees to learn how they work and how to get involved.


Speakers
BP

Brandon Philips

Brandon Philips is the CTO of CoreOS where he is helping to build modern Linux server infrastructure. Prior to CoreOS, he worked at Rackspace hacking on cloud monitoring and was a Linux kernel developer at SUSE. As a graduate of Oregon State's Open Source Lab he is passionate about open source technologies.


Friday August 21, 2015 11:30am - 12:10pm
Willow

12:10pm

Lunch
Friday August 21, 2015 12:10pm - 1:30pm
Grand Ballroom

1:30pm

Tactical Mesos: How Internet-Scale Ad Bidding Works on Mesos / Aurora - Dobromir Montauk, TellApart
Real Time Bidding on the large Internet exchanges (Doubleclick, Facebook, etc) requires large-scale, low-latency serving systems: >100K QPS at peak with <100ms tail response times. Time is very literally money. Dobromir will present TellApart's full stack in excruciating detail (if you want it), which includes Mesos/Aurora, ZK service discovery, Finagle-Mux RPC, and a Lambda architecture with Voldemort as the serving layer.

Speakers
DM

Dobromir Montauk

Software Engineer, Twitter
Nine years at Google, including work on Google+ Stream backend and Search Infrastructure. Recently joined TellApart as Uber Tech Lead working on infrastructure. In charge of TellApart bidding platform performance: >100K QPS with sub-10ms latencies. Optimized the Bidders to run at 250 QPS/core and migrated bidding platform to Mesos. | | Presentation at... Read More →



Friday August 21, 2015 1:30pm - 2:10pm
Grand Ballroom B

1:30pm

Satellite: A Monitoring Application for Mesos - Sunil Abraham, Two Sigma Investments
Satellite is a monitoring and alerting solution for Mesos. Satellite’s powerful event streams processing engine allows /ops teams to build custom monitoring and alerting solutions that will improve uptime and reliability of the underlying compute resources in the cluster. In this talk, you'll learn how Two Sigma is using Satellite to monitor and take action on the hosts in our Mesos cluster. After a quick introduction to the architecture of Satellite, we'll walk through the process of defining a rule, adding a Pagerduty alert, and programmatically taking action on the state of the cluster by enabling and disabling hosts in the cluster.

Speakers
SA

Sunil Abraham

Sunil Abraham is a developer at Two Sigma Investments. Previously, Sunil was a Fulbright Scholar at the University of Waterloo working on public health and studied math and statistics at Oxford and Cambridge.


Friday August 21, 2015 1:30pm - 2:10pm
Willow

1:30pm

Docker Swarm on Mesos - Tim Chen, Mesosphere & Victor Vieux, Docker
Docker has become a popular way for developers to package their applications for deployment, but as applications get more complex there is a growing need to deploy them at scale, orchestrating them across dozens or thousands of machines. With Docker Swarm, you can talk to Mesos using the Docker command line client and start Docker containers on your Mesos cluster as easily as you would do it on your development laptop. In this presentation, Victor Vieux (Docker Engineer, Docker) and Timothy Chen (Mesosphere Engineer, Mesosphere) will explain why Mesos is the “gold standard” for running Docker containers at scale and will demo how to setup Docker Swarm on Mesos (using DCOS) and how to do a `docker run` against your Mesos cluster easily.

Speakers
TC

Tim Chen

Tim Chen is on the Mesosphere core team and focuses on containerization and big data frameworks. He is also a PMC/committer on Apache Drill and Apache Mesos, and contributed to other open source projects such as Spark, Kafka, Hive. Before joining Mesosphere, Tim past experiences includes working on data services on Halo, CloudFoundry (PaaS) and search engines.
avatar for Victor Vieux

Victor Vieux

Docker Core Engineer, Docker, Inc
Victor Vieux works as a software engineer at Docker, Inc. He was one of the few guys working on the Docker internals, he's now working on Docker Swarm, a Docker-native clustering system. Originally from Paris, France, where he was famous for his work on robots (programming them in C/C++), Victor joined the company and project at the very beginning. Since then he's been helping both on programming in Go and helping the Docker community, 24/7, with... Read More →


Friday August 21, 2015 1:30pm - 2:10pm
Grand Ballroom C

1:30pm

Workshop: A Field Guide to Mesos Frameworks - Adam Bordelon, Mesosphere; Elizabeth Lingg, Mesosphere; Niklas Nielsen, Mesosphere
Frameworks are the lifeblood of Mesos, since they provide the distributed applications and services that run on top of Mesos. Many users are perfectly content using the various frameworks already built for Mesos, but others may need support for a new application or require custom behavior that is not already available. In this workshop, experts from Mesosphere will explain why, when, and how you should build your own framework. In addition to the basics of resource offers and status updates, they will dive into “enterprise-grade” requirements like high availability, authentication, advanced scheduling, and persistent state. Whether you are building your own custom framework or just getting to know the existing Mesos frameworks, this workshop will give you the tools and knowledge you need. 

Speakers
AB

Adam Bordelon

Mesosphere, Distributed System Engineer
Adam Bordelon is a distributed systems architect at Mesosphere and an Apache Mesos committer. Before joining Mesosphere, Adam lead development on Hadoop core at MapR, built distributed systems for recommendations at Amazon, and re-architected the LabVIEW compiler at National Instruments. He completed his Master’s degree at Rice University, building a tool to analyze supercomputer performance data for bottlenecks and anomalies.
EL

Elizabeth Lingg

Elizabeth Lingg a Technical Lead Software Engineer at Mesosphere and applies her interest in algorithms and A.I. to distributed systems. Prior to Mesosphere, she worked on reasoning engines at Oracle, using Apache Lucene, Open NLP, WikiDb, various graph traversal algorithms, WordNet, machine learning algorithms, and semantic web. | Niklas Nielsen is an engineer at Mesosphere. He is a C++ hacker and is works on the Mesos core. Before joining... Read More →
NN

Niklas Nielsen

Niklas Nielsen is an engineer at Mesosphere. He is a C++ hacker and is works on the Mesos core. Before joining Mesosphere he worked at Adobe on the Flash VM and Google V8, as well as a secret next-generation VM. He did his Master's thesis at Lawrence Livermore National Lab, working on debugging tools for future exascale systems.


Friday August 21, 2015 1:30pm - 3:00pm
Grand Ballroom D

2:20pm

Simplifying Maintenance with Mesos - Benjamin Mahler, Twitter
You have computing resources that your developers want to leverage. However, providing developers with direct access to machines would set you up for an operational nightmare: a critical security vulnerability comes along and now you need to reboot all of your kernels to apply the fix, without any downtime.. good luck!

Mesos is a “cluster management” layer that provides developers with access to computing resources while encouraging behavior that leads to simplified maintenance. This talk will address the challenges of maintenance when dealing with Mesos clusters running multiple frameworks (e.g. services, storage, batch compute). We’ll explore a current proposal for adding simple and flexible maintenance primitives in Mesos to address these concerns and enable tooling for automated maintenance.

Speakers
avatar for Benjamin Mahler

Benjamin Mahler

Senior Software Engineer, Mesosphere
Benjamin Mahler is a Software Engineer at Mesosphere. Previously, he worked on Mesos at Twitter on some of the largest Mesos deployments in the world. Benjamin is a committer and PMC member of the Mesos project. He studied Software Engineering at the University of Waterloo and is passionate about software craftsmanship and sustainability.



Friday August 21, 2015 2:20pm - 3:00pm
Grand Ballroom B

2:20pm

Scaling a Highly-Available Scheduler Using the Mesos Replicated Log: Pitfalls and Lessons Learned - Kevin Sweeney, Twitter
Mesos frameworks can go from prototype to production at scale very quickly. As your users realize how easy it is to spin up tasks, they will start doing so gratefully and rapidly. This talk will arm you with techniques for writing a framework scheduler for a large-scale Mesos cluster using Apache Aurora as a case study. Aurora routinely manages 100s of thousands of tasks per Mesos cluster. To do this it maintains the state of each task and quickly reacts to changes in the cluster. To talk to Mesos Aurora relies on the Mesos JNI library. For high-availability support it relies on the Mesos replicated log for durable storage. As its clusters grew, Aurora pushed against the limits of these libraries. This talk will explore the tools the Aurora scheduler has used to meet these challenges, including Apache Thrift for schema management, and the importance of normalization.

Speakers
avatar for Kevin Sweeney

Kevin Sweeney

Software Engineer - Cloud Infrastructure, Twitter
"Kevin Sweeney is a software engineer at Twitter and committer on Apache Aurora, working on it since 2012. You can find him on Twitter as @kts.



Friday August 21, 2015 2:20pm - 3:00pm
Grand Ballroom C

2:20pm

The Scheduler Meets the Network - Arunabha Ghosh, Moz
A vital part of datacenters is the network which enables communications. As the size
and complexity of applications deployed on Mesos grows, close integration with the underlying network will become crucial. Traditionally, networks have been statically configured, but the recent rise of network virtualization combined with techniques like overlay networks have enabled unprecedented network programability and control.

Mesos provides an ideal platform to leverage the dynamic capabilities of modern computer networks due to the scheduler’s complete knowledge and control over jobs in the system. The talk will showcase several features enabled by such integration such as IP per job, programmable network isolation, easy service discovery etc. The talk will conclude with a look towards the future evolution of networks and tighter integration with Mesos.

Speakers
AG

Arunabha Ghosh

Moz
Arunabha Ghosh is the Director of Engineering at Moz where among other things he leads the effort to move Moz onto Mesos. Prior to Moz, Arunabha worked at Yahoo and Google focusing on building large scale infrastructure. Arunabha leads the Systems Research group at the HackerDojo in the Bay Area.


Friday August 21, 2015 2:20pm - 3:00pm
Willow

3:00pm

Break
Friday August 21, 2015 3:00pm - 3:30pm
2nd Floor Foyer

3:30pm

Keynote: Cloud Trends, DevOps and Microservices - Adrian Cockcroft - Technology Fellow, Battery Ventures
What has happened in the last year, and where are we heading next?

Speakers
avatar for Adrian Cockcroft

Adrian Cockcroft

Technology Fellow, Battery Ventures
Adrian Cockcroft has had a long career working at the leading edge of technology. He’s always been fascinated by what comes next, and he writes and speaks extensively on a range of subjects. At Battery, he advises the firm and its portfolio companies about technology issues and also assists with deal sourcing and due diligence. Before joining Battery, Adrian helped lead Netflix’s migration to a large scale, highly available... Read More →


Friday August 21, 2015 3:30pm - 4:10pm
Grand Ballroom

4:15pm