Loading…
This event has ended. View the official site or create your own event → Check it out
This event has ended. Create your own
View analytic
Thursday, August 20 • 11:50am - 12:30pm
Preemptive Task Scheduling in Mesos Framework - Li Jin, Two Sigma

Sign up or log in to save this to your schedule and see who's attending!

Two Sigma uses Apache Mesos as the core of its computing infrastructure. To solve the scheduling problem of lots of resource-hungry computation tasks, we extended Dominant Resource Fairness (DRF), the core algorithm in Mesos allocator, to support preemption so the resource allocation remains fair with continuously changing demands. We will explain our extension of DRF and how it allows us to achieve very short per-user wait times in a constantly saturated Mesos cluster. We will also discuss how the core Mesos allocator itself could be enhanced to use this technique.

Speakers
LJ

Li Jin

Two Sigma LLC, Software Engineer
Li Jin is a distributed systems developer in Two Sigma. Li works on a Mesos framework that runs thousands of computing tasks every day. In his spare time, Li likes to read distributed system papers, hiking and winter sports.


Thursday August 20, 2015 11:50am - 12:30pm
Grand Ballroom C

Attendees (27)