Loading…
#MesosCon 2015 has ended
Thursday, August 20 • 11:50am - 12:30pm
Preemptive Task Scheduling in Mesos Framework - Li Jin, Two Sigma

Sign up or log in to save this to your schedule and see who's attending!

Two Sigma uses Apache Mesos as the core of its computing infrastructure. To solve the scheduling problem of lots of resource-hungry computation tasks, we extended Dominant Resource Fairness (DRF), the core algorithm in Mesos allocator, to support preemption so the resource allocation remains fair with continuously changing demands. We will explain our extension of DRF and how it allows us to achieve very short per-user wait times in a constantly saturated Mesos cluster. We will also discuss how the core Mesos allocator itself could be enhanced to use this technique.

Speakers
LJ

Li Jin

Two Sigma LLC, Software Engineer
Li Jin is a distributed systems developer in Two Sigma. Li works on a Mesos framework that runs thousands of computing tasks every day. In his spare time, Li likes to read distributed system papers, hiking and winter sports.


Thursday August 20, 2015 11:50am - 12:30pm
Grand Ballroom C

Attendees (0)