#MesosCon 2015

In 2014 Sailthru launched a new machine learning platform, named Sightlines, using Mesos. This platform predicts the behavior of hundreds of millions of users for hundreds of clients. In this talk, Jeremy Stanley and Alex Gaudio will tell the story of how (and why) this platform was built on Mesos, and outline some of the key challenges and detours taken along the way. This will include an overview of the goals and architecture of the system, and an account of the 10 month effort to develop, scale and stabilize the system for production.

Topics covered will include our use of AWS spot instances and Netflix’s Asgard to save costs, Marathon and Zookeeper to schedule jobs, detours with Chronos, Spark and Redis as our architecture evolved, and challenges we faced with maximizing resource utilization and ensuring we had identical and isolated test and production environments.