#MesosCon 2015

Predicting performance and debugging problems is critical to successfully running a distributed system in production, and a simple system is easier to reason about and understand. But for many systems, simplicity is fundamentally at odds with performance and scalability -- oftentimes, optimizing our systems to reduce latency and improve scalability requires more and more complex designs. Running complex systems at scale is even riskier because rare corner cases are guaranteed to occur.

An important example of this is scaling an application's data plane. Improving performance and fault tolerance in the data plane requires techniques like caching, replication, partitioning, and precomputation. These techniques greatly increase complexity by introducing problems like cache invalidation, rebalancing, managing staleness, and maintaining consistency.

In this talk, Neha will discuss this tension between simplicity and performance, and describe how to think about and consciously manage these trade-offs.