Reaching Zen in Your Cluster Coordination
Distributed systems are hard and unfortunately often less perfect than we would hope. This talk shows a concrete example: Elasticsearch’s cluster coordination, called Zen Discovery, got a rewrite in version 7.0. Starting from a formal model, the coordination layer was rebuilt to address multiple issues discovered over the years. We discuss the main improvements of the new implementation: Master elections are much faster, the infamous minimum_master_nodes setting has been removed, growing and shrinking clusters becomes safer and easier, and leaves less room to misconfigure the system. Let us join the new, more Zen way of cluster coordination.
Demo: Take a look at the demo code for the talk.