During my lunch break today, I tried to glean what “news for nerds – stuff that matters” had happened lately — but, when trying to access slashdot.org, I was met with this:
While I, admittedly, do not know what infrastructure Slashdot is running over, I bet that incoming requests hit a load balancer, which distributes it amongst a set of application instances serving content — and, in this case, by whatever metrics the load balancer was using, it couldn’t serve the request.
Much a-propos, this was (one of) the topics of a recent publication of mine, in IEEE Transactions on Networking, with my colleagues Yoann Desmouceaux, Pierre Pfister, Jerome Tollet, and W. Mark Townsley, entitled “6LB: Scalable and Application-Aware Load Balancing with Segment Routing” — wherein we introduce a load-balancer running exclusively within the IP forwarding plane, i.e., in an application protocol agnostic fashion – yet which still provides application-awareness and makes real-time, decentralised decisions.
We use Segment Routing to direct data packets from a new flow through a chain of candidate (in slashdot-error-message-lingo) backend servers, until one is available and decides to accept the connection, based solely on its local state. This way, applications themselves naturally decide on how to fairly share incoming connections, while incurring minimal network overhead, and no out-of-band signalling.
One of the remarkable results is, that using this approach — called 6LB — we’re able to satisfy more requests, but with less “backend servers”. Look at this graph:
For all the glory details of the experiments and approach, go read the paper (and, while you’re at it, this paper might be of interest, also) — but the run-down is this: using 6LB (the yellow line) as load balancer yields, using only 40 “backend servers”, lower mean response times to incoming requests than what is possible when using a “more classic” load balancing algorithm (purple line) with 48 “backend servers”.
So, Slashdot, feel free to reach out to us for an infrastructure scalability boast through 6LB and Segment Routing 😉