Lyft Envoy and the birth of the service mesh

01 · CONTEXT

From monolithic architecture to the network as a problem

Envoy was built at Lyft to move the company's architecture away from a monolith. According to the CNCF announcement, Matt Klein led the team that designed the technology to abstract most of the network complexity away from the developer.

The project's core idea is that the network should be transparent to applications: when network or application problems arise, it should be easy to find the source. The documentation describes Envoy as an application-layer proxy and communication bus for large, modern service-oriented architectures.

The documentation also notes that it is increasingly common for these architectures to use multiple application frameworks and languages.

02 · THE IDEA

One proxy next to each service

Envoy is a standalone process, designed to run alongside every application server. All the Envoys form a transparent communication mesh. Each application sends and receives messages over localhost and does not know the network topology.

1. The calling service only knows the proxy running next to it and talks to it over localhost.

2. The caller's proxy carries the call across the network to the destination's proxy, which is part of the same mesh.

3. The destination's proxy hands the call to the local application, which also does not need to know the topology.

4. The proxies also emit statistics to a collector.

According to the CNCF, Envoy is written in C++ for performance reasons, and its out-of-process architecture serves any application, in any language or runtime.

The documentation points to two substantial benefits of this architecture over the traditional library approach: it works with any application language and allows quick deployment and upgrades across the entire infrastructure.

03 · DISCOVERY

Service discovery and dynamic configuration

Envoy can optionally consume a layered set of dynamic configuration APIs for centralized management. The layers bring updates about the hosts of a backend cluster, the clusters themselves, HTTP routing, listening sockets, and cryptographic material.

For a simpler deployment, backend host discovery can be done through DNS resolution. The recommended way to build a mesh, however, is to treat service discovery as an eventually consistent process.

1. Dynamic configuration reaches the proxy from a central management service.

2. Among the updates are the hosts that make up each backend cluster.

3. The proxy can actively check the health of upstream clusters.

4. It combines what discovery reports with the result of health checking to define the healthy load-balancing targets.

5. In addition to active checking, there is passive health checking, done by an outlier detection subsystem.

04 · BEHAVIOR

What a single proxy centralizes

Because it is a standalone proxy and not a library, Envoy implements advanced load balancing techniques in one place, accessible to any application. The list in the CNCF announcement includes zone awareness, retries, timeouts, circuit breaking, rate limiting, shadowing, and outlier detection.

The documentation also sees substantial benefit in using the same software at the edge. Envoy has a routing subsystem that routes requests based on path, authority, and content type, among other criteria.

1. External traffic enters through an edge proxy, the same type of software that runs next to the services.

2. The edge proxy routes the request to the destination service.

3. From there on, calls between services go through the mesh of proxies next to each service.

The main goal is to make the network transparent, but the documentation acknowledges that problems occur at both the network level and the application level. That is why Envoy includes robust statistics support for all subsystems and supports distributed tracing through third-party providers.

05 · LEGACY

From Lyft to the service mesh ecosystem

According to the CNCF, service meshes provide a dedicated infrastructure layer to make service-to-service communication secure, fast, and reliable. Klein described Envoy as a universal data plane, in which the mesh works wherever the code is running.

The CNCF announcement was published on September 13, 2017 and records the vote to host Envoy as the 11th project. It states that several companies contribute to the project, including Google and IBM, and that Lyft is collaborating with Google and IBM to bring Envoy to Kubernetes through the Istio project.