Decentralized Routing For a Sharded Application On Service Mesh

The presentation discusses the implementation of a routing policy using Envoy filters and a lookup service to provide flexibility for multiple use cases.

• The solution involves using a lookup service to extract the ID to route the chart or throughout the request
• The routing policy creates an HTTP sidecar outbound filter on the clients
• The solution provides flexibility for service owners to define the config they want to use and the lookup service to be used
• The solution was pre-built with the Envoy proxy and the Vasan binary was copied into the image and referenced in the Envoy filter
• Config Maps in Kubernetes have a limit of 100 MB and may not be able to store binaries larger than that

The presenters had issues with storing the Vasan binary in the container due to its size, so they used nginx proxy to store the binaries and called it from the container. They also tried using plugins, but at the time, there were authentication issues. They were able to pull the plugin from artifactory or DockerHub and it worked fine. They also mentioned that if a plugin is larger than 100 MB, it may not be able to be loaded in Kubernetes.

Intuit has a few monolithic applications that are scaled horizontally by sharding. It is common practice to use a dedicated frontend layer to route requests to a specific application shard based on request attributes. The attributes used for shard determination are non static and include tens of millions of users and thousands of services. Hence maintaining a static mapping of these attributes to shards is not feasible. For maintainability and separation of concerns, a dedicated look up service could be used to store and retrieve this shard information. Currently at Intuit, the sharded routing is performed in a central API Gateway using this lookup service. However, as we move our monolithic applications to service mesh, our next step was to decentralize the sharded routing to happen on a client’s service mesh proxy. A service mesh that utilizes a client side proxy like Envoy does not have an out of the box support for lookup based dynamic routing to the destination shards. This session discusses and demos how Intuit uses WASM to extend Envoy in a service mesh to provide decentralized routing for a sharded application.