Build a Cloud Native Asynchronous Messaging System For Scale With Redis

The presentation discusses the use of Redis as a message broker for pub/sub messaging and event streaming, and provides recommendations for capacity planning and durability.

• Redis is a popular choice for pub/sub messaging and event streaming due to its performance and scalability.
• Capacity planning is important to avoid overwhelming the system and causing outages.
• Durability can be ensured through replication and backup mechanisms.
• Redis integrates well with other CNCF projects such as Prometheus.
• A demo is provided to illustrate the use of Redis in a web application for order processing.

The speaker emphasizes the importance of capacity planning and recommends rejecting messages when there is no spare memory. They also suggest using aof for durability and caution against restarting nodes without backup data. The demo shows how Redis can be used in a simple web application for order processing.

Large scale microservices architectures often rely on synchronous APIs to implement their functionality by communicating with a large number of other services. This pattern works well when all services are aware of when and how data changes. However, as the system size increases, the tight coupling and complexity can become difficult to maintain. A solution for this growing complexity is asynchronous message passing, where services register what types of messages they are interested in and submit messages to be broadcasted to the system. This highly decoupled system allows new services to be added without changing code of the existing components. In this talk, hear from Madelyn Olson, one of the Redis core maintainers and a software development engineer who will deep dive into best practices for implementing this asynchronous message design pattern. She will also demonstrate how to build, deploy, and monitor a highly scalable message broker system by using standard CNCF components, specifically Kubernetes and Prometheus, along with Redis, the “most loved” database.