Tutorial: Building an Open Source Observability Stack

The conference presentation discusses the practicality of managing storage for multiple integrated applications in a busy environment, with a focus on the use of open telemetry and Pixie solutions.

• The speaker addresses a question from an audience member regarding the storage of multiple integrated applications in a busy environment
• The speaker explains that open telemetry and Pixie solutions can be used to capture and store data locally, and then filter and batch the data as needed
• The speaker emphasizes the importance of considering sample strategies, filtering, and batching when designing an architecture for managing storage in a busy environment

The speaker explains that Pixie has a scalable solution for capturing and storing data locally, and then allowing on-demand access to the data through a portal. This approach is similar to the concept of Prometheus, where data is not constantly being pushed to a backend, but rather accessed on demand.

This tutorial offers attendees a quick way to experience open source observability for Kubernetes. The three basic pillars of observability are metrics, logs and traces. In the past, instrumenting an application to provide these signals could be a multi-month initiative. However, the latest and greatest in CNCF projects have taken this slog down to minutes. In this hands-on introduction to the FOP stack (Fluentbit, OpenTelemetry, Prometheus), we’ll show you how to go from zero observability to deep coverage in minutes. Next, we’ll dive into how you can utilize the OpenTelemetry format to analyze these diverse telemetry sources together when debugging incidents. Finally, we’ll go beyond metrics, traces and logs by adding Pixie to our (now) FOPP stack, and show how you can use progressive instrumentation to collect application profiles, kernel events, and even function tracing without any redeploys.