Observability with Fluent Bit: Logs, Metrics & Traces

Controlling data flow is crucial for cost reduction and efficient use of resources in logging and metrics management. Fluent Bit offers processors for modifying data and labels to optimize indexing and querying.

• Companies generate 20-30% more logs each year, making control of data flow important for cost reduction and efficient resource use
• Fluent Bit offers processors for modifying data and labels to optimize indexing and querying
• Lua scripting can be used for log processing
• Labels can be added, updated, or deleted using Fluent Bit processors
• Fluent Bit can be used for metrics management and data scraping

The speaker mentions that companies often end up paying for more data than they actually use because developers and logs send unnecessary data. He also shares an example of using Lua scripting in the configuration for log processing. Additionally, he demonstrates how labels can be added to metrics using Fluent Bit processors.

Observability is an art, and it is not necessary start analyzing data right away, it starts with a journey of collecting data from different sources and formats, the need to perform pre-processing, sanitization and finally having an end-to-end solution that allows you to centralize the information for further analysis.The following presentation will focus on various concepts around Logs, Metrics and traces, how they are implemented and how developers can take the most of them. Understanding the concepts that rule the technology helps to implement a scalable solution that can deal with common failure scenarios from your infrastructure.