Observe with Rust: OpenTelemetry and Tremor

Tremor is a program that helps manage unstructured data and provides cost savings and flexibility for enterprise companies.

• Tremor uses pictures and gifs to help manage unstructured data and provide observability for critical services
• Tremor allows for rate limiting of services and applications to manage flow of input
• Tremor provides significant cost savings compared to other log management systems
• Tremor can be used as an upstream piece of an open telemetry agent to manage unstructured data
• Tremor provides flexibility for enterprise companies to manage their data in a programmatic interface

Tremor was able to process 5 gigabytes per second during a peak event like Cyber Five, which resulted in significant cost savings for Wayfair. Before switching to Tremor, Wayfair used Logstash and had to keep about 100 nodes at a time to keep up with the pace of their infrastructure, resulting in a cost of $40,000 per month. After switching to Tremor, Wayfair was able to process similar loads with a significantly smaller footprint, resulting in a cost of less than $1,000 per month.

Tremor is an early stage event processing system for unstructured data with rich support for structural pattern matching, filtering and transformation. Hundreds of companies around the world are using Rust in production today for fast, low-resource, cross-platform solutions. Gather round your screens and lecterns to hear the story of open source infrastructure helping a tech company sell furniture. We will show how it all happens with Rust and Tremor, and how we tune in and out of what matters; all with a lower cost of infrastructure and maintenance. Tremor users also appreciate the “software-like” syntax of querying through their systems, without sacrificing the performance of Rust. We're proud as supporters of Tremor to integrate OpenTelemetry into our codebase and feature set. We are excited to share details of this integration, how it works, and how we use it at Wayfair. Come to hear bad jokes and good technical content!