Conference:  KubeCon + CloudNativeCon Europe 2023

Authors: Wilfred Spiegelenburg, Peter Bacsko

2023-04-20

The scheduling framework Apache YuniKorn has extended the Kubernetes scheduler to add batch-focused functionality, including workload queuing, gang scheduling, and application sorting.

• Batch and data processing workloads require different scheduling requirements than service-oriented workloads.
• Apache YuniKorn provides batch-focused functionality on top of the existing Kubernetes scheduler.
• Features include workload queuing, gang scheduling, and application sorting.
• These features are useful for bursty deployments and high-performance computing.
• Apache YuniKorn is designed to be flexible and customizable.

Conference:  KubeCon + CloudNativeCon North America 2021

Authors: Alexander Matyushentsev, Remington Breeze

2021-10-15

Kubernetes provides powerful features and empowers developers to solve lots of use-cases. Do you want to do GitOps, Progressive Delivery, batch processing? Easy - there is a tool that provides an effective way of solving each problem. The email that notifies the team about successful deployment is the cherry on the cake and should not be hard to do, right? Well, the notifications support is not as straightforward as it sounds. Does your team prefer Slack, Telegram, or all of the above? Do you want to fine-tune notifications criteria and avoid spamming your team about each and every change? Do you need customized notification messages that include details specific to your environment? We have solved this problem for Argo by introducing a generic Notification Engine that powers a notification experience for Argo projects. You will learn how to leverage the engine to configure notifications for Argo projects as well as how to use it for any other Kubernetes-native application.

Conference:  KubeCon + CloudNativeCon North America 2021

Authors: Gibbs Cullen

2021-10-14

The debate over stream vs. batch processing has been ongoing for years. While batch processing is optimized for large volumes of data, stream processing allows for real-time analysis. With monitoring workflows aimed at minimizing time to detect incidents, having real-time insights is critical for maintaining reliable cloud-native applications. Monitoring business-critical applications can become difficult at scale. How do you continue processing large volumes of real time data while maintaining valuable insights? There are OSS metrics solutions designed to ingest high volumes of data, but they also need to efficiently aggregate metrics for viewing and analyzing these volumes in real time. This talk will explore how two popular OSS projects, M3 and Thanos, have approached the problem of real time aggregation. The audience will learn how stream and batch processing methodologies have been leveraged by the community to aggregate data in real time, and the tradeoffs of each approach.