Dates
Conferences
Tags
Sort by:
Most recent
Conference: KubeCon + CloudNativeCon Europe 2023
Authors: Wilfred Spiegelenburg, Peter Bacsko
2023-04-20
tldr - powered by Generative AI
The scheduling framework Apache YuniKorn has extended the Kubernetes scheduler to add batch-focused functionality, including workload queuing, gang scheduling, and application sorting.
- Batch and data processing workloads require different scheduling requirements than service-oriented workloads.
- Apache YuniKorn provides batch-focused functionality on top of the existing Kubernetes scheduler.
- Features include workload queuing, gang scheduling, and application sorting.
- These features are useful for bursty deployments and high-performance computing.
- Apache YuniKorn is designed to be flexible and customizable.
Conference: KubeCon + CloudNativeCon North America 2022
Authors: Krzysztof Adamski, Tinco Boekestijn
2022-10-26
tldr - powered by Generative AI
ING Wholesale Banking Advanced Analytics team built a self-service platform using open source technology to empower employees to solve business needs. The platform has grown with over 400 projects and 2000 users. The team adapted cloud native tools and added a specialized Kubernetes scheduler (Volcano) to enable multi-tenant large scale processing capabilities.
- ING built a self-service platform using open source technology to empower employees to solve business needs
- The platform has grown with over 400 projects and 2000 users
- The team adapted cloud native tools and added a specialized Kubernetes scheduler (Volcano) to enable multi-tenant large scale processing capabilities
Conference: KubeCon + CloudNativeCon Europe 2022
Authors: Holden Karau
2022-05-18
tldr - powered by Generative AI
The presentation discusses the challenges of working with big data matrices and how Apache Spark, Apache Mahout, Kubeflow, and Kubernetes can be used together to solve these challenges.
- Kubernetes allows for elastic scaling but has limitations when it comes to fitting large matrices in memory
- Apache Spark and Mahout can distribute matrices across an unbounded number of pods/nodes
- Kubeflow can be used to make the process easily reproducible
- The presentation provides an anecdote about using these tools to denoise DICOM images of lungs of COVID patients