logo

Efficient Model Exploring and Continuous Delivery With Polyaxon + Kubeflow

Authors:   Shotaro Kohama


Summary

Efficient Model Exploring and Continuous Delivery With Polyaxon + Kubeflow
  • Machine learning projects are highly iterative and require smooth transitions from the experimental stage to the productionization phase
  • Polyaxon is a cloud-native machine learning experiments platform that allows developers to run parallel and scalable hyperparameter tuning job in a declarative way
  • KubeflowPipelines is a workflow engine for machine learning pipelines
  • The Machine Learning Platform team at Mercari helps accelerate machine learning projects by Polyaxon and KubeflowPipelines
  • The team uses these open source tools to automate manual processes and accelerate iterations
  • The team uses Polyaxon for the model exploring phase and KubeflowPipelines for continuous training
  • The team also uses Spinnaker for continuous delivery
Melcalli uses machine learning models to design item price lunch and floor plans. They use Polyaxon and KubeflowPipelines to accelerate the iterations and automate manual processes. With these tools, they can manage end-to-end workflows and trigger deployed pipelines when new images are pushed to the registry. This allows them to achieve continuous training and deployment, avoiding early implementation in transition from the modern exploration phase to the continuous training phase. They also built their tools and set up configurations and integrations to be able to write and end the workflow easily.

Abstract

Machine learning projects are highly iterative. A project begins with an experiment phase, a productionization phase, an operation phase, and the next experiment phase. The smooth transition from the experimental stage to the productionization phase is the key to accelerate the iterations. Polyaxon is a cloud-native machine learning experiments platform that allows developers to run parallel and scalable hyperparameter tuning job in a declarative way. KubeflowPipelines is a workflow engine for machine learning pipelines. This talk shows how the Machine Learning Platform team at Mercari helps accelerate machine learning projects by Polyaxon and KubeflowPipelines.

Materials:

Tags:

Post a comment

Related work

Authors: Meenakshi Kaushik, Neelima Mukiri
2021-10-15



Conference:  Transform X 2022
Authors: Susan Zhang, Faisal Siddiqi, Bryan Catanzaro, Erhan Bas, Elliot Branson
2022-10-19