Building a Portable Kubernetes Deployment Pipeline with Argo Workflows and Events

The presentation discusses the challenges faced in building a Kubernetes-based platform using Argo workflows and events, and the solutions implemented to achieve the goal of a portable platform that can be deployed anywhere.

• The platform being built is a synthetic environment platform for the defense industry that can simulate movements across multiple domains.
• The platform needed to be deployed on any Kubernetes service, including air-gapped facilities, and be infrastructure agnostic.
• Kubernetes and Google Kubernetes Engine were chosen as the orchestrator cluster, with Kubernetes CRDs used for versioning.
• Namespace separation with RBAC was used for customer projects, and a dedicated REST API was built to abstract the Kubernetes and infrastructure layer.
• The presentation covers best practices for building a cloud-native platform to work across multi-cloud environments, rejecting the typical CI/CD model in favor of a custom solution, isolating workflows in multi-tenanted environments, running Terraform in Argo, and managing highly scalable cluster configurations.

The platform needed to be deployed in various environments, including military contexts with zero internet connectivity. This posed challenges such as not being able to pull containers down, which required finding alternative deployment methods such as using USB sticks. The team also had to consider access control and limited access to only the necessary components.

In this session, Jetstack and Improbable will uncover the key learnings from building a Kubernetes based platform, driven by Argo workflows and events. Featuring discussion on the platform's strong data aggregation and the ability to deploy on any estate (including air-gapped facilities), this session will investigate the problems and benefits of using Argo and Kubernetes as the key ingredients towards the goal of a holistic approach to portable platforms. The technical content of the presentation will cover: - A general overview of why Argo Workflows and Events were selected - Best practice when building a cloud native platform to work across multi-cloud environments - Rationale for rejecting the typical CI/CD model in favour of a custom solution - Techniques for isolating workflows in multi-tenanted environments - Running Terraform in Argo - Ephemeral environments and managing highly scalable cluster configurations - Platform portability and being infrastructure agnostic