Recommended strategies and best practices

Before we end this chapter, we will quickly discuss some of the recommended strategies and best practices when working with Kubeflow on EKS.

Let’s start by identifying the ways we can improve how we designed and implemented our ML pipeline. What improvements can we make to the initial version of our pipeline? Here are some of the possible upgrades we can implement:

• Making the pipeline more reusable by allowing the first step of our pipeline to accept the dataset input path as an input parameter (instead of it being hardcoded in a similar way to what we have right now)
• Building and using a custom container image instead of using the packages_to_install parameter when working with pipeline components
• Saving the model artifacts into a storage service such as Amazon S3 (which will help us make sure that we are able to keep the artifacts even if the Kubernetes cluster has been deleted)
• Adding resource limits (such...