Basics of a machine learning-based project
Machine learning is formally defined as the use and development of computer systems that learn and adapt, by using algorithms and statistical models to analyze and draw inferences from patterns in data. Systems that utilize these practices have some common functional components, including intaking and preparing data for use, applying a machine learning algorithm to the data to create a model, distributing the model artifact for use, and using the model to make inferences on new data.
Note
In addition to these components, there is usually a discussion of specialized telemetry collection relating to the performance of learned models and data. Those practices, while important, are well outside the scope of this book.
Airflow is a great tool for orchestrating portions of these common components – the intaking and processing of data, loading pre-computed features into backend services, the learning of models, and the distribution...
