Moving and transforming data

We copied data in our storage account in the previous recipe, Copying data from the internet. In this recipe, we will join and transform the two datasets using a mapping data flow. They contain many tools that help us join, map, and aggregate our source datasets to produce another output (sink) dataset.

Mapping dataflows are a new addition for Data Factory. They were not part of the initial Data Factory v2 that was available in July 2018. The ADF team added the mapping data flows about a year later. For those who are familiar with SSIS, they will understand how to work with them in no time.

So, let's dig into our recipe!

Getting ready

This recipe assumes that you have access to an Azure subscription. It can be a free trial one as described in the recipe Creating an Azure subscription, in Chapter 1, Getting Started with Azure and SSIS 2019. It also assumes that you have created some Data Factory artifacts such as linked services, datasets...