Data loading best practices
Azure Synapse Analytics has a rich set of tools and methods available to load data into SQL pool. You can load data from relational or non-relational data stores; structured or semi-structured data; on-premises systems or other clouds; in batches or streams. The loading can be done using various methods, such as with PolyBase, using the COPY into command, using ADF, or creating a data flow.
How to do it…
In this section, we'll look at some basic best practices to keep in mind as you work.
Retaining a well-engineered data lake structure
Retaining a well-engineered data lake structure allows you to know that the data you're loading regularly is consistent with the data requirements for your system.
When loading large datasets, it's recommended to use the compression capabilities of the file format. This ensures that less time is spent on the process of transferring data, using instead the power of Azure Synapse's...
