Real-time data analytics – Azure Stream Analytics, Azure Synapse Data Explorer, and Spark streaming

Some analytical use cases aim to monitor near real-time processes to assist in decision-making. For these cases, waiting for a dump from the source database to the target database is not effective; for this reason the concept of a data stream emerged.

In Azure, there are some service options for working data streams; let’s explore each of them.

Azure Stream Analytics

This is an Azure PaaS service and is very simple and efficient. Usually accompanied by the Azure event queue service, called Azure Event Hubs, Azure Stream Analytics consists of running SQL scripts on each of the events that arrive to be processed; that is, it is a data passing service, with some filtering, groupings, transformations, and so on.

The following figure reflects the data flow performed by Azure Stream Analytics:

Figure 11.11 – Azure Stream Analytics concept...