Optimized query statistics make the queries that run on your database perform faster. The query optimizer runs automatically, so in most cases the query plan, as shown in Figure 9.23,
Monitor Cluster Performance – Monitoring Azure Data Storage and ProcessingMonitor Cluster Performance – Monitoring Azure Data Storage and Processing
When you see the word cluster, your mind should navigate to the Apache Spark context. When running your data analytics on the Azure platform, you then need to determine if
Understand Custom Logging Options – Monitoring Azure Data Storage and ProcessingUnderstand Custom Logging Options – Monitoring Azure Data Storage and Processing
As you now know, there are many built‐in monitoring logs you can configure to feed into Azure Monitor using Log Analytics. All the categories within each Azure product’s diagnostic settings
Azure Stream Analytics – Monitoring Azure Data Storage and ProcessingAzure Stream Analytics – Monitoring Azure Data Storage and Processing
The same monitoring‐related features exist for Azure Stream Analytics as for other Azure products: alert rules, metrics, diagnostic settings, and logs. In addition to those, Azure Stream Analytics has a
Monitor and Manage Azure Synapse Analytics Logs – Monitoring Azure Data Storage and Processing-2Monitor and Manage Azure Synapse Analytics Logs – Monitoring Azure Data Storage and Processing-2
As shown in Figure 9.17, the DWU usage reached close to 100 percent on two occasions, and 41 percent of the memory was used. The performance level of the dedicated
Azure Databricks – Monitoring Azure Data Storage and ProcessingAzure Databricks – Monitoring Azure Data Storage and Processing
The “Azure Synapse Analytics” section described the logging of the execution of a notebook on an Apache Spark cluster (refer to Figure 9.16). In Chapter 3, Exercise 3.14, you provisioned
Monitor Data Pipeline Performance – Monitoring Azure Data Storage and ProcessingMonitor Data Pipeline Performance – Monitoring Azure Data Storage and Processing
There are numerous options for determining the performance of pipelines running on your Azure Synapse Analytics dedicated SQL pool. Two that are very useful are Metrics and the Monitor hub
Interpret Azure Monitor Metrics and Logs – Monitoring Azure Data Storage and ProcessingInterpret Azure Monitor Metrics and Logs – Monitoring Azure Data Storage and Processing
Metrics and logs located in the Monitoring section of most Azure products in the Azure portal are within the purview of Azure Monitor. Figure 9.2 shows an example of metrics,
LINK CONNECTIONS – Monitoring Azure Data Storage and Processing-2LINK CONNECTIONS – Monitoring Azure Data Storage and Processing-2
The previous DMV functions target the SQLPool database, which means they were focused on the database. The following functions target the master database on the dedicated SQL pool, which means
APACHE SPARK APPLICATIONS – Monitoring Azure Data Storage and ProcessingAPACHE SPARK APPLICATIONS – Monitoring Azure Data Storage and Processing
The Apache Spark Applications page was discussed in Chapter 6, “Create and Manage Batch Processing and Pipelines.” Figure 6.14 illustrates the details of an invocation of a Spark notebook. The