Azure Spring Clean : Diagnostic Settings

Hey everyone !

I hope you are doing, first, I would like to thank Joe and Thomas for this event !

All the contributions are available on : Azure Spring Clean official website : https://www.azurespringclean.com/

I will talk about the Diagnostic Settings. We often tend to forget it even if it is part of the Microsoft recommendations in Azure Security Center.

What is the Diagnostic Settings ?

As you know, Azure Monitor is based on different Logs and Metrics that we can collect from multiple resources. All these data may be analyzed together using a set of tools in Azure Monitor.

Microsoft organizes the sources of monitoring where the highest one is the application itself and the lower one is the components of Azure platform.

Image from MS Docs

For more information, please visit this documentation.

Back on track, Azure Resources generate a significant amount of monitoring data. these data may be used in several scenarios. and splited into two types :

  • Metrics :

Metrics are collected at regular intervals, they are numerical values that describe some aspect of a system at a particular point of time.

  • Logs :

Logs are events that occurred within the system.

Let’s talk now about the Azure platform logs which provide detailed diagnostic and auditing information for Azure Resources and the Azure platform they depend on.

All these logs may be exported to one of the following :

Any of these destinations must be created before creating the diagnostic settings.

The Log Analytics does not need to be in the same region as the resource being monitored.

The Storage Account should not contain other non-monitoring data so we can control access to the data. you may use the same storage account to keep all monitoring data in a central location. The storage needs to be in the same region as the resource being monitored.

The Event Hub namespace needs to be in the same region as the resource also. Streaming to Event Hubs requires Manage, Send and Listen permissions.

How to set Diagnostic Settings ?

  • Categories of Logs and metric data sent to the destination
  • Destinations to send the logs

So to make a universal script that allows Diagnostic Settings won’t be easy.

Since the diagnostic settings it’s an important piece in the security puzzle, the idea is to create an Azure Policy (DeployIfNotExists) for each resource that you want to monitor so you automatically get a diagnostic settings for each resource you create.

When a client deploys an Azure resource that you want to be monitored (A storage account by example), the Azure Policy does not find the diagnostic settings so it will run a DeployIfNotExists Policy.

  • How to achieve this ?

The use case is as following :

  • The Policy :

The “deployIfNotExists” policy will be as following :

By deploying and assigning this policy we will get a policy that automatically deploys the diagnostic settings on a storage account when it gets deployed :

So when we create a storage account, we go from this state :

to this one :

With the following diagnostic settings :

Alright, the Diagnostic Settings are created, what’s next ?

Well, we need to know how to explore these logs. Each Azure Resource logs may be explored differently.

Taking it to the next level

Well, don’t worry, we can control it !

Huh ? how can we do all this ?

Well first, we will set up an alert :

Then we set an Action Group that triggers a Function which got our logic :

The idea know is to execute this script inside the function so it put back the same Diagnostic Settings that we had :

So to end this article, I made a video that summarize all this blog post with the demos :

Thank you !