Azure Spring Clean : Diagnostic Settings
Hey everyone !
I hope you are doing, first, I would like to thank Joe and Thomas for this event !
All the contributions are available on : Azure Spring Clean official website : https://www.azurespringclean.com/
I will talk about the Diagnostic Settings. We often tend to forget it even if it is part of the Microsoft recommendations in Azure Security Center.
What is the Diagnostic Settings ?
Before talking about the Diagnostic Settings, we must understand the data source on Azure Monitor.
As you know, Azure Monitor is based on different Logs and Metrics that we can collect from multiple resources. All these data may be analyzed together using a set of tools in Azure Monitor.
Microsoft organizes the sources of monitoring where the highest one is the application itself and the lower one is the components of Azure platform.
For more information, please visit this documentation.
Back on track, Azure Resources generate a significant amount of monitoring data. these data may be used in several scenarios. and splited into two types :
- Metrics :
Metrics are collected at regular intervals, they are numerical values that describe some aspect of a system at a particular point of time.
- Logs :
Logs are events that occurred within the system.
Let’s talk now about the Azure platform logs which provide detailed diagnostic and auditing information for Azure Resources and the Azure platform they depend on.
All these logs may be exported to one of the following :
Any of these destinations must be created before creating the diagnostic settings.
The Log Analytics does not need to be in the same region as the resource being monitored.
The Storage Account should not contain other non-monitoring data so we can control access to the data. you may use the same storage account to keep all monitoring data in a central location. The storage needs to be in the same region as the resource being monitored.
The Event Hub namespace needs to be in the same region as the resource also. Streaming to Event Hubs requires Manage, Send and Listen permissions.
How to set Diagnostic Settings ?
Each resource has its own diagnostic setting defined by :
- Categories of Logs and metric data sent to the destination
- Destinations to send the logs
So to make a universal script that allows Diagnostic Settings won’t be easy.
Since the diagnostic settings it’s an important piece in the security puzzle, the idea is to create an Azure Policy (DeployIfNotExists) for each resource that you want to monitor so you automatically get a diagnostic settings for each resource you create.
When a client deploys an Azure resource that you want to be monitored (A storage account by example), the Azure Policy does not find the diagnostic settings so it will run a DeployIfNotExists Policy.
- How to achieve this ?
The use case is as following :
- The Policy :
The “deployIfNotExists” policy will be as following :
By deploying and assigning this policy we will get a policy that automatically deploys the diagnostic settings on a storage account when it gets deployed :
So when we create a storage account, we go from this state :
to this one :
With the following diagnostic settings :
Alright, the Diagnostic Settings are created, what’s next ?
Well, we need to know how to explore these logs. Each Azure Resource logs may be explored differently.
Taking it to the next level
Alright, now that we have our security compliance (so the Diagnostic Settings up). How can we maintain it ? I mean how can we avoid the deletion of the diagnostic settings by someone and don’t wait until the deployIfNotexists run a compliance recovery :
Well, don’t worry, we can control it !
Huh ? how can we do all this ?
Well first, we will set up an alert :
Then we set an Action Group that triggers a Function which got our logic :
The idea know is to execute this script inside the function so it put back the same Diagnostic Settings that we had :
So to end this article, I made a video that summarize all this blog post with the demos :
Thank you !
