How to reduce noise in your SIEM?
Your SIEM (Security Information and Event Management)’s ROI depends on how effectively you tune it. Data sources like Firewall, AD server, File server, generates millions of logs every day. Without proper audit policy and SIEM filters, 80% of logs are noise. If you do not tune it, you will not get real value from your SIEM investment. Some of the SIEM’s pricing is based on log size, so reducing noise will not only help you to get a clear visibility of critical events, but saves your money. If you have the policy of ‘send everything’ to SIEM, you will be looking for needle in a haystack. So here we go,
There are two different ways to do it.
- Configure the audit policy to get only what is required.
- Create filters in SIEM to stop storing events which are not required.
Audit policy enforcement
By configuring audit policy selectively, we will not only avoid the noise but improve the performance of hosts which are generating logs. If a host is writing too many events every second, it will use hefty physical memory.
Windows noise filtering and tuning
Audit policy in Windows should be properly configured to get what we want in Windows Event viewer. One of the noisiest audit policy in Windows is Audit Object Access. Subcategories
|Audit File Share|
|Audit File System|
|Audit Filtering Platform Connection|
|Audit Filtering Platform Packet Drop|
|Audit Handle Manipulation|
|Audit Kernel Object|
|Audit Other Object Access Events|
If you prefer Microsoft’s official recommendation for setting audit policy, these subcategories are not mentioned anywhere. However, if you are monitoring file, registry changes you have to enable these subcategories in your audit policy. In addition to setting up the audit policy, you have to add auditing and attributes to a file/folder/registry . This is a crucial step. You should only enable the required ‘user groups’ or user. Also, select only the required permissions to be monitored instead of selecting full control. If you are looking for details about each file/folder attribute, refer this guide
Let’s say that you want to monitor only when a user deletes file/folder, then you have to select the permissions as shown below.
Selecting only what you need helps to reduce noise and provides clear visibility in the ocean of logs. This applies to registry changes as well.
Another noisy subcategory under Audit Object Access is Audit Windows Filtering Platform Connection . It contains Windows firewall allowed and denied connection. You should get rid of the denied connection Event ID 5157 as it does not have real value.
auditpol /set /subcategory:”Filtering Platform Connection” /success:disable /failure:enable
Another noisy Windows event is DNS Debug logs. From Windows Server 2012, Windows has introduced DNS Audit and Analytic events which has less performance impact comparing with Debug logs. Refer to this documentation from Microsoft to enable DNS Audit and Analytic events.
Firewalls noise filtering and tuning
When it comes to a firewall, log severity level should be configured properly. Ensure that logging severity is set as Informational level. Unless required do not enable ‘Debug’ severity in the firewall. Debug events will easily flood your SIEM. Each vendor has their recommendations to filter the noise. Cisco is one of the top vendors and has the following recommendations
Palo Alto – https://splunk.paloaltonetworks.com/tune-or-reduce-firewall-logs.html
Checkpoint – Logging less data:
- You can choose to stop logging traffic that is considered expected, such as DNS queries and more, by changing the Track option in its rule to none. Logging less data per second can decrease the disk I/O load and the processing of the log indexing.
- Security policies in R80.10 include new Track options: Detailed log and Extended log. These options add additional data in every log card regarding the relevant application, resource and file processed even for rules which do not explicitly specify applications or content. However, adding information to each log entry results in an increased workload at the log server. If you experience performance problems with your Log Management Server, consider re-evaluating these options.
- You can also reduce the track level at various other places in your security policy, such as adding packet capture to IPS protections and Threat Prevention policy rules, logging implied rules, and more.
Filtering at SIEM level
This step is required if you can’t change the audit or logging policy in your hosts right away or for some reasons. Every SIEM has a feature to create a filter logs before they are stored. Some SIEM products lets you archive everything before filtering, and store the filtered data in it’s DB. If you are using an additional SIEM or log management product, you can send everything to that secondary SIEM and filter the data in your primary SIEM. After identifying the nosiest events based on the recommendations mentioned above, create filters in SIEM. Here are user guides from different SIEM vendors to create filters
If you have guides for any other SIEM, please post it in the comments and I will be happy to add it to this post.
By following the above strategy will help you to reduce noise and improve the performance of the hosts, SIEM and licensing. In the end, it will make the security analysts to query faster, hunt better and perform forensics of security incidents without a need of going through noisy events.