Validating your SIEM Rules

Here are some thoughts on SIEM ‘Use Case Validation Testing’.

Most of the time I work with SIEMs that have decent out of the box rules/correlations, so you can match the available log sources to the available correlations and you’re basically done.

Occasionally I’ll be asked to validate the correlations through some sort of testing process.

This works fine for old-school detections and basic compliance rules, where it’s easy to have the customer perform some action like 3 login denies to trigger a matching correlation.

However it’s not so easy to say “perform a Teardrop and a Hafnium attack”, so specific threat based attack simulations are not practical.

So here’s an outline for how to approach SIEM rule validation:

• List out the high level ‘tactics’, using MITRE or just a simpler list to get things started. Remember these tests must be associated with your available log sources eg:
  • Authorizations – password spray against a domain controller – using Active Directory or Azure AAD logs.
  • Threat – triggering a virus detection – using Microsoft Defender for endpoint
  • Keep adding to this list based on: <tactic> <available log source>
  • Try to provide at least one test for each available log source and grow from there.
  • Note that some log sources may not be use for correlations, but as investigative evidence for post-detection analysis.

Now that you have a list of detections, you need a ‘toolkit’ of methods for performing your attack simulations. Consider these:

• Create a lab space
  • Many of these attack simulations should not be performed on production systems! A lab space provided the freedom to experiment more freely without the risk of doing harm.
• Purple Teaming
  • Hire a pro and do end to end validations.
  • This approach helps educate your entire SOC/blue team on how the correlations work and how to tune them for appropriate detections.
  • It may also include a Compliance and/or MITRE APT planning session to map your correlations to appropriate controls.
• Attack Simulations
  • Start simple – download the eicar (fake) virus. This will trigger an alert from most AV and EDR tools.
  • Get open source – Red Canary’s Atomic Red Team tool is easy to get started with and will definitely wake up your EDR agent.
  • Make sure you have rules in your SIEM to detect your attack simulations.
• Active Defenses
  • Configuring ‘cyber deception‘ within your network is a good way to make your red teamers cry. Simply having some user accounts enabled (with no login privileges) is enough to trigger an alarm in your SIEM when someone tries to login with it.
  • Simply create some user accounts, spread around some files and start playing minesweeper with your red team (DJ I can’t get minesweeper out of my articles after you mentioned it).

So SIEM use case validation testing is an excellent task for all cybersecurity teams, but it does require some effort and coordination between all of your security teams – often more than the initial SIEM setup itself.