Category: siem

Validating your SIEM Rules

Here are some thoughts on SIEM ‘Use Case Validation Testing’.

Most of the time I work with SIEMs that have decent out of the box rules/correlations, so you can match the available log sources to the available correlations and you’re basically done.

Occasionally I’ll be asked to validate the correlations through some sort of testing process.

This works fine for old-school detections and basic compliance rules, where it’s easy to have the customer perform some action like 3 login denies to trigger a matching correlation.

However it’s not so easy to say “perform a Teardrop and a Hafnium attack”, so specific threat based attack simulations are not practical.

So here’s an outline for how to approach SIEM rule validation:

  • List out the high level ‘tactics’, using MITRE or just a simpler list to get things started. Remember these tests must be associated with your available log sources eg:
    • Authorizations – password spray against a domain controller – using Active Directory or Azure AAD logs.
    • Threat – triggering a virus detection – using Microsoft Defender for endpoint
    • Keep adding to this list based on: <tactic> <available log source>
    • Try to provide at least one test for each available log source and grow from there.
    • Note that some log sources may not be use for correlations, but as investigative evidence for post-detection analysis.

Now that you have a list of detections, you need a ‘toolkit’ of methods for performing your attack simulations. Consider these:

  • Create a lab space
    • Many of these attack simulations should not be performed on production systems! A lab space provided the freedom to experiment more freely without the risk of doing harm.
  • Purple Teaming
    • Hire a pro and do end to end validations.
    • This approach helps educate your entire SOC/blue team on how the correlations work and how to tune them for appropriate detections.
    • It may also include a Compliance and/or MITRE APT planning session to map your correlations to appropriate controls.
  • Attack Simulations
  • Active Defenses
    • Configuring ‘cyber deception‘ within your network is a good way to make your red teamers cry. Simply having some user accounts enabled (with no login privileges) is enough to trigger an alarm in your SIEM when someone tries to login with it.
    • Simply create some user accounts, spread around some files and start playing minesweeper with your red team (DJ I can’t get minesweeper out of my articles after you mentioned it).

So SIEM use case validation testing is an excellent task for all cybersecurity teams, but it does require some effort and coordination between all of your security teams – often more than the initial SIEM setup itself.

SIEM Use Case Guide – Part 2

And I’m talking real smoking guns, not crappy anomaly alerts.

From my experience, the most effective use cases for threat detection are those which simply:

  • Provide a list of detections which have a high confidence of threat
  • Look for validations which follow the detection.

So:

Here’s an example for logic that would provide good use cases involving malicious intent:

  • First create a list of detections that provide very high confidence of a threat.
    • This is easier than you think if you’ve been using a SIEM for some time.
    • Just look back at both rules that have fired and those which have not.
      • The ‘have not fired’ rules like ‘Hafnium detected’ are likely very fined rules so when they do alert it’s likely for good reason.
      • The rules that have fired, are consistently a true positive, and identify as a threat, is likely a very short list.
  • Follow that with evidence from 1 or more additional alerts to ‘validate’ the first alert.

The ‘direction’ of the action might also be important, eg:
Inbound > outbound detection with validation of sustained activity.

Here’s a couple of specific examples:

  1. Detection: EDR callback
    1. Relevant entities: src_ip and dest_ip
    2. Validation1: EDR detection fires a 2nd distinct threat alert at src_ip
    3. Validation2:Firewall traffic continues for 15 min to dest_ip
  2. Email Malware ‘onclick=true’ (user clicked on a malicious link)
    1. Then followed by logic from rule #1 above.

Additional thoughts:

Validations do not have to be part of the SIEM threat detection. They can be added from SOAR playbook(s) and apply ‘tags’ to the alert, which can in turn be used by other correlations or SOAR playbooks. Eg. tag: EDR_distinct_threats=2, callback_distinct_count=2

This means the correlations become VERY simple and the validation or ‘recorrelations’ steps are done by the SOAR action.

By using SOAR you can provide full automation of an action by isolating a machine, disabling a user etc.

There should be pressure put on vendors to provide a ‘metric of confidence’ on specific detections. i.e. does ‘High’ really mean High? This makes it easier to create use cases with clear malicious intent since you wouldn’t have to manually compile a list of high confidence detections.

Some thoughts on high confidence alerts:

  • Most Sentinel analytic rules with a value of High are high confidence.
  • Deception related alerts – this is a new term for many, but it’s catching on.
  • Callbacks – Fireeye first made these famous and they’re still a good example of a reliable alert.

Part 1 – Super Simple SIEM Use Case Guide

Part 3 – Real World Use Cases

Super Simple SIEM Use Case Guide

Here are some very practical suggestions on the basic steps needed to develop SIEM related use cases.

Also see: part 2 and part 3

  • Create a list of all your available logs (see ‘next step’ below for a long list of suggested logs)
  • Configure your SIEM to collect those logs.
  • Create your ‘use cases’, with whatever tools you have in your SIEM.
  • Note that all logs are not created equal. The type of log will determine the options for the use cases. Generally speaking you’ll find the following types of logs in your environment.
  1. login activity
  2. configuration changes
  3. errors
  4. network traffic
  5. threats
  • Before considering correlations/alerts, create search queries to validate what information exists in the logs. Here’s a high level list of search queries to make for most log sources:
  1. AUDIT – Basic queries that present the most important fields in the log. eg: date,username, source, destination, action
  2. HUNT – More advanced queries designed to search for a specific pattern, eg. login denies, high severity intrusions
  3. INVESTIGATE – Queries to search for more details on a given ‘entity’ – eg. search all events for a given username or hostname.


Here’s a good range of detections that any SIEM should have:

Use Case Examples
  • Endpoint Detections (EDR logs – endpoint detect and response)
  • Network Intrusion Detections
  • DNS threats (eg. Cobalt Strike Beacon Detection to known IPs)
  • Firewall threats (eg. high rate of denies by source, destination)
  • Unauthorized Changes (to roles, resources, assets, etc)
  • Policy Violations (eg. detected those detected by Microsoft Defender for Cloud)
  • Suspicious/Risky User Activity (eg. Defender for Identity agent on Domain Controllers)
  • Threat Intelligence (eg. outbound traffic to known bad IP)
  • Endpoint and Identity alerts on sensitive assets (eg. maintain a list of sensitive/critical assets)
  • Alerts for sensitive users (maintain a list of sensitive/critical/privileged users)
  • Command and Control alerts (from EDR logs)
  • Lateral Movement alerts (Threat based alerts like EDR and unusual traffic across network zones – eg. port scan from workstation zone to server zone)
Next Step Suggestions
  • Apply ‘meta’ information to your use cases by creating lists of information such as:
    • Administrators
    • Privileged Users
    • Critical Assets
    • Risky users
  • Identify gaps in your log collection, based on a ‘master list’ of desired log sources, acquire these missing sources and build more use cases to improve your attack surface monitoring.

Example log source gap list:

5G Wireless WAN
App
App Gateway
Application/COTS/Custom
AV/Anti-Malware
Call Management
CASB
Cloud Audit
Cloud Infrastructure Security
Cloud Resource Management
Data Management (Labeling & Protection)
Database
Database Monitoring
DDoS
Decoy
DHCP
Diagnostics & Monitoring
DLP
DNS
DNS Security
eCommerce
EDR
Email Protection
Email Server/Mail gateway
Enterprise Risk / Data Lake
External/Internal TI Feed
Fiber Switch
File Integrity Monitor (FIM)
Firewall/IDS
Honeypot
Identity & SSO
Identity Management
Identity Protection
IoT
Key Storage
Load Balancer/VPN
MDM
NAC
NetFlow
Network Traffic
NIDS
NIDS/NIPS
Privileged Access Management (PAM)
Remote Access and VPN
Router/Switch
SaaS
SAN
Security Configuration Management
Servers
Streaming Data Storage (Revenue Accounting)
System (OS)
UBA/Threat Analytics
Virtual Machines
Vulnerability scanner
WAF
WAF/Web Proxy/Web Content Filtering
Web
WebApp Scanner
Wireless Access
WirelessLAN

Need more ideas?

Simply search for ‘siem use cases’ and you’ll likely find a lot.

Some suggestions:

blueteamblog.com/siem-use-case-writing-guide

Microsoft Sentinel Analytic Rules

2 ways to get (free) Threat Intelligence feeds into Microsoft Sentinel

Like most things in life, there’s an easy way and a hard way…

The Easy Way

Anomali has a threat feed that supports Sentinel’s TAXII connector.

If you open a linux shell you can run this command to get the available channels:

curl -u guest https://limo.anomali.com/api/v1/taxii2/feeds/collections/

To configure one or more of the threat feeds in Sentinel, go to Connectors > Threat intelligence – TAXII. Note the API root URL will be:

https://limo.anomali.com/api/v1/taxii2/feeds

Once the threat data is loaded you can use it in your alerting and kql searches, eg:

ThreatIntelligenceIndicator| where NetworkIP != “”

The Hard(er) Way

You can pull in the TI data from Anomali or any other TAXII server using a playbook.

Although there’s a learning curve around using playbooks (Logic Apps), they’ve very powerful, and you can do things like filter out undesired data from your TI feed before it’s stored in Sentinel.

Here’s a great example:

https://techcommunity.microsoft.com/t5/microsoft-sentinel-blog/use-microsoft-azure-sentinel-and-anomali-match-for-actionable/ba-p/2080763