Month: May 2023

Highlights of Microsoft Build 2023!

Although much of Microsoft Build is centred around helping developers, there’s much for you an me as well, like avatars in Teams! Well, read on..

Microsoft MVPs get early access to the full list of topics from Microsoft Build so we can review all of the topics and then share back to the community with some fresh perspectives right after the public release date has passed.

So here’s my choice of interesting topics, hot off the press, in case you decide to check them out at https://build.microsoft.com/en-US/home or https://news.microsoft.com/source

For a full list of Microsoft Build topics go here.

Configuring the ‘NEW’ AMA (and Arc) agent to forward syslog to Sentinel

(Note: I have an older blog on this topic but based on new insights this article supersedes the old one)

Microsoft has already labeled the old ‘OMS’ syslog collector agent as legacy, so it’s important to think about switching to the new AMA agent.

If you have the word ‘syslog’ stuck in your brain you might think that in order to configure syslog you’d go into the Sentinel data connectors and search for ‘syslog’ – don’t do this.

The procedure for getting syslog into Sentinel is a bit different for VMs that are in Azure vs on-prem.

I’ve tested the recommendations below with the latest versions of Redhat And Ubuntu, on both on-prem VMs and in Azure.

For Azure VMs:
– Create a DCR and configure your syslog facilities. Done..

– Note that a DCR is a Data Collection Rule. This is a fairly new way to define what data needs to be collected. It somewhat hides the underlying method of collection – eg. you tell it to collect syslog from a list of servers in scope, and it takes care of communicating with the AMA agent to make it happen.

– In Sentinel, you don’t need to do anything! The DCR points the data to the servers in scope and the log analytics workspace.

For an on-prem VM
– Just make sure you install the Arc agent first, then create your DCR for syslog, just like for the Azure VM. Done!

A very simple test:

On your linux server, ssh in and type “logger testing123”

In Sentinel > Logs, type “search testing123” . You will see your logs show up in the Syslog table in about 5-10 minutes, depending on when you pushed out your DCR.

(TIP: If you don’t see your logs in Sentinel but you see them with tcpdump, then check your firewall rules, eg. by default RedHat7 will block incoming syslog)

To check if your logs are event getting to your syslog server use tcpdump eg:
tcpdump -i any port syslog -A -s0 -nn

And note that you could be seeing your logs with the above tcpdump command but they’re still not getting to Sentinel. In that case check if the local firewall rules are blocking syslog.

If you need to troubleshoot the actual AMA agent on linux (or windows), there’s a script for that here.

What about CEF Logs?

If you need to collect CEF formatted syslog then you will need to go into Sentinel > Data Connectors – and search for “Common Event Format (CEF) via AMA (Preview)”. There’s a script in here you’ll need to run for both your Azure VMs and your on-prem linux VMs/servers.

The AMA/CEF script is commonly used for when you are trying to collect CEF logs from somewhere like PaloAlto firewalls and you want to forward them into Sentinel.

After installing the CEF forwarder script, create a DCR to forward the logs to Sentinel’s workspace. MAKE SURE YOU CREATE THIS DCR from Sentinel> Data Connectors CEF Via AMA. This will provide a special CEF configuration with the string ‘SECURITY_CEF_BLOB’ inside a json file located at /etc/opt/microsoft/azuremonitoragent/config-cache/configchunks/nnn.json. If this configuration doesn’t exist, your CEF data will only be forwarded to the Syslog table, not the CommonSecurityLog table.

Tip: Don’t use the same ‘syslog facility’ for CEF as you use for syslog or you may get log duplications. eg. if you use ‘user.info’ for CEF, don’t use it for your ‘regular’ syslog DCR.

There’s a great troubleshooter script for CEF:

Read this or jump the bottom of the page and run the troubleshooter:


sudo wget -O sentinel_AMA_troubleshoot.py https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/DataConnectors/Syslog/Sentinel_AMA_troubleshoot.py &&sudo python sentinel_AMA_troubleshoot.py

Don’t get this confused with the ‘old’ CEF troubleshooter that was created for the OMS agent.

After running all of the troubleshooting tests, go to the Sentinel logs and look for the ‘MOCK’ test logs:

CommonSecurityLog
| where DeviceProduct == "MOCK"

If you need to run the MOCK test manually, here’s the command (sometimes the troubleshooting script doesn’t catch the test log so you may need to run it in another terminal session while running the script):


echo -n "<164>CEF:0|Mock-test|MOCK|common=event-format-test|end|TRAFFIC|1|rt=$common=event-formatted-receive_time" | nc -u -w0 <YOUR IP ADDRESS> 514

Another tip: Keep the /var mount point separate from root and give it about 500GB of disk space just for /var. Keep an eye on your disk space and increase this if necessary.

Another tip: There’s no need to keep the incoming syslog data in /var/log, so disable any default filters in /var/log/syslog.d/* that would store data there. The AMA agent will queue the data it needs in /var/opt/microsoft/azuremonitoragent/events/user.info and clear it out daily or sooner. If you don’t disable these default filters you may need to double the disk you need for /var.

References:

https://learn.microsoft.com/en-us/azure/sentinel/forward-syslog-monitor-agent

https://learn.microsoft.com/en-us/azure/sentinel/connect-cef-ama

OpenAI – What Should be Monitored?

Since the explosion of publicly accessible OpenAI, the question of how to monitor its use within an organization has been a frequently asked question.

Below are some topics relevant to the most common OpenAI services/features available today. Consider using these topics/suggestions as a starting point to creating a scope of topics relevant to security governance, and to help develop security policies for your organization.

Publicly Accessible OpenAI services

  • Description: Web sites like OpenAI’s ChatGPT provide a wealth of knowledge and an opportunity to accelerate a user’s knowledge on an infinite number of topics.
  • Security Policy Consideration: Pasting corporate information into a public facing site of any kind should be considered prohibitive.

Corporate Licensed OpenAI services

  • Description: OpenAI resources such as Azure OpenAI can be enabled at low cost within the cloud. These AI models can be customized to solve complex challenges within an organization or provide public facing features which enhance a corporation’s service offerings.
  • Security Policy Consideration: Creation of resources in openAI based tools such as Azure OpenAI Studio and PowerApps should be controlled and monitored by the security team.

End User OpenAI Related Productivity Tools

  • Description: Microsoft’s Copilot is an example of end-user OpenAI tools that will change they way people work, and it will have a dramatic affect on their productivity.
  • Security Policy Consideration: Authorized use of AI tools, such as Copilot should be monitored.

Be aware of ‘Self-Aware’ OpenAI Tools

Description: If you’ve used Auto-GPT, you might be concerned about the ability of OpenAI tools to be given full root/admin control to do whatever it takes to provide the answer to a question. This includes creation of scripts, adding/deletion of files, and even rebooting your pc.

Security Policy Consideration: Strict monitoring of any open source OpenAI tools that are running on enduser pc’s or on servers should be strictly monitored and approved for use.

Security Monitoring and Best Practices

  • Monitoring of all use of AI generated activity should be monitored via EDR, CASB, SIEM etc.
  • Discuss with your vendors the best practices on how their OpenAI tools can be monitored.
  • Test/simulate the use of each OpenAI tool and validate your ability to monitor its activities, including individual user access and change controls.

Creating Your Own Threat Actor Research Bot

There is nothing perfect about auto-gpt but like chatgpt it’s another tool that if used creatively can be used to achieve amazing things I wouldn’t have even considered doing 2 months ago.

If you want to read about my odd path of discovery in building this script, see the short story below, otherwise just enjoy the script.

Ramon Gomez on LinkedIn had the idea of using auto-gpt to find threat actor in the new as they relate to the United States Energy sector.

His attempts at using auto-gpt failed but I gave it a try anyways.

Sure enough it failed for me too, but I carefully read the output from auto-gpt and I could see what it was trying to do:

  • download the enterprise-attack.json file from Mitre – this is a full ‘database’ of all things Mitre ATT&CK and it includes information about threat actors and some of the industries that they’re associated with.
  • create an run a python script that reads enterprise-attack.json and extract the threat actors associated with the US energy sector. – this script had syntax errors so it was never going to run, but it tried…
  • find a list of reliable new web sites that are related to cyber news. – this worked so I had a list of possible sites, but they weren’t perfect..
  • create another python script that scraped the news sites for information associated with the threat actors – again it tried and failed.

Although auto-gpt tried and failed, it had an excellent approach to the problem.

And using ‘regular’ chatgpt I was able to ask the same sorts of questions and get much better answers.

Finally, as a result, chatgpt (and I) came up with the script you see below.

Note that this script has flaws, like some of the urls aren’t useful (but some are), but it does in fact work! enjoy.

import requests
from bs4 import BeautifulSoup

# Define a dictionary of threat actor names and their aliases
threat_actors = {
    'APT1': ['Comment Crew'],
    'Lazarus': ['Lazarus'],
    'APT29': ['Cozy Bear'],
    'APT32': ['OceanLotus Group']
}

# Define the URLs for the news resources
# Loop through the URLs and extract relevant web page URLs
# Define the URLs of the news resources

urls = [

  'https://www.fireeye.com/blog/threat-research.html',

  'https://www.kaspersky.com/blog/tag/apt',

  'https://www.ncsc.gov.uk/news/reports',

  'https://thehackernews.com/search/label/apt',

  'https://www.recordedfuture.com/apt-group-threat-intelligence/',

  'https://www.anomali.com/blog/threat-research'

]

webpage_urls = []
for url in urls:
    html = requests.get(url).text
    soup = BeautifulSoup(html, 'html.parser')
    for link in soup.find_all('a'):
        href = link.get('href')
        for actor in threat_actors:
            if actor in link.text or any(alias in link.text for alias in threat_actors[actor]):
                webpage_urls.append(href)

# Print the extracted webpage URLs
for url in webpage_urls:
    print(url)

Cheat Sheet for Configuring Carbon Black Cloud (EDR) for Sentinel

(And a pretty good example of configuring ANY Sentinel data connectors that use APIs and Azure Functions.)

There are several ways to get data into Sentinel.

If your log source requires an API pull, it’s very likely that Sentinel will use an Azure Function app to pull the data.

Vendors have many uses for their APIs so you can’t assume that theirs was built just to please your SIEM.

As a result there is often a bit of a learning curve when trying to understand how to get API logs into Sentinel and how Azure functions play a role.

Here are the 3 basic steps for getting your API logs into Sentinel:

  • Spend some time to understand the vendor’s API – List out all of the available API options. For Carbon Black, there are over 15 APIs!!! But you need just 2 of them for Sentinel.
  • Collect the variables needed to configure the Azure Function – for Carbon Black there are about 15 variables!!! (see cheat sheet below). You’ll need the help of your friendly AWS admin as well as your Carbon Black admin.
  • Get familiar with how to troubleshoot a vendor’s API and Azure Functions – If you’re familiar with curl, wget and/or Postman, then you’re on your way, but that subject is out of scope for this article. For Azure Functions it’s mostly about being able to tweak your variables and how to monitor the output logs (see cheat sheet below).

My cheat sheet for Sentinel Carbon Black Cloud (EDR) Connector

Converting this cheat sheet from Excel to fit in this blog is beyond my skills, so I’ve attached a pdf version.

I’m confident that if you can configure Carbon Black Cloud, you’ll probably have no issue with most other Azure Function API connectors for Sentinel.

I doubt any other connectors will have this many variables!

Addendum: Which audit logs should I collect?

When configuring this connector, there are 4 check boxes to choose from. Here ‘s a brief explanation for them:

CB Alerts Only:
If you just need the CB alerts, then just check off ‘Alert – supported with SIEM API credentials’. This is a good starting point since you don’t have to worry about connecting to AWS.

CB Alerts with All Events
If you need all of the events associated with the alerts, then check off these 2:
Event – supported with aws s3 bucket credentials
Alert – supported with aws s3 bucket credentials
And then proceed to set up an s3 bucket along with everything else.

Audit
These are just basic Carbon Black audit logs, no alerts/events here.
And no need for s3 buckets so enable it with no difficulty.

Click to access carbonblackcloud-sentinel-collector-configuration-notes-5.1-1.pdf