Configuring Sentinel to Collect CEF Syslog with the AMA agent

(NOTE: I suggest you read my new blog post on this topic here. Some of the suggestions below may be outdated)

Here’s a quick guide to installing the AMA agent for collecting syslog data into Microsoft Sentinel.

Note: the ‘legacy’ method involved installing the old OMS agent and a separate step for the CEF collection script. The new procedure only requires a single python script, but it also requires the Azure Arc agent.
(which arguably is an added benefit for many reasons we won’t get into here).

Note: the AMA agent relies on the Azure Arc agent, unless your syslog server is in the Azure cloud.

This procedure is using Ubuntu 20.04, but it will be almost identical for most other linux platforms:

1. Install Python3 and other tools:
   • sudo apt-get update;sudo apt-get install python3;sudo apt-get install net-tools;sudo ln -s /usr/bin/python3 /usr/bin/python
2. Install ARC agent. 
   • (if you haven’t done this before, go to the Azure portal and search for Arc.)
   • Verify: In the Azure portal, verify the server shows up in the list of Arc servers.
3. In Sentinel, enable the CEF AMA connector and Create a DCR (data collection rule).
   • Enable these syslog Facilities: LOG_AUTH, LOG_AUTHPRIV and all of the LOG_LOCAL*. Use the log level of LOG_DEBUG.
   • WAIT 5 MINUTES for the Arc agent heartbeats to get up to Sentinel.
   • Verify: In the Sentinel log analytics workspace, query the ‘Heartbeat’ table – verify your arc server is here. 
4. Install CEF AMA for syslog:
   • From the CEF AMA connector in syslog get this command and run it on your syslog server to install the syslog component:
   • sudo wget -O Forwarder_AMA_installer.py https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/DataConnectors/Syslog/Forwarder_AMA_installer.py&&sudo python3 Forwarder_AMA_installer.py
   • Verify: Download the cef troubleshooter and run it:
   • sudo wget -O cef_AMA_troubleshoot.py https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/DataConnectors/CEF/cef_AMA_troubleshoot.py&&sudo python cef_AMA_troubleshoot.py
   • Verify: Generate a test CEF message:
   • echo -n "<164>CEF:0|Mock-test|MOCK|common=event-format-test|end|TRAFFIC|1|rt=$common=event-formatted-receive_time" | nc -u -w0 localhost 514

In Sentinel, verify the CEF logs are parsing by querying the CommonSecurityLog table:

References:

https://learn.microsoft.com/en-us/azure/sentinel/connect-cef-ama

https://learn.microsoft.com/en-us/azure/sentinel/forward-syslog-monitor-agent#configure-linux-syslog-daemon

https://learn.microsoft.com/en-us/azure/sentinel/data-connectors/common-event-format-cef-via-ama