SIEM CEF connector

About the sample CEF connector

The sample CEF connector receives security events in near real time using the SIEM OPEN API and converts them from JSON into CEF format. The connector pulls from one single API endpoint based on a pull rate you can configure and processes security events generated from security policies within multiple security configurations.

You can configure the connector to save security events locally in addition to forwarding to a destination host over UDP or TCP using Syslog protocol. For Syslog, the connector leverages CEF format. CEF is an open messaging standard introduced by ArcSight, Inc.

The application outputs data over the Syslog protocol using CEF data format in order to provide a flexible solution that meets the requirements of various SIEM tools.

CEF-Syslog works with HP ArcSight and other SIEM solutions that support syslog/CEF format.

Install CEF connector

System requirements

Akamai’s CEF Connector requires Sun JRE 1.8+ to be installed. The latest JRE can be downloaded from the Sun Java site (Java Platform, Standard Edition) or installed from a software distribution package on Linux.

Verify Java version (Linux): To verify that your JRE is installed and available, use the command java -version

Hardware requirements

This application is designed to run on a Linux server with at least:

• 2 CPU cores
• 6GB RAM
• 2GB Free Disk Space
• Run a Linux Kernel greater than 2.6

Proxy server

To access the SIEM API from behind a proxy server, ensure that your proxy:

• whitelists the domains *.cloudsecurity.akamaiapis.net
• does not interfere with HTTP request headers for those domains. If, due to a strict enterprise security policy, your proxy does change these headers, make sure that at a minimum you allow and don't change the Host and Authorization headers.

Install

Visit https://developer.akamai.com/tools/siem-integration to get the latest CEFConnector distribution package. Transfer the package using either Linux commandwget http://[server]/CEFConnector-1.0.zip (replace [server] with your specific server name) or using SFTP (SSH File Transfer Protocol).

Unzip the distribution package anywhere on the file system. You can install Unzip from a software distribution package on Linux (for example yum install unzip ).

To install a service, create a symbolic link to bin/AkamaiCEFConnector.sh shell script in /etc/init.d . You can execute the shell script with the following commands: start | stop | status | resetdb . Resetdb deletes cefconnector.db which contains the last successful offset data pull. Removing the file causes the connector to process offset=NULL as long as timebased setting is false . If timebased is true, a new offset is saved after the first successful pull. Learn more about offset and time-based pulls.

Configure your SIEM event data pull

Set up and manage your connector through the source pull configuration settings located in the config/CEFConnector.properties file.

• connector.refresh.period
  • Default Value: 300
  • Description: Defines the pull rate from the Akamai source in seconds.
  • Allowed Values:Positive integer value greater than 0. No decimals allowed. Unallowed values will default.
  • Example: 60 (1 minute)
  • Required: Yes
• akamai.data.requesturlhost
  • Description: Akamai web service URL (example https://cloudsecurity.akamaiapis.net) This along with the security configurations ids will be used to generate the request url (https://cloudsecurity.akamaiapis.net/siem/v1/configs/14227?offset=NULL).
  • Allowed Values: This value cannot be blank or commented out.
  • Example:https://cloudsecurity.akamaiapis.net
  • Required: Yes
• akamai.data.configs
  • Description: Add multiple security configuration IDs by separating them with a semicolon";".
  • Example: 12345:67890
  • Required: Yes
• akamai.data.timebased
  • Description: Set to false to pull security events continuously by offset token (this should be the usual, everyday setting) and set to true in order to pull security events that fall within a specific time frame. If timebased is set to true , the from parameter (see next configuration entry) is required and to is optional. Learn more about retrieving past events.
  • Allowed Values: Positive integer value greater than 0. No decimals allowed. Unallowed values will default.
  • Allowed Values: true or false
  • Example: false
  • Required: Yes
• akamai.data.timebased.from
  • Description: If timebased parameter is true, this a required field. Enter the time (in epoch format) to start pulling past security events. This can be any time within the 12 hours preceding the present moment. If regular offset event collection occurred within the time window, you may see duplicate data in your SIEM software.
  • Allowed Values: Positive integer in epoch format. Value has to be less than timebased.to .
  • Example: 1491588763
  • Required: Yes, if timebased parameter is true , No, if timebased parameter is false .
• akamai.data.timebased.to
  • Default Value: No value
  • Description: If timebased parameter is true, you can optionally enter the end timestamp (in epoch format) to a set period for which to pull past security events. If no value or invalid format is provided, default value is used.
  • Allowed Values: Positive integer in epoch format. Value must be greater than or equal to the value in timebased.from .
  • Example: 1491588763
  • Required: No
• akamai.data.limit
  • Default Value: No value
  • Description: Limit the number of security events to pull. Use this field to prevent overload and protect your SIEM system if it’s suddenly flooded with event data. If no value is provided or an invalid value is provided, the default limit defined by the SIEM OPEN API side is used.
  • Allowed Values: Positive integer values. No decimals allowed. Unallowed values will default.
  • Example: 100
  • Required: No
• akamai.data.accesstoken
  • Description: Enter the access token you copied when you provisioned the SIEM OPEN API in Akamai’s LUNA portal.
  • Allowed Values: This value cannot be blank or commented out.
  • Example: akab-fc7lrkvv57lgxjpx-mizhsadsasdasym
  • Required: Yes
• akamai.data.clienttoken
  • Description: Enter the client token you copied when you provisioned the SIEM OPEN API in Akamai’s LUNA portal.
  • Allowed Values: This value cannot be blank or commented out.
  • Example: akab-fc7lrkvv57lgxjpx-mizhsadsasdasym
  • Required: Yes
• akamai.data.clientsecret
  • Description: Enter the client secret you copied when you provisioned the SIEM OPEN API in Akamai’s LUNA portal.
  • Allowed Values:This value cannot be blank or commented out.
  • Example: LWcGK6h2121GdfdDSD8m+4wllsdfdsG8wgFS+dfDfZQ=
  • Required: Yes
• akamai.data.baseurl
  • Description: Enter the URL you copied when you provisioned the SIEM OPEN API in Akamai’s LUNA portal.
  • Allowed Values: This value cannot be blank or commented out.
  • Example: akab-dsfdsdypmwkj7a-eqkdfgfdsswsfaoclec.cloudsecurity.akamaiapis.net
  • Required: Yes
• akamai.cefformatheader
  • Description: CEF Header Values are separated by | . If | is part of a static string, then it must be escaped with \ . Values can be static or
  • generated from available functions: requestURL(), eventClassId(), name(), severity(), appliedAction(),ipv6src()
  • Allowed Values: This value can not be blank or commented out.
  • Example: CEF:0|Akamai|akamai_siem|1.0|eventClassId()|name()|severity()
  • Required: Yes
• akamai.cefformatextension
  • Description: CEF Extension Values are separated by a space. Values can be static, generated from available functions ( eventClassId() , name(), severity() , appliedAction() , ipv6src() ) or pulled from JSON Api. JSON api is defined by ${} and each json object is separated by a period. Static Values are defined by quotation marks. Function-generated values are defined by () and must be one of the available functions defined in documentation.
  • Allowed Values: This value can not be blank or commented out.
  • Example: act=appliedAction() app=${httpMessage.protocol}c6a2=ipv6src()c6a2Label="Source IPv6 Address"
  • cs1=${attackData.rules} cs1Label="Rules"
  • cs2=${attackData.ruleMessages} cs2Label="Rule Messages"
  • cs3=${attackData.ruleData} cs3Label="Rule Data"
  • cs4=${attackData.ruleSelectors} cs4Label="Rule Selectors"
  • Required: No
• akamai.base64fields
  • Description: If an API json object is base64 encoded, it must be defined here.
  • Example: ${attackData.ruleVersions};${attackData.rules};${attackData.ruleActions}
  • Required: No
• akamai.urlencoded
  • Description: If an API json object is URL-encoded, it must be defined here.
  • Example: ${attackData.ruleVersions};${attackData.rules};${attackData.ruleActions}
  • Required: No
• akamai.multivaluedelim
  • Default Value: , (comma)
  • Description: Delimiter used to separate multi valued CEF fields Or commented out.
  • Allowed Values: " " is treated the same as "" and a blank will default.
  • Example:, (comma), \u0020 (white space), \n (newline), \t (tab)
  • Required: No
• connector.consumer.count
  • Default Value: 3
  • Description: Limit the number of consumer threads
  • Example: 10
  • Required: No

Set up in Arcsight

1. Log in to Arcsight with administrator rights.
2. From the menu choose Configuration > Data > Receivers.
3. Add a new TCP CEF Receiver with a distinct name like Akamai CEF Data. By default, this assigns the receiver to listen on all IP/HOST, port 546 (if not already in use), UTF-8 encoding, and source type CEF. The receiver is enabled by default. The port can be altered if needed.
4. To group security events by device, go to the menu and choose Configuration > Data > Device Groups.
5. Add a new Device Group and pick from a list of auto-populated devices. If the connector is installed locally and is sending logs to localhost, the device is Logger Internal Event Device with the receiver name you added in Step 3.
6. To view the device group security logs, go to the menu and choose Analyze > Search. In the search bar enter _deviceGroup in ["<device group name>"] . Example: _deviceGroup in ["Akamai SIEM -On Server Connector"]

Logging

You can set log configurations in the config/log4j2.xml file.

• log-path: Path location of file logs. Use relative or specific path (example: logs).
• log-name: File name for logs (example: filename).
• SizeBasedTriggeringPolicy: Log size rollover limit (example: 1MB).
• DefaultRolloverStrategy: Max number of log files stored for each log (info, warn, and error) on disk (example: 20).
• CEFHost: Remote CEF Syslog Server Host (example: 127.0.0.1).
• CEFPort: Remote CEF Syslog Server Port (example: 514).
• CEFProtocol: Remote CEF Syslog Server Protocol (example: UDP/TCP).

To run in debug mode:

1. In the log4j2.xml file, scroll down to the bottom and under <Loggers> find <Root level="info"
2. Change info to debug and save the file.

Find logs in the /bin/logs/cefconnector.log file.

SIEM API data format for CEF

The security events data available from the Akamai endpoint arrives in JSON format. Each line is a separate JSON object representing a single request.

Calculated fields

$appliedAction

If attackData.ruleActions contains any action other than 'alert' or 'deny', use it (there will be only one) otherwise, check if there is a 'deny' action and if so, set the applied action to 'deny' otherwise, check if attackData.slowPostAction equals to 'A', set the applied action to 'abort' otherwise, set the applied action to 'alert'

$eventClassId

If $appliedAction in ('alert', 'monitor') then 'detect' otherwise, 'mitigate'

$name

If $eventClassId = 'detect' then 'Activity detected' otherwise 'Activity mitigated'

$severity

If $eventClassId = 'detect' then 5 otherwise 10

$requestURL

If $httpMessage.tls is not empty then request = 'https://' + $httpMessage.host otherwise request = 'http://' + $httpMessage.host

If $httpMessage.path is not empty then request = $request + $attackData.path if $httpMessage.query is not empty then request = $request + '?" + $httpMessage.query

CEF fields

CEF Extension fields

Retrieve past security events using the CEF connector

Akamai’s CEF connector offers 2 modes of operation:

• Offset-based. Under normal circumstances, this is the mode you want. The connector automatically logs security events as they’re collected. The connector operates in offset mode when akamai.data.timebased configuration is set to false .
• Time-based. If your SIEM connection is disrupted, you may want to go back and replay events. You can retrieve security event data that occurred within the last 12 hours.

To retrieve missing/past events, switch to a time-based feed:

1. Open your connector’s configuration file: config/CEFConnector.properties .
2. Change the akamai.data.timebased configuration to true .
3. Under akamai.data.timebased.from , enter the start time (in epoch format) for which you want to pull past security events. This can be any time within the 12 hours preceding the present moment.
4. (optional) If you want to retrieve events that fall only within a set time period, under akamai.data.timebased.to , enter the end time (in epoch format) for the period. If you enter nothing here, the connector pulls events up to the present and continues to log events as they’re collected until the connector is restarted.

If regular offset event collection occurred within the time window you set, duplicate data may appear in your SIEM reports.

Don't see the data you expected? When you set akamai.data.timebased.from and akamai.data.timebased.to to retrieve security events for a specific period, the connector makes only 1 call to the API. If the number of events in the specified time window exceeds the value in the Limit field (or the default limit of 10,000) the connector won't retrieve data. As a workaround, decrease the time window to include all events.

To return the connector to offset mode, change akamai.data.timebased configuration back to false.

Uninstall CEF connector

Stop the Akamai CEF Connector service by running bin/AkamaiCEFConnector.sh stop or /etc/init.d/symboliclink stop

Remove the installation folder by running rm –rf CEFConnector-1.0

Release notes

CEF Connector version 1.5 released in August 2017. Changes include:

• Debug flag has moved out of CEFConnector.properties. To turn on logging, you now make a quick change to the log4j2.xml. To read how, see Logging.
• All logs now consolidated in the /bin/logs/cefconnector.log file.