About the sample CEF connector
The sample CEF connector receives security events in near real time using the SIEM OPEN API and converts them from JSON into CEF format. The connector pulls from one single API endpoint based on a pull rate you can configure and processes security events generated from security policies within multiple security configurations.
You can configure the connector to save security events locally in addition to forwarding to a destination host over UDP or TCP using Syslog protocol. For Syslog, the connector leverages CEF format. CEF is an open messaging standard introduced by ArcSight, Inc.
The application outputs data over the Syslog protocol using CEF data format in order to provide a flexible solution that meets the requirements of various SIEM tools.
CEF-Syslog works with HP ArcSight and other SIEM solutions that support syslog/CEF format.
Install CEF connector
System requirements
Akamai’s CEF Connector requires Sun JRE 1.8+ to be installed. The latest JRE can be downloaded from the Sun Java site (Java Platform, Standard Edition) or installed from a software distribution package on Linux.
Verify Java version (Linux): To verify that your JRE is installed and available, use the command java -version
Hardware requirements
This application is designed to run on a Linux server with at least:
- 2 CPU cores
- 6GB RAM
- 2GB Free Disk Space
- Run a Linux Kernel greater than 2.6
Proxy server
To access the SIEM API from behind a proxy server, ensure that your proxy:
- allowlists the domains
*.cloudsecurity.akamaiapis.net
and*.luna.akamaiapis.net
- does not interfere with HTTP request headers for those domains. If, due to a strict enterprise security policy, your proxy does change these headers, make sure that at a minimum you allow and don't change the
Host
andAuthorization
headers.
Install
Visit https://developer.akamai.com/tools/siem-integration to get the latest CEFConnector distribution package. Transfer the package using either Linux commandwget http://[server]/CEFConnector-1.0.zip
(replace [server]
with your specific server name) or using SFTP (SSH File Transfer Protocol).
Unzip the distribution package anywhere on the file system. You can install Unzip from a software distribution package on Linux (for example yum install unzip
).
To install a service, create a symbolic link to bin/AkamaiCEFConnector.sh
shell script in /etc/init.d
. You can execute the shell script with the following commands: start | stop | status | resetdb
. Resetdb
deletes cefconnector.db
which contains the last successful offset data pull. Removing the file causes the connector to process offset=NULL
as long as timebased
setting is false
. If timebased
is true, a new offset is saved after the first successful pull. Learn more about offset and time-based pulls.
Configure your SIEM event data pull
Set up and manage your connector through the source pull configuration settings located in the config/CEFConnector.properties file.
- connector.refresh.period
- Default Value: 60
- Description: Defines the pull rate from the Akamai source in seconds.
- Allowed Values:Positive integer value greater than 0. No decimals allowed. Unallowed values will default.
- Example: 60 (1 minute)
- Required: Yes
- akamai.data.requesturlhost
- Description: Akamai web service URL (example https://cloudsecurity.akamaiapis.net) This along with the security configurations ids will be used to generate the request url (https://cloudsecurity.akamaiapis.net/siem/v1/configs/14227?offset=NULL).
- Allowed Values: This value cannot be blank or commented out.
- Example:https://cloudsecurity.akamaiapis.net
- Required: Yes
- akamai.data.configs
- Description: Add multiple security configuration IDs by separating them with a semicolon";".
- Example: 12345:67890
- Required: Yes
- akamai.data.timebased
- Description: Set to
false
to pull security events continuously by offset token (this should be the usual, everyday setting) and set totrue
in order to pull security events that fall within a specific time frame. Iftimebased
is set totrue
, thefrom
parameter (see next configuration entry) is required andto
is optional. Learn more about retrieving past events. - Allowed Values: true or false
- Example: false
- Required: Yes
- Description: Set to
- akamai.data.timebased.from
- Description: If
timebased
parameter is true, this a required field. Enter the time (in epoch format) to start pulling past security events. This can be any time within the 12 hours preceding the present moment. If regular offset event collection occurred within the time window, you may see duplicate data in your SIEM software. - Allowed Values: Positive integer in epoch format. Value has to be less than
timebased.to
. - Example: 1491588763
- Required: Yes, if
timebased
parameter istrue
, No, iftimebased
parameter isfalse
.
- Description: If
- akamai.data.timebased.to
- Default Value: No value
- Description: If
timebased
parameter is true, you can optionally enter the end timestamp (in epoch format) to a set period for which to pull past security events. If no value or invalid format is provided, default value is used. - Allowed Values: Positive integer in epoch format. Value must be greater than or equal to the value in
timebased.from
. - Example: 1491588763
- Required: No
- akamai.data.limit
- Default Value: 200000
- Description: Limit the number of security events to pull. Use this field to prevent overload and protect your SIEM system if it’s suddenly flooded with event data. If no value is provided or an invalid value is provided, the default limit defined by the SIEM OPEN API side is used.
- Allowed Values: Positive integer values. No decimals allowed. Unallowed values will default.
- Example: 100
- Required: No
Note: This sample connector supports transfer of up to 200,000 events per minute.
- akamai.data.accesstoken
- Description: Enter the access token you copied when you provisioned the SIEM OPEN API in Akamai’s LUNA portal.
- Allowed Values: This value cannot be blank or commented out.
- Example: akab-fc7lrkvv57lgxjpx-mizhsadsasdasym
- Required: Yes
- akamai.data.clienttoken
- Description: Enter the client token you copied when you provisioned the SIEM OPEN API in Akamai’s LUNA portal.
- Allowed Values: This value cannot be blank or commented out.
- Example: akab-fc7lrkvv57lgxjpx-mizhsadsasdasym
- Required: Yes
- akamai.data.clientsecret
- Description: Enter the client secret you copied when you provisioned the SIEM OPEN API in Akamai’s LUNA portal.
- Allowed Values:This value cannot be blank or commented out.
- Example: LWcGK6h2121GdfdDSD8m+4wllsdfdsG8wgFS+dfDfZQ=
- Required: Yes
- akamai.data.baseurl
- Description: Enter the URL you copied when you provisioned the SIEM OPEN API in Akamai’s LUNA portal.
- Allowed Values: This value cannot be blank or commented out.
- Example: akab-dsfdsdypmwkj7a-eqkdfgfdsswsfaoclec.cloudsecurity.akamaiapis.net
- Required: Yes
- akamai.cefformatheader
- Description: CEF Header Values are separated by
|
. If|
is part of a static string, then it must be escaped with\
. Values can be static or - generated from available functions: requestURL(), eventClassId(), name(), severity(), appliedAction(),ipv6src()
- Allowed Values: This value can not be blank or commented out.
- Example: CEF:0|Akamai|akamai_siem|1.0|eventClassId()|name()|severity()
- Required: Yes
- Description: CEF Header Values are separated by
- akamai.cefformatextension
- Description: CEF Extension Values are separated by a space. Values can be static, generated from available functions (
eventClassId()
,name()
,severity()
,appliedAction()
,ipv6src()
) or pulled from JSON Api. JSON api is defined by${}
and each json object is separated by a period. Static Values are defined by quotation marks. Function-generated values are defined by () and must be one of the available functions defined in documentation. - Allowed Values: This value can not be blank or commented out.
- Example: act=appliedAction() app=${httpMessage.protocol}c6a2=ipv6src()c6a2Label="Source IPv6 Address"
- cs1=${attackData.rules} cs1Label="Rules"
- cs2=${attackData.ruleMessages} cs2Label="Rule Messages"
- cs3=${attackData.ruleData} cs3Label="Rule Data"
- cs4=${attackData.ruleSelectors} cs4Label="Rule Selectors"
- Required: No
- Description: CEF Extension Values are separated by a space. Values can be static, generated from available functions (
- akamai.base64fields
- Description: If an API json object is base64 encoded, it must be defined here.
- Example: ${attackData.ruleVersions};${attackData.rules};${attackData.ruleActions}
- Required: No
- akamai.urlencoded
- Description: If an API json object is URL-encoded, it must be defined here.
- Example: ${attackData.ruleVersions};${attackData.rules};${attackData.ruleActions}
- Required: No
- akamai.multivaluedelim
- Default Value: , (comma)
- Description: Delimiter used to separate multi valued CEF fields Or commented out.
- Allowed Values: " " is treated the same as "" and a blank will default.
- Example:, (comma), \u0020 (white space), \n (newline), \t (tab)
- Required: No
- connector.consumer.count
- Default Value: 3
- Description: Limit the number of consumer threads
- Example: 10
- Required: No
- connector.proxy.host
- connector.proxy.port
- Description: Set your proxy host and port
- Allowed Values: This value should be left blank if you don't have a local proxy
- Required: No
Set up in Arcsight
- Log in to Arcsight with administrator rights.
- From the menu choose Configuration > Data > Receivers.
- Add a new TCP CEF Receiver with a distinct name like Akamai CEF Data. By default, this assigns the receiver to listen on all IP/HOST, port 546 (if not already in use), UTF-8 encoding, and source type CEF. The receiver is enabled by default. The port can be altered if needed.
- To group security events by device, go to the menu and choose Configuration > Data > Device Groups.
- Add a new Device Group and pick from a list of auto-populated devices. If the connector is installed locally and is sending logs to localhost, the device is Logger Internal Event Device with the receiver name you added in Step 3.
- To view the device group security logs, go to the menu and choose Analyze > Search. In the search bar enter
_deviceGroup in ["<device group name>"]
. Example:_deviceGroup in ["Akamai SIEM -On Server Connector"]
Logging
You can set log configurations in the config/log4j2.xml
file.
- log-path: Path location of file logs. Use relative or specific path (example:
logs
). - log-name: File name for logs (example:
filename
). - SizeBasedTriggeringPolicy: Log size rollover limit (example:
1MB
). - DefaultRolloverStrategy: Max number of log files stored for each log (info, warn, and error) on disk (example:
20
). - CEFHost: Remote CEF Syslog Server Host (example:
127.0.0.1
). - CEFPort: Remote CEF Syslog Server Port (example:
514
). - CEFProtocol: Remote CEF Syslog Server Protocol (example:
UDP/TCP
).
To run in debug mode:
- In the
log4j2.xml
file, scroll down to the bottom and under<Loggers>
find<Root level="info"
- Change
info
todebug
and save the file.
Find logs in the /bin/logs/cefconnector.log
file.
SIEM API data format for CEF
The security events data available from the Akamai endpoint arrives in JSON format. Each line is a separate JSON object representing a single request.
Calculated fields
- $appliedAction
- If attackData.ruleActions contains any action other than 'alert' or 'deny', use it (there will be only one) otherwise, check if there is a 'deny' action and if so, set the applied action to 'deny' otherwise, check if attackData.slowPostAction equals to 'A', set the applied action to 'abort' otherwise, set the applied action to 'alert'
- $eventClassId
- If $appliedAction in ('alert', 'monitor') then 'detect' otherwise, 'mitigate'
- $name
- If $eventClassId = 'detect' then 'Activity detected' otherwise 'Activity mitigated'
- $severity
- If $eventClassId = 'detect' then 5 otherwise 10
- $requestURL
-
If $httpMessage.tls is not empty then request = 'https://' + $httpMessage.host otherwise request = 'http://' + $httpMessage.host
If $httpMessage.path is not empty then request = $request + $attackData.path if $httpMessage.query is not empty then request = $request + '?" + $httpMessage.query
CEF fields
CEF Field | JSON Field/Value | Description |
Device Vendor | Akamai | |
Device Version | 1.0 | |
Device Product | akamai_siem | |
Device Event Class ID | $eventClassId | calculated field |
Name | $name | calculated field |
Severity | $severity | calculated field |
CEF Extension fields
CEF Extension | JSON Field/Value | Description |
act | $appliedAction | calculated field |
app | $httpMessage.protocol | |
c6a2 | $ipv6src | IP v6 address of the source. Only populated if $attackData.clientIP is in IP v6 format. |
c6a2Label | Source IP6 Address | |
cs1 | $attackData.rules | Rule IDs of rules that triggered for this request |
cs1Label | Rules | |
cs2 | $attackData.ruleMessages | Messages of rules that triggered for this request |
cs2Label | Rule Messages | |
cs3 | $attackData.ruleData | User data of rules that triggered for this request |
cs3Label | Rule Data | |
cs4 | $attackData.ruleSelectors | Selectors of rules that triggered for this request |
cs4Label | Rule Selectors | |
cs5 | $attackData.clientReputation | Client IP scores for Client Reputation |
cs5Label | Client Reputation | |
cs6 | $attackData.apiId | API ID for API Protection |
cs6Label | API ID | |
devicePayloadId | $httpMessage.requestId | A globally unique ID created to identify this specific message |
dhost | $httpMessage.host | The value of the host header of the incoming client request |
dpt | $httpMessage.port | The port number used by the incoming request. Should be equal to the value of AK_IN_PORT |
flexString1 | $attackData.configId | The ID of the Security Configuration applied to this request |
flexString1Label | Security Config Id | |
flexString2 | $attackData.policyId | This is the id of the Firewall Policy applied to this request |
flexString2Label | Firewall Policy Id | |
out | $httpMessage.bytes | The content bytes served int eh client response |
request | $requestURL | calculated field |
requestMethod | $httpMessage.method | The HTTP method of the incoming request |
src | $attackData.clientIP | The IP address of the client that connects to make the request |
start | $httpMessage.start | This is the time, in epoch format, to millisecond precision, when the Edge Server initiated the connection for the message exchange being monitored |
AkamaiSiemSlowPostAction | $attackData.slowPostAction | If a Slow POST attack is detected, this shows the action taken: either W for Warn or A for deny (abort) |
AkamaiSiemSlowPostRate | $attackData.slowPostRate | If a Slow POST attack is detected, this shows the recorded rate of the Slow POST attack |
AkamaiSiemRuleVersions | $attackData.ruleVersions | Versions of rules that triggered for this request, base64-encoded |
AkamaiSiemRuleTags | $attackData.ruleTags | Tags of rules that triggered for this request, base64-encoded. See WAF rule list for all tags |
AkamaiSiemApiKey | $attackData.apiKey | API Key for API Protection |
AkamaiSiemTLSVersion | $httpMessage.tls | TLS version, if applicable |
AkamaiSiemRequestHeaders | $httpMessage.requestHeaders | All request headers collected |
AkamaiSiemResponseHeaders | $httpMessage.responseHeaders | All response headers collected |
AkamaiSiemResponseStatus | $httpMessage.status | The HTTP Response status sent to the client |
AkamaiSiemContinent | $geo.continent | A 2-letter code for the continent that the IP address maps to |
AkamaiSiemCountry | $geo.country | An ISO-3166, 2-letter code for the state, province, or region where the IP address maps to |
AkamaiSiemCity | $geo.city | City that the IP address maps to |
AkamaiSiemRegion | $geo.regionCode | An ISO-3166, 2-letter code for the state, province, or region where the IP address maps to |
AkamaiSiemASN | $geo.asn | AS number or numbers that the IP belongs to |
AkamaiSiemCusomData | $custom | Base64 custom formatted value. Size limit is 2KB |
Retrieve past security events using the CEF connector
Akamai’s CEF connector offers 2 modes of operation:
-
Offset-based. Under normal circumstances, this is the mode you want. The connector automatically logs security events as they’re collected. The connector operates in offset mode when
akamai.data.timebased
configuration is set tofalse
. - Time-based. If your SIEM connection is disrupted, you may want to go back and replay events. You can retrieve security event data that occurred within the last 12 hours.
To retrieve missing/past events, switch to a time-based feed:
- Open your connector’s configuration file:
config/CEFConnector.properties
. - Change the
akamai.data.timebased
configuration totrue
. - Under
akamai.data.timebased.from
, enter the start time (in epoch format) for which you want to pull past security events. This can be any time within the 12 hours preceding the present moment. - (optional) If you want to retrieve events that fall only within a set time period, under
akamai.data.timebased.to
, enter the end time (in epoch format) for the period. If you enter nothing here, the connector pulls events up to the present and continues to log events as they’re collected until the connector is restarted.
If regular offset event collection occurred within the time window you set, duplicate data may appear in your SIEM reports.
Don't see the data you expected? When you set akamai.data.timebased.from
and akamai.data.timebased.to
to retrieve security events for a specific period, the connector makes only 1 call to the API. If the number of events in the specified time window exceeds the value in the Limit field (or the default limit of 200,000) the connector won't retrieve data. As a workaround, decrease the time window to include all events.
To return the connector to offset mode, change akamai.data.timebased
configuration back to false
.
Uninstall CEF connector
Stop the Akamai CEF Connector service by running bin/AkamaiCEFConnector.sh stop
or /etc/init.d/symboliclink stop
Remove the installation folder by running rm –rf CEFConnector-1.0
Release notes
CEF Connector version 1.7.0 released in October 2020. Changes include:
- Support for new Open API domain *.luna.akamaiapis.net
CEF Connector version 1.5 released in August 2017. Changes include:
- Debug flag has moved out of
CEFConnector.properties
. To turn on logging, you now make a quick change to thelog4j2.xml
. To read how, see Logging. - All logs now consolidated in the
/bin/logs/cefconnector.log
file.