Setting up Splunk Enterprise Log Delivery

RTLD may automatically deliver compressed log data to Splunk Enterprise by submitting HTTPS POST requests to it. The Splunk HTTP Event Collector (HEC) will collect and log each request. Each request contains a compressed JSON document that describes one or more log entries.

Learn more: RTLD CDN | RTLD Rate Limiting | RTLD WAF

The format for log data delivered to Splunk Enterprise is JSON Lines. This log format does not provide information that uniquely identifies a set of log data. As a result, there is no way to check for gaps in sequence numbers when attempting to identify missing log data.

To create a log delivery profile

  1. Set up Splunk Enterprise's HTTP Event Collector to accept CDN log data in JSON format.

    1. Verify your Splunk Enterprise 7.x setup.

      • Your instance of Splunk Enterprise 7.x must be secured with SSL.
      • SSL must be enabled on the HTTP Event Collector.

      For information on how to set up Splunk Enterprise, please refer to their documentation.

    2. From with Splunk Enterprise, click Settings and then Add Data.

    3. Click Monitor.

    4. Click HTTP Event Collector.

    5. In the Name option, define a name for the CDN log data that will be collected.

    6. Click Next >.
    7. Click Select to display the Select Source Type option. Click that option, type "_json" to filter source types, and then select it.

    8. Click Review.
    9. Click Submit > to finish setting up the HTTP Event Collector. An HEC token will be generated. Use this token to authorize requests posted to the HEC.
  2. Perform the following steps if you have hosted Splunk Enterprise within your network:

    1. Configure your firewall to allow POST requests from the following IP blocks:

    2. If you plan to deliver log data via a custom port, then you should also configure your firewall to allow requests on that port.

    3. Set up support for the HTTPS protocol.

      Log delivery requires a certificate whose trust anchor is a publicly trusted certificate authority (CA). Additionally, the certificate must include a chain of trust for all intermediate certificate(s) and a leaf certificate.

  3. Navigate to the Real-Time Log Delivery CDN | Rate Limiting | WAF page. From the main menu, navigate to More and then find Real-Time Log Delivery under Analytics. Select either CDN, WAF, or RL.

  4. Click Add Profile.
  5. From the Log Delivery Method option, select Splunk Enterprise.
  6. Set the Splunk URL option to a URL that points to your Splunk Enterprise's HTTP Event Collector configuration.

    Default URL syntax:

  7. Set the HEC Token option to the token generated for the HTTP Event Collector configuration created in step 1.

  8. From the Downsample the Logs option, determine whether all or downsampledReduces the amount of log data that will be delivered. For example, you may choose to only deliver 1% of your log data. log data will be delivered.

    • All Log Data: Verify that the Downsample the Logs option is disabled.
    • Downsampled Log Data: Downsample logs to 0.1%, 1%, 25%, 50%, or 75% of total log data by enabling the Downsample the Logs option and then selecting the desired rate from the Downsampling Rate option.

      Use this capability to reduce the amount of data that needs to be processed or stored by Splunk Enterprise.
      RTLD CDN Only: Downsampling log data also reduces usage charges for this service.

  9. Log delivery setup varies according to whether you are delivering log data for CDN traffic, threats identified by WAF, or rate limited requests.

  10. Set the Log Delivery Enabled option to the "on" position.

  11. Click Save.

To test connectivity

  1. From the Splunk Enterpise's home page, click Search & Reporting.
  2. Observe the What to Search section. Log data pushed from the CDN to Splunk Enterprise will be reported as events in this section.
More Information