Show search and navigation.

 

Real-Time Log Delivery (RTLD) Frequently Asked Questions

Applies To:
HTTP LargeThis platform has been optimized to cache and deliver static content (e.g., HTML, CSS, JavaScript, ISO, multimedia, and software downloads, etc.) over HTTP or HTTPS.
HTTP SmallLegacy. If you are currently serving traffic over this platform, then you may continue to do so. However, we recommend that you serve your traffic over our more robust HTTP Large platform.
ADNThe Application Delivery Network platform has been optimized to deliver dynamic content (e.g., login credentials, account information, etc.) over HTTP or HTTPS. Typically, user-specific and database-driven content are served over this platform.
Real-Time Log DeliveryDelivers near real-time log data to your web server, an AWS S3 bucket, an Azure Block Blob, Splunk Enterprise, Sumo Logic, or Datadog.
ClosedWhat is Real-Time Log Delivery?

RTLD delivers log data in near real-time to a variety of destinations. It consists of two modules, which are:

  • Real-Time Log Delivery CDN

    Delivers log data that describes requests submitted to our CDN service.

    This feature must be purchased separately. For more information, please contact your CDN account manager.

  • Real-Time Log Delivery Rate Limiting

    Delivers log data that describes requests for which Web Application Firewall (WAF) enforced a rate limit as defined through a rate rule.

    RTLD Rate Limiting requires WAF Premier, WAF Standard, or WAF Essentials. If you currently have WAF Insights and would like to use this capability, please contact your CDN account manager to upgrade to the full version.

  • Real-Time Log Delivery WAF

    Delivers log data that describes requests identified as threats by Web Application Firewall (WAF).

    RTLD WAF requires WAF Premier, WAF Standard, or WAF Essentials. If you currently have WAF Insights and would like to use this capability, please contact your CDN account manager to upgrade to the full version.

    RTLD WAF delivers log data for threats identified by WAF. It excludes log data for rate limited requests as determined by rate rules. Use RTLD Rate Limiting to deliver log data for rate limited requests.S

RTLD delivers compressed log data to one or more of the following destination(s):

ClosedWhy can't I find Real-Time Log Delivery CDN?

Load the Real-Time Log Delivery CDN page from the main menu by navigating to More, finding Real-Time Log Delivery under Analytics, and then selecting CDN.

If this menu item is not present, then check the following items:

  • Verify that Real-Time Log Delivery CDN has been activated on your account. For more information, please contact your CDN account manager.
  • Contact your CDN administrator to verify that you have sufficient permissions.
ClosedWhy can't I find Real-Time Log Delivery Rate Limiting?

Load the Real-Time Log Delivery Rate Limiting page from the main menu by navigating to More, finding Real-Time Log Delivery under Analytics, and then selecting RL.

If this menu item is not present, then check the following items:

  • Verify that Real-Time Log Delivery Rate Limiting has been activated on your account. This feature requires Web Application Firewall (WAF). For more information, please contact your CDN account manager.
  • Contact your CDN administrator to verify that you have sufficient permissions.
ClosedWhy can't I find Real-Time Log Delivery WAF?

Load the Real-Time Log Delivery WAF page from the main menu by navigating to More, finding Real-Time Log Delivery under Analytics, and then selecting WAF.

If this menu item is not present, then check the following items:

  • Verify that Real-Time Log Delivery WAF has been activated on your account. This feature requires Web Application Firewall (WAF). For more information, please contact your CDN account manager.
  • Contact your CDN administrator to verify that you have sufficient permissions.
ClosedMay I use Real-Time Log Delivery CDN to verify billing data?

Yes. However, you cannot filter or downsample log data if you plan to use it for billing verification.

ClosedMay I deliver log data to an existing AWS S3 bucket?

Yes. However, a specific bucket policy must be applied to it.

Learn more.

ClosedMay I deliver log data to an existing Azure Blob container?

Yes. However, you must authorize log data uploads via either a SAS token or an access key.

Learn more.

ClosedMay I deliver log data to an existing Google Cloud Storage bucket?

Yes. However, you must authorize our Google Cloud Storage user to upload log data.

Learn more.

ClosedMay I securely deliver log data to my web server?

Yes. Authorize log data delivery via the Authorization request header by passing a token or user account credentials. Alternatively, log data may be delivered to your web server(s) without authorization.

Learn more.

ClosedMay I limit the amount of log data delivered?

Yes. Log data delivery may be limited by:

  • Filtering the set of log data that will be delivered.

    • RTLD CDN: Filter log data by:

      Delivery Platform | Status Code | Edge CNAME | User Agent

    • RTLD Rate Limiting: Filter log data by:

      Security Application Manager | Rate Rule | Edge CNAME | Country | User Agent | Enforcement Action | Request Method | URL

    • RTLD WAF: Filter log data by:

      Security Application Manager | Access Rule | Custom Rule | Managed Rule | Edge CNAME | Country | User Agent

  • Downsampling logs to 0.1%, 1%, 25%, 50%, or 75% of the set of log entries that will be delivered.

    Example:

    Downsampling 1 million log entries to 1% results in 10,000 log entries.

    1,000,000 * 0.01 = 10,000
  • Selecting the type of data (i.e., fields) that will be delivered.
ClosedHow should I configure my firewall to allow log delivery?

Firewall configuration is only required when delivering log data to either your web server(s) or an instance of Splunk Enterprise hosted within your network. Set up your firewall to allow POST requests from the following CIDR network addresses:

198.7.21.0/24

If you plan to deliver log data via a custom port, then you should also configure your firewall to open that port for the above IP blocks.

ClosedWhat happens if log data cannot be delivered?

If our service is unable to deliver log data, then we will store it for up to 3 days and deliver it when communication resumes. If we cannot deliver log data within 3 days, then it will be permanently deleted.

ClosedCan the same log data be sent multiple times because of multiple profiles?

Yes. Each profile is:

  • Independent.
  • Defines a set of log data.
  • Determines where data will be delivered.

This means that more than one profile may be configured to deliver the same set of log data.

ClosedCan I mix and match log data from RTLD CDN, RTLD Rate Limiting, and RTLD WAF?

No. RTLD CDN, RTLD Rate Limiting, and RTLD WAF use separate logging mechanisms.

ClosedDoes RTLD deliver compressed log data?

Yes. RTLD compresses all log data using gzip.

Key information:

  • Web Server Log Delivery: You should decompress log data before transforming it. Your web framework may automatically handle compression on your behalf.
  • AWS S3, Azure Blob Storage, and Google Cloud Storage Log Delivery: Each object contains compressed log data with a gz file extension.
  • All Other Destinations: Your third-party data analytics platform manages compression on your behalf. No additional action is required.
ClosedWill RTLD CDN log traffic from my Staging environment?

Yes. RTLD CDN logs traffic from both your Production and Staging environments.

Learn more about environments.

ClosedWhy do I not see data for custom log fields?

RTLD CDN allows you to log data for custom fields through the following options:

Custom Request Headers | Custom Response Headers | Custom Cookies

Our CDN may take longer to propagate custom log fields as opposed to our standard log fields. Prior to propagation, RTLD CDN may return an empty value for custom log fields.

Although other settings take effect quickly, it may take up to 90 minutes before data for custom request/response headers and cookies is logged.

Example (JSON):

"resp_hdr_user_location": "",

Example (CSV):

..."",...