Overview
Log streaming captures performance logs at the Edge in near real-time. You have access to either view all the available metrics and their related values, or choose which request-response cycle metrics you want to collect. We currently support Amazon S3 and Google Cloud Storage as log streaming endpoints.
This solution is in the Limited Availability phase, which means, current features and capabilities are subject to change and new features and fields may be added. Consequently, as we continue to make changes and add new features, this page is subject to updates.
This document contains details on the available log fields that we offer.
Support
Your Account or Sales representative can assist you in enabling any of the fields listed in the Optional Fields section below, or you may reach out to Support at hi@stackpath.com for assistance with this as well.
Depending on the sink you would like to stream logs to, Support will require that you provide the following information when requesting assistance with enabling logs or additional fields.
AWS S3
- Site name
- Desired fields
- AWS bucket name
- AWS bucket region
- Role ARN for an AWS IAM role
- This role must be associated with the StackPath AWS Account ID 517500695256
- This role must have a policy attached with write and list access to the bucket
GCS
- Site name
- Desired fields
- Bucket name (e.g my-stackpath-logs-bucket)
- Subdirectory to use (e.g if you want to use
my-stackpath-logs-bucket/cdn/accesslogs, the subdirectory iscdn/accesslogs)
For more details on the above requirements, please visit either Configuring a GCS Endpoint or Configuring an AWS S3 Endpoint.
Log Fields
Default Fields
The following fields are enabled by default and cannot be changed or removed:
|
Index |
Field |
Field Key |
Type |
Format |
Description |
|
-1 |
Site Hash |
sitehash |
string |
|
In json, this field will be at the top level and not part of "m1" |
|
-2 |
Box |
box |
string |
|
The box and service identifier |
|
-3 |
Request ID |
request_id |
string |
|
UUID request number |
|
-4 |
Sequence Number |
sequence number |
integer |
|
Incrementing sequence number. This number does not persist will begin at 1 on restart |
|
-5 |
Account Hash |
accounthash |
string |
|
The immediate parent account hash for the site |
|
-6 |
Account Hierarchy |
account-hierarchy |
json array |
|
The full list of account hierarchy |
Available Fields
The table below contains a list of all of the fields that are available for log streaming.
If you do not provide StackPath with a list of specified fields, then we will enable all of them for you by default.
|
Index |
Field |
Field Key |
Type |
Format |
Description |
|
1 |
Request Date |
date |
string |
YYYY-MM-DD |
The date the request was made to StackPath's network. |
|
2 |
Request Time |
time |
string |
HH:MM:SS |
The time represented in GMT when the request was made to StackPath's network. |
|
3 |
HTTP Method |
cs-method |
string |
string |
Request Method: The HTTP method used for the request (HEAD, GET). |
|
4 |
Client IP |
obfuscated-c-ip |
string |
a.b.c.d |
The Obfuscated IP address of the user agent making the request. |
|
5 |
HTTP Version |
cs-version |
string |
string |
Request Protocol: This should either be HTTP or HTTPS depending on how the content was requested on StackPath's network. |
|
6 |
Referrer |
cs-referrer |
string |
URL |
The referrer header sent by the user agent. A referrer is not a required header so this field may not always have a URL. |
|
7 |
User-Agent String |
cs-user-agent |
string |
string |
The string sent by the user agent to the identity itself. |
|
8 |
File Size |
filesize |
integer |
bytes |
Size of the asset being delivered. |
|
9 |
Request Size |
cs-bytes |
integer |
bytes |
The total size of the request header. Request Bytes (Header + Body) |
|
10 |
Response Bytes Sent |
sc-bytes |
integer |
bytes |
Bytes Sent: Total bytes in the response to the client. |
|
11 |
Web Server IP |
s-ip |
string |
a.b.c.d [port] |
Server IP: The IP address of the edge server - we default to the Anycast IP. |
|
12 |
Response Seconds |
time-taken |
float |
floating-point number with 3 decimal places |
The number of seconds (accurate to the millisecond) taken to deliver the asset. |
|
13 |
HTTP Status Code |
sc-status |
integer |
number |
Response Status: The HTTP status code returned to the user agent. This number should be a valid HTTP status code. |
|
14 |
Query String |
cs-uri-query |
string |
string |
The query string parameters provided in the request. |
|
15 |
Path |
cs-uri-stem |
string |
URI |
The full URI of the request. |
|
16 |
Range Bytes |
x-byte-range |
string |
range |
Request Range Header: For range request, you'll see the range requested and delivered to the client in the form bytes=<range requested>. |
|
17 |
Host Header |
host-header |
string |
The domain name, for example, example.com. |
|
|
19 |
Debug Header |
x-hw |
string |
|
X-HW Header |
|
20 |
Cache Hit Information |
cache-hit |
string |
HIT/MISS |
Indicates whether a request is hit. Valid values: HIT and MISS. |
|
21 |
Origin Pull Request ID |
op-id |
string |
|
Similar to Request ID but for Origin Pull request |
|
22 |
Client IP Address |
c-ip |
string |
|
The IP address of a client, for example, 34.26.127.44/240e:ff:a008:30:3::12. |
|
23 |
Client Port |
c-port |
integer |
|
The TCP port between a client and StackPath's network, for example, port 43245. |
|
24 |
Node Port |
server-port |
integer |
|
The server port used to provide external services, for example, port 80. |
|
25 |
Request Time |
unixtime |
integer |
|
The UNIX time in seconds, for example, 1562142849. |
|
26 |
Request Time in Milliseconds |
time-taken-ms |
integer |
|
The interval between the time when a request is received and the time when all responses are returned, in milliseconds. |
|
27 |
Response Bytes without Header |
sc-body-bytes-sent |
integer |
|
The number of bytes in an actual response body, excluding the HTTP header. |
|
28 |
Server Country |
country |
string |
XX |
The 2-alphabet code of the country where the request is served in upper caseases conforming to ISO 3166 standard https://www.iban.com/country-codes |
|
29 |
Time to First Byte |
response-fbt |
integer |
|
Time from processing to us putting bytes into the buffer |
|
30 |
Client Request XFF |
x-forwarded-for |
string |
|
Value of the request header X-Forwarded-For header |
|
31 |
Response Content Type |
response-content-type |
string |
|
The content of Header Content-Type in an HTTP response. Example: image/webp. If no value is available, this field is empty. |
|
32 |
Response Last Modified |
response-last-modified |
string |
|
The content of Header Last-Modified in an HTTP response, for example, Mon, 23 Jan 2017 13:27:36 GMT. If no value is available, this field is empty. |
|
33 |
Response Content Length |
response-content-length |
integer |
|
The value of the Header Content-Length field in a response. It denotes the exact byte length of the HTTP body. |
|
34 |
Response Content Range |
response-content-range |
string |
|
The range information (created by the origin site) in the response header, for example, bytes=0-99/200. When server supports range request, this field denotes where in the full resource this partial message belongs. |
|
35 |
Response Header Via-Info |
via-info |
string |
|
The content of the Via header in an HTTP response. If no value is available, this field is empty. |
|
36 |
POP Location |
pop-location |
string |
|
POP location |
|
37 |
POP Region |
pop-region |
string |
|
Region Location of the POP |
|
39 |
Origin Header |
origin-header |
string |
|
Origin Header from Request Header |
Log Format
Logs are sent to your repository in the JSON format shown below. The fields are comma delimited and structured in a "Field Key : Content" format, for example, "cs-method":"post". Each JSON file is a gzipped line delimited file with multiple objects.
Note, not all of the available fields are represented in this sample:
{"account-hierarchy":["s2s4e9p8","i9j2f5d8","m5e3w3b2"],"accounthash":"m5e3w3b2","attempted-bytes":2617,"box":"gfs-cds289.fr8.hwcdn.net","c-ip":"161.97.143.54","c-port":60492,"cache-hit":"MISS","country":"DE","cs-bytes":754,"cs-method":"POST","cs-origin":"","cs-referrer":"","cs-uri-query":"","cs-uri-stem":"/xmlrpc.php","cs-user-agent":"Nuclei
- Open-source project (github.com/projectdiscovery/nuclei)","cs-version":"http","date":"2022-01-27","filesize":1728,"host-header":"blog.stackpath.com:80","obfuscated-c-ip":"161.97.143.0","op-id":"05c83050-45b8-40ae-b024-ad16a9d131a3","pop-location":"fr8","pop-region":"eu","request-id":"f2d743e7-e2a8-4664-a3a3-97643a57a6e5","response-content-length":0,"response-content-range":"","response-content-type":"text/html;
charset=UTF-8","response-fbt":13,"response-last-modified":"","s-ip":"151.139.86.128","sc-body-bytes-sent":1728,"sc-bytes":2617,"server-port":80,"sitehash":"k2e9a3p5","time":"00:00:01","time-taken-ms":14,"unixtime":1643241601,"vendor-id":"3rdstackpath_eu","via-info":"","x-byte-range":"","x-forwarded-for":"","x-hw":"1643241601.cds004.fr8.h2,1643241601.cds289.fr8.sc,1643241601.cdn2-wafbe01-fra1.stackpath.systems.-.w,1643241601.cds289.fr8.p"}
Streaming CDN and WAF Logs
It is possible to use the same bucket for a site that has both CDN and WAF logging enabled, however, if you select this method, we do recommend configuring the bucket to use a separate subdirectory/file prefix to prevent the mixing of both log types.
We recommend configuring the subdirectories for both your CDN and WAF logs as follows:
bucketname/stackpath/cdn-access/{{sitehash}}
bucketname/stackpath/waf-access/{{sitename}}