Help Center

  1. StackPath Help
  2. WAF
  3. WAF Features

Advanced Rules | Rate Limitation

Avatar
Joelle Lanke
Updated:

Overview

The idea of the rate limitation feature is to limit the number of specific requests within a specific period.

StackPath’s unique implementation of rate limitation is that it is a conditioning segment within a rule and it can be embedded within any other condition segments.

This guide aims to expand our existing and focus on rate limitation's advanced functionality.

Rate Limitation Method

request.rate_limit(
ip = [<string>, ...], 
url = <string>, 
time = <int>, 
requests = <int>, 
method = [<string>,...], 
status_code = [<int>,...], 
content_type = <string>, 
scope = <string>
)
When creating tag based rate limit rules use request.limit_rate instead of request.rate_limit.
request.limit_rate(
ip = [<string>, ...], 
url = <string>, 
time = <int>, 
requests = <int>, 
method = [<string>,...], 
status_code = [<int>,...], 
content_type = <string>, 
scope = <string>,
tag = <string>
)

Returns true if the count of requests(4) under the granularity(8) with the filters(1,2,5,6,7,9) exceeded the limit for a given time (3).

  Parameter Name Required Description
1 ip False

List of IP addresses that the rule applies to.

  • If there are no IPs in the list, counting will be done for all IPs.
2 url True Regex pattern to match requests' URI (URL + query_params) against.
3 time True The time limit, in seconds, within which we are only allowing n number of requests' to URI matching the pattern.
4 requests True The maximum number of requests accepted within the given period before an action is taken (minimum value 20 requests).
5 method False List of method types the request aggregation will be applied to.
6 status_code False List of status codes the request aggregation will be applied to.
7 content_type False Regex pattern to match request content_types against.
8 scope(granularity)

False - If the granularity will not be set to IP, the default aggregation will be set to cluster.

  • Scope: 'cluster' - Counts all the traffic of a POP.

    • Interval counting will be made on each POP separately. This can cause unsynchronized counting.

    • IP counting can be counted separately on different POPs (each request individually, not overlapped).

    • POPs could be related to certain locations, meaning that some clients may never be addressed to certain POPs

  • Scope: 'ip' - Counts the traffic per given IP
9 tag

False

Aggregation of tagged(User-Defined tags) requests will be applied for each IP.

Implementation Methods

Advanced Rule via API

curl --request POST \
--url https://gateway.stackpath.com/waf/v1alpha/stacks/STACK_ID/sites/SITE_ID/advanced_rules \
--header 'accept: application/json' \
--header 'content-type: application/json' \
--data '
{
"ruleAction": {"block": {"statusCode": "FORBIDDEN_403"}},
"phase": "ACCESS",
"name": "Block Scrappers",
"description": "Block IPs that hit more than 200 requests per 5 seconds for any `events` paths",
"enabled": false,
"source": "request.rate_limit([], '.*events', 5, 200, [], [], '', 'ip') and not ('mb-web-ui' in request.headers['Cookie'] or 'mb-mobile-ios' in request.headers['Cookie'] or 'mobile-android' in request.headers['Cookie'] or 'mb-mobile-android' in request.headers['Cookie'] or 'session-token' in request.headers['Cookie']) and not request.headers['session']"
}
'

Custom rule via Portal

 

Best Practices

The following examples will be shown via the Advanced Rules API syntax, as some of the complex condition functionality is not available via Custom Rules:

  1. Block certain IPs when the number of requests per interval is exceeded for any events path - each request will be counted individually for each IP, i.e. if IP '1.2.3.4' exceeds 200 requests per 5 seconds, it will be blocked, but IP '1.2.3.5' will not (if it was not exceeded too):


     
    curl --request POST \
    --url https://gateway.stackpath.com/waf/v1alpha/stacks/STACK_ID/sites/SITE_ID/advanced_rules \
    --header 'accept: application/json' \
    --header 'content-type: application/json' \
    --data '
    {
    "ruleAction": {"block": {"statusCode": "FORBIDDEN_403"}},
    "phase": "ACCESS",
    "name": "Limit Certain IPs",
    "description": "Limit Certain IPs",
    "enabled": false,
    "source": "request.rate_limit(['1.2.3.4', '1.2.3.5'], '.*events', 5, 200, [], [], '', 'ip')"
    }
    '
  2. Embedding additional conditions into the rate limit feature (Available only via API - Advanced Rule): 


     
    curl --request POST \
    --url https://gateway.stackpath.com/waf/v1alpha/stacks/STACK_ID/sites/SITE_ID/advanced_rules \
    --header 'accept: application/json' \
    --header 'content-type: application/json' \
    --data '
    {
    "ruleAction": {"block": {"statusCode": "FORBIDDEN_403"}},
    "phase": "ACCESS",
    "name": "Embedding additional condition to rate limit feature",
    "description": "Embedding additional condition to rate limit feature",
    "enabled": false,
    "source": "request.headers['User-Agent'] == 'Firefox' AND request.rate_limit(['1.2.3.4', '1.2.3.5'], '.*events', 5, 200, [], [], '', 'ip')"
    }
    '
    This can be done, but note that an additional condition such as an IP is bad practice, for example, request.ip in ['1.2.3.4'] and request.rate_limit([], '', 5, 200, [], [], '', 'ip'), since the rate limit, in this case, will count requests per interval for every IP but will set an action only when 1.2.3.4 will exceed the number of requests. This is linked to the rule you are creating and not to the rate limit condition, i.e. blank IP list and embedded IP condition, the rate limit will still count for all the IP requests.
  3. Rate limit complexed URL regex:

    curl --request POST \
    --url https://gateway.stackpath.com/waf/v1alpha/stacks/STACK_ID/sites/SITE_ID/advanced_rules \
    --header 'accept: application/json' \
    --header 'content-type: application/json' \
    --data '
    {
    "ruleAction": {"block": {"statusCode": "FORBIDDEN_403"}},
    "phase": "ACCESS",
    "name": "Rate limit complexed URL regex",
    "description": "Rate limit complexed URL regex",
    "enabled": false,
    "source": "request.rate_limit([], '.*(?<!aif|aiff|au|avi|bin|bmp|cab|carb|cct|cdf|class|css|doc|dor|dtd|exe|flv|gcf|gff|gif|grv hdmt hqx|ico|ini|jpeg|jpg|js|mov|mp3|nc|pct|pdf|png|ppc|pws|svg|swa|swf|txt|vbs|w32|wav|wbmp|wml|wmlc|wmls|wmlsc|xsd|zip|webp|jxr|hdp|wdp|webm|ogv|mp4|tif|woff|wot|woff)$', 120, 20, [], [], '', 'ip')"
    }
    '
  4. Embedding IP range to the condition (Available only via API - Advanced Rule):

    curl --request POST \
    --url https://gateway.stackpath.com/waf/v1alpha/stacks/STACK_ID/sites/SITE_ID/advanced_rules \
    --header 'accept: application/json' \
    --header 'content-type: application/json' \
    --data '
    {
    "ruleAction": {"block": {"statusCode": "FORBIDDEN_403"}},
    "phase": "ACCESS",
    "name": "Embedding IP range to the condition",
    "description": "Embedding IP range to the condition",
    "enabled": false,
    "source": "request.ip_in_range('10.0.0.0', '10.255.255.255') and request.rate_limit([], '.*[.]jpg', 120, 20, [], [], '', 'ip')"
    }
    '
  5. Cluster (PoP) granularity - Rate limit all GET or HEAD redirected (status code = 302) requests with specific content type:

     
    curl --request POST \
    --url https://gateway.stackpath.com/waf/v1alpha/stacks/STACK_ID/sites/SITE_ID/advanced_rules \
    --header 'accept: application/json' \
    --header 'content-type: application/json' \
    --data '
    {
    "ruleAction": {"block": {"statusCode": "FORBIDDEN_403"}},
    "phase": "ACCESS",
    "name": "Embedding IP range to the condition",
    "description": "Embedding IP range to the condition",
    "enabled": false,
    "source": "request.rate_limit([], '.*url', 120, 20, ['GET', 'HEAD'], [302], 'text/html; charset=[uU][tT][fF]-8', 'cluster')"
    }
    '
  6. Rate Limit by Tag Filter - This functionality will allow embedding User-Defined Tags into the rate limit condition so only tagged IPs requests will be aggregated into the rate limit mechanism:

     
    curl --request POST \
    --url https://gateway.stackpath.com/waf/v1alpha/stacks/STACK_ID/sites/SITE_ID/advanced_rules \
    --header 'accept: application/json' \
    --header 'content-type: application/json' \
    --data '
    {
    "ruleAction": {"block": {"statusCode": "FORBIDDEN_403"}},
    "phase": "ACCESS",
    "name": "Embedding tag to the condition",
    "enabled": false,
    "source": "request.limit_rate(tag="my tag",ips=["2.3.4.5"], url="/my_url/.*", time=10, requests=120, scope="ip")"
    }
    '
    my tag is a User-Defined tag that should be defined in a separate rule. For more information, please see Understanding Tag Rules.

 

Was this article helpful?

Related articles

Not finding what you need?

95% of questions can be answered using the search tool. This is the quickest way to get a response.

Contact Us