Consul
Agent traffic rate limiting
This topic provides overview information about the traffic rates limits you can configure for Consul datacenters.
Introduction
Configuring rate limits on RPC and gRPC traffic mitigates the risks to Consul servers when client agents or services send excessive read or write requests to Consul resources. A read request is defined as any request that does not modify Consul internal state. A write request is defined as any request that modifies Consul internal state. Configure read and write request limits independently.
Workflow
You may set limits on the rate of read and write requests that affect individual servers in the datacenter. You can set limits for all source IP addresses, which lets you specify a budget for read and write requests to prevent any single source IP from overwhelming the Consul server and negatively affecting the network. Enterprise users may also set up cluster-wide rate limits for Consul to apply in case of intense traffic peak periods. Cluster-wide rate limits help to avoid server malfunctions and DoS attacks in order to keep your datacenter operational.
The following steps describe the general process for setting global read and write rate limits:
Set arbitrary limits to begin understanding the upper boundary of RPC and gRPC loads in your network. Refer to Initialize rate limit settings for additional information.
Monitor the metrics and logs and readjust the initial configurations as necessary. Refer to Monitor rate limit data.
Define your final operational limits based on your observations. If you are defining server rate limits, refer to Set server traffic rate limits for additional information. For information about setting limits per source IP address, refer to Limit traffic rates for a source IP.
Define cluster-wide rate limits using lower limits compared to the ones observed in the previous steps to re-gain control over your cluster in case of heavy traffic peaks. For information about setting cluster-wide rate limits, refer to Set a cluster-wide limit on traffic rates.
Enterprise
Setting limits per source IP and global rate limits requires Consul Enterprise.
Key differences among different limits
The table shows the main differences between the different rate limits available in Consul.
| Feature | Server config | Global config entry | IP limiter |
|---|---|---|---|
| Configuration | Static (agent config) | Dynamic (config entry) | Dynamic (config entry) |
| Scope | per-server | cluster-wide | per-IP address |
| Granularity | Read/Write | Read/Write | Category specific |
| Enterprise | NO | YES | YES |
| Runtime update | Requires restart | Hot-reloadable | Hot-reloadable |
| Traffic support | Server and client | Server and client | Client only |
| Precedence | 2nd | 1st (if priority=true) | 3rd (IP global) and 4th (IP category) |
Precedence and application scenarios
The table shows the precedence applied when considering server configuration limits and global rate limits.
| Server config | global config entry | applied limit |
|---|---|---|
| configured | None or config.priority=false | server config |
| configured | config.priority=true | global entry config |
| none | None or config.priority=false | server config |
| none | config.priority=true | global entry config |
Order of operations
You can define request rate limits in the agent configuration and in the control plane request limit configuration entry. The configuration entry also supports rate limit configurations for Consul resources. Consul performs the following order of operations when determining request rate limits:
Kubernetes
To define server rate limits, configure the request_limits settings in the
Consul Helm chart. Refer to
the following resources:
The Helm chart reference for additional information.
Control plane request limit configuration entry reference for information about applying a CRD for limiting traffic rates from source IPs
Global rate limit configuration entry reference for information about applying a CRD for limiting traffic rates cluster-wide