NudgeNow is a fast-growing platform requiring robust infrastructure to handle high volumes of API traffic while ensuring security, performance, and compliance with SLA agreements. As the service scales, fine-grained rate limiting is essential to manage traffic effectively, prevent abuse, and maintain operational stability.

The client required an expressive, high-performance rate-limiting solution capable of:

• Enforcing rules based on diverse parameters such as IP address, device ID, and SDK headers.
• Ensuring near-instant application of rate-limiting policies with minimal latency impact.
• Scaling to support increasing traffic without bottlenecks.

Existing solutions like AWS ALB and CloudFront lacked the granularity and responsiveness needed. The client also required the flexibility to combine multiple rate-limiting rules dynamically.

We proposed and implemented a rate-limiting solution leveraging Envoy Proxy integrated with a central rate-limiting service and Redis. This solution offered:

1. Granular Rules: Support for hierarchical and dynamic rules, enabling combinations of parameters like IP and SDK headers.
2. Performance Optimization: Near-instant application of rules using Envoy's native gRPC integration and Redis as the backend for rule storage.

Scalability: A lightweight architecture designed to scale with traffic, ensuring high availability and low latency.

1. System Architecture

• Envoy Proxy was deployed as a sidecar for handling API traffic and performing rate limiting.
• A central rate-limiting service integrated with Redis evaluated incoming requests against pre-defined rules.
• Rules were defined using YAML descriptors, enabling clear and flexible configuration.

2. Key Features

• Expressive Rule Definitions: Hierarchical rate limits based on multiple request parameters.
• Real-Time Feedback: Headers (x-ratelimit-limit, x-ratelimit-remaining) provided clients with immediate insights into their quota usage.

Resiliency: Fallback mechanisms allowed requests to proceed without rate-limiting in case of service unavailability.

3. Testing and Benchmarking

• Used Autocannon to simulate traffic and evaluate system performance under various configurations.
• Key metrics:
• Single-param tests processed 6,000–12,000 requests per minute with average latencies of 8–16 ms.
• Multi-param configurations handled up to 53,000 requests per minute with predictable latencies and minimal resource usage (0.1 vCPU, 100MB RAM).

4. Monitoring

• Envoy Admin Interface provided real-time insights into metrics such as request rates, cluster health, and memory usage.
• Integrated with Prometheus for continuous monitoring and alerting.

1. Improved API Security: Enforced granular rate limits, mitigating risks such as DDoS attacks.
2. Scalable Performance: Supported traffic surges without compromising latency or reliability.
3. Client Feedback: Immediate feedback through response headers enabled better API consumer experience.

Performance Metrics

• Average request latency: 11–17 ms across tests.
• CPU usage: 0.5 vCPU per 1,000 requests/sec.
• Memory usage: 50 MB per 1,000 requests/sec.

Cost Efficiency

• Optimized resource allocation ensured minimal infrastructure costs while supporting high throughput.

Resiliency

• System gracefully degraded during rate-limiting service outages, allowing traffic to flow without disruptions.

1. Flexibility Matters: Combining static and dynamic rate limits provided unmatched granularity, meeting diverse business needs.
2. Monitoring Is Critical: Comprehensive monitoring ensured rapid detection of bottlenecks and anomalies.
3. Scalability by Design: The modular architecture ensured effortless scaling for future growth.

This case study highlights the successful implementation of a cutting-edge rate-limiting solution tailored to NudgeNow’s unique requirements. By leveraging Envoy Proxy, Redis, and robust testing practices, the system ensures security, performance, and future scalability, setting a new standard for API traffic management.