NexusBlog

Introduction

Rate limiting is critical for protecting upstream services, preventing cascading failures, and enforcing API tier quotas. When scaling API gateways handling hundreds of thousands of concurrent requests, in-memory rate limiting on individual instances falls short because traffic is load-balanced across multiple nodes.

Core Objective

Coordinating rate limit counters across horizontally scaled NestJS instances with sub-millisecond overhead using Redis and atomic Lua scripts.

Architecture

text

1234567sequenceDiagram
    autonumber
    Client->>Gateway: HTTP GET /api/v1/resource
    Gateway->>Redis: EVALSHA sliding_window.lua (Key, Limit, Window)
    Redis-->>Gateway: [Allowed: 1, Remaining: 42, Reset: 15000]
    Gateway->>Backend: Forward Request
    Backend-->>Client: HTTP 200 OK

Distributed Clock Skew

Ensure all Redis nodes and API gateway instances synchronize via NTP. Time drift beyond 50ms will distort sliding window bucket calculation.

Benchmark Results

Sliding Window vs Token Bucket (1M ops/sec)

Benchmarked Live

Measured p99 latency under simulated 20ms network jitter.

1.Sliding Window (Lua)

0.84ms

-42% latency

2.Token Bucket (Redis)

1.12ms

-28% latency

3.Memory Footprint

64MB

O(1) memory

Implementation Code

text

1234567891011121314export async function acquireRateLimit(key: string, limit: number, windowMs: number): Promise<boolean> {
  const now = Date.now();
  const clearBefore = now - windowMs;
  
  const result = await redis.eval(
    SLIDING_WINDOW_SCRIPT,
    1,
    key,
    now,
    clearBefore,
    limit
  );
  return result === 1;
}

Designing a Distributed Rate Limiter with Redis and Lua Scripts

System Design: From Zero to Production