SPDT/sapl - sapl - Gitea: Git with a cup of tea

5862 Commits

66 Branches

269 Tags

239 MiB

Branch: feat/rate-limiter-2026

Commit Graph

Author	SHA1	Message	Date
Edward Ribeiro	2f2a976dfa	Add Redis-backed rate limiting, anonymous caching, and nginx Lua IP-block layer Implements a multi-layer DDoS/abuse mitigation stack across nginx, Django middleware, and Redis: nginx hardening (Phase 0) - GeoIP2 ASN-based bot blocking (MaxMind GeoLite2-ASN, offline build) - Request-rate and connection-rate limit zones; scanner probe zones - Gunicorn worker/timeout tuning; N+1 query fix in relatorio views - X-Accel-Redirect for private /media/ files; nginx internal media path Redis infrastructure (Phase 1) - Shared Redis pod for cross-pod state (k8s manifests: Deployment, Service, ConfigMap) - Django dual-backend cache: default (file/Redis) + dedicated `ratelimit` Redis DB 1 - configure_redis_cache() wired into settings startup; KEY_PREFIX namespace isolation RateLimitMiddleware (Phase 2) - Per-IP request rate counter (60/min sliding window) → block key in Redis DB 1 - Per-IP 404-scan counter and UA Redis deny list - NAT-aware: blocks authenticated users behind a shared blocked IP - Scoped API rate limit (60/min) with daily/weekly quota per consumer - IP-prefix blocklist (SET `rl:ip_prefix:blocked`) with 4-candidate SMEMBERS check - Blocked-IP ZSET indexes with inline pruning; atomic block writes via Lua script - Returns 403 for prefix-list hits, 429 for rate/quota hits - Django block metrics tracked per layer; CORS headers on 429 responses - Collapse rl:metrics STRING keys into a HASH per tenant per day Anonymous page caching (Phase 4) - AnonCachePageMixin applied to 8 public search/filter views (materia, sessao, pesquisar-sessao, comissoes, norma, audiencia, relatorios) - Mitigates pesquisar-sessao DDoS; sanitises page= param pollution - ETag/Last-Modified conditional request support on norma and media views nginx Lua early-rejection layer - `blocklist.lua`: per-request access_by_lua_file hook — checks shared dict for prefix matches (zero Redis I/O), then pipelines GET rl:ip:{ip}:blocked + GET rl:api:ns:{ns}:ip:{ip}:blocked; returns 429 before Gunicorn is reached - `init_worker_by_lua_block`: 60s timer refreshes `ngx.shared.ip_prefix_blocked` from Redis SMEMBERS; fail-open on Redis error - Uses `libnginx-mod-http-lua` + `libnginx-mod-http-ndk` (Debian Bookworm dynamic modules); load order in nginx.conf: NDK → GeoIP2 → Lua - `resty_redis.lua` vendored (lua-resty-redis is not in Debian Bookworm repos) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	1 month ago
Edward Ribeiro	4792c784bf	Fix read-only mount on k8s continuous-integration/drone/push Build is passing Details	11 months ago

Author

SHA1

Message

Date

Edward Ribeiro

2f2a976dfa

Add Redis-backed rate limiting, anonymous caching, and nginx Lua IP-block layer

Implements a multi-layer DDoS/abuse mitigation stack across nginx, Django
middleware, and Redis:

**nginx hardening (Phase 0)**
- GeoIP2 ASN-based bot blocking (MaxMind GeoLite2-ASN, offline build)
- Request-rate and connection-rate limit zones; scanner probe zones
- Gunicorn worker/timeout tuning; N+1 query fix in relatorio views
- X-Accel-Redirect for private /media/ files; nginx internal media path

**Redis infrastructure (Phase 1)**
- Shared Redis pod for cross-pod state (k8s manifests: Deployment, Service, ConfigMap)
- Django dual-backend cache: default (file/Redis) + dedicated `ratelimit` Redis DB 1
- configure_redis_cache() wired into settings startup; KEY_PREFIX namespace isolation

**RateLimitMiddleware (Phase 2)**
- Per-IP request rate counter (60/min sliding window) → block key in Redis DB 1
- Per-IP 404-scan counter and UA Redis deny list
- NAT-aware: blocks authenticated users behind a shared blocked IP
- Scoped API rate limit (60/min) with daily/weekly quota per consumer
- IP-prefix blocklist (SET `rl:ip_prefix:blocked`) with 4-candidate SMEMBERS check
- Blocked-IP ZSET indexes with inline pruning; atomic block writes via Lua script
- Returns 403 for prefix-list hits, 429 for rate/quota hits
- Django block metrics tracked per layer; CORS headers on 429 responses
- Collapse rl:metrics STRING keys into a HASH per tenant per day

**Anonymous page caching (Phase 4)**
- AnonCachePageMixin applied to 8 public search/filter views (materia, sessao,
  pesquisar-sessao, comissoes, norma, audiencia, relatorios)
- Mitigates pesquisar-sessao DDoS; sanitises page= param pollution
- ETag/Last-Modified conditional request support on norma and media views

**nginx Lua early-rejection layer**
- `blocklist.lua`: per-request access_by_lua_file hook — checks shared dict for
  prefix matches (zero Redis I/O), then pipelines GET rl:ip:{ip}:blocked +
  GET rl:api:ns:{ns}:ip:{ip}:blocked; returns 429 before Gunicorn is reached
- `init_worker_by_lua_block`: 60s timer refreshes `ngx.shared.ip_prefix_blocked`
  from Redis SMEMBERS; fail-open on Redis error
- Uses `libnginx-mod-http-lua` + `libnginx-mod-http-ndk` (Debian Bookworm dynamic
  modules); load order in nginx.conf: NDK → GeoIP2 → Lua
- `resty_redis.lua` vendored (lua-resty-redis is not in Debian Bookworm repos)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

1 month ago

Edward Ribeiro

4792c784bf

Fix read-only mount on k8s

11 months ago

2 Commits (feat/rate-limiter-2026)